Clinical risk assessment tools in anaesthesia

Key points.

• •Risk assessment tools aid the identification of patients at increased risk of perioperative morbidity and mortality.
• •Risk scores assign a weighting to factors identified as independent predictors of an outcome. They are simple to use.
• •Risk prediction models estimate an individual probability by entering the patient's data into the multivariable risk prediction model.
• •Postoperative morbidity is common, decreases quality of life and long-term survival, but is absent from most risk prediction models.
• •All risk assessment tools have limitations. They should be used within an overall clinical decision-making process.

Learning objectives.

By reading this article, you should be able to:

• •Describe the risk assessment tools commonly used in anaesthesia.
• •Outline the difference between risk scores and risk prediction models.
• •Discuss the benefits and limitations of using anaesthetic risk assessment tools.

Perioperative morbidity and mortality are significant health issues as they impact on patients' short- and long-term survival and resource utilisation within health services.¹ Clinical judgment alone is not a reliable predictor of an adverse outcome.² Accurate risk stratification helps identify those at increased risk of adverse perioperative events who may benefit from targeted interventions. These include preoperative optimisation, intraoperative goal-directed fluid therapy, postoperative respiratory support, and admission to critical care.² It facilitates meaningful informed patient consent and shared decision-making, empowering patients to make decisions in partnership with clinicians.²

A variety of risk assessment tools have been developed and are used in clinical practice. Risk assessment tools may be divided into risk scores and risk prediction models, both of which are normally developed using multivariable analysis of risk factors leading to a specific outcome.

Risk scores ‘assign a weighting to factors identified as independent predictors of an outcome; the weighting for each factor is often determined by the value of the regression coefficient in the multivariable analysis. The sum of the weightings in the risk score then reflects increasing risk’.³ Risk scores place patients on a scale that allows comparisons with others. They are simple to use but do not provide an individualised risk prediction of an adverse outcome.³

Risk prediction models ‘estimate an individual probability of risk for a patient by entering the patient's data into the multivariable risk prediction model’.³

Risk prediction models are more accurate in predicting an individual patient's risk than risk scores. However, they are more complex to use in routine clinical practice.³ Commonly used risk scores and risk prediction models are explored in more detail below.

Risk scores

Risk score for population-based mortality

The ASA physical status score (ASA-PS) categorises patients into six subgroups using subjective preoperative measures of physical fitness.¹ It was devised in 1941 as a statistical tool for retrospective analysis of hospital records, and has been since revised on a number of occasions.¹ In 2014, it was updated to include BMI, smoking, and alcohol intake (Table 1).⁴

Table 1.

ASA score and associated mortality (after [4] with permission from the ASA).

ASA class	Description	Examples	Mortality (%)
1	A normal healthy patient	Healthy, non-smoking, no or minimal alcohol use	0.1
2	A patient with mild systemic disease	Mild diseases only without substantive functional limitations Examples include (but are not limited to): current smoker, social alcohol drinker, pregnancy, obesity (30<BMI<40), well-controlled diabetes or hypertension, mild lung disease	0.7
3	A patient with severe systemic disease	Substantive functional limitations; one or more moderate-to-severe diseases Examples include: poorly controlled diabetes or hypertension, lung disease, BMI ≥40, alcohol dependence or abuse, implanted pacemaker, end-stage renal disease undergoing regular dialysis, history (>3 months) of MI, CVA, TIA, or CAD/stents	3.5
4	A patient with severe systemic disease that is a constant threat to life	Examples include: recent (<3 months) MI, CVA, TIA, or CAD/stents, ongoing cardiac ischaemia or severe valve dysfunction, severe reduction of ejection fraction, end-stage renal disease not undergoing regular dialysis	18.3
5	Moribund patient unlikely to survive without the operation	Examples include (but are not limited to): ruptured abdominal/thoracic aneurysm, massive trauma, intracranial bleed with mass effect, ischaemic bowel in the face of significant cardiac pathology or multiple organ/system dysfunction	93.3
6	A declared brain-dead patient whose organs are being removed for donor purposes

CAD, coronary arterial disease; CVA, cerebrovascular accident; MI, myocardial infarction; TIA, transient ischaemic attack.

ASA-PS score correlates with outcome in a number of different clinical settings.¹ Underlying fitness is an important predictor of survival after surgery; a high ASA score is predictive of both increased postoperative complications and mortality after non-cardiac surgery.⁵

ASA-PS is easy to use and understand. It is widely used as a component of preoperative assessment.¹ However, it does not take into account any preoperative optimisation of the patient, the planned surgery, or the level of postoperative care.¹

ASA scoring is subjective, so may lead to significant inter-operator variability. Despite this, and the existence of more complex scoring systems with greater prognostic accuracy, ASA scoring still remains useful in conveying risks of anaesthesia and surgery.⁵

Risk score for cardiac complications

Lee's Revised Cardiac Risk Index (RCRI) is the most commonly used risk score for the development of cardiac complications after major non-cardiac operations. Six independent predictors of complications were identified and included: (i) high-risk surgery (intraperitoneal, intrathoracic, or suprainguinal vascular procedures); (ii) ischaemic heart disease; (iii) history of congestive heart failure; (iv) cerebrovascular disease (transient ischaemic attack or cerebrovascular accident); (v) insulin therapy for diabetes mellitus; and (vi) preoperative creatinine concentration >176 μmol l⁻¹ (Table 2).⁶

Table 2.

Lee's revised cardiac risk index and the risk of cardiac complications (reproduced with permission from the publishers of Circulation⁶).

Points	Lee class	Risk (%)
0	1	0.4
1	2	0.9
2	3	6.6
>3	4	11

For each predictor, one point is added to the final score, which then determines the Lee class and predicted incidence of major cardiac complications. These include myocardial infarction, pulmonary oedema, ventricular fibrillation or primary cardiac arrest, and complete heart block.⁶

Lee and colleagues used a prospective cohort study of 4315 patients in a tertiary hospital to develop and validate an updated index for risk of cardiac complications in 1999.⁶ This updated the work of Goldman and colleagues,⁷ who had identified nine independent risk factors for the development of life-threatening and fatal cardiac complications in a 1977 study of 1001 patients more than 40 yrs of age.

The RCRI is a simple, quick, and non-invasive method for identifying those at increased risk of perioperative cardiac complications. It can be used to stratify patients who may warrant further investigation, and those who do not.

The limitations in using RCRI were noted by the original authors. The data are from a single teaching hospital in patients undergoing non-emergent operations.⁶ It is therefore not generalisable to lower-risk populations, such as patients having only minor procedures, or in high-risk populations or those undergoing emergency surgery.⁶ Finally, the original data were collected 20 yrs ago and therefore may not reflect current patients presenting for surgery.

Risk score for postoperative pulmonary complications

Assess respiratory risk in surgical patients in Catalonia (ARISCAT) is the most widely used risk score for predicting postoperative pulmonary complications (PPCs). A seven-variable regression model stratifies patients into low-, intermediate-, and high-risk groups for the development of a PPC (Table 3).⁸

Table 3.

Ariscat score table (after [8] with permission from the publishers of Anesthesiology). Low risk=<26 points; intermediate risk=26–44 points; high risk=>45 points.

	Multivariate analysis OR (95% CI)	β coefficient	Risk score
Age (yrs)
<50	1		0
51–80	1.4 (0.6–3.3)	0.331	3
>80	5.1 (1.9–13.3)	1.619	16
Preoperative SpO2
>96%	1		0
91–95%	2.2 (1.2–4.2)	0.802	8
<90%	10.7 (4.1–28.1)	2.375	24
Respiratory infection in the last month
No	1		0
Yes	5.5 (2.6–11.5)	1.698	17
Preoperative anaemia (Hb<10 g dl−1)
No	1		0
Yes	3.0 (1.4–6.5)	1.105	11
Surgical incision
Peripheral	1		0
Upper abdominal	4.4 (2.3–8.5)	1.480	15
Intrathoracic	11.4 (4.9–26.0)	2.431	24
Duration of surgery
<2 h	1		0
2–3 h	4.9 (2.4–10.1)	1.593	16
>3 h	9.7 (4.7–19.0)	2.268	23
Emergency procedure
No	1		0
Yes	2.2 (1.0–4.5)	0.768	8

CI, confidence interval; OR, odds ratio.

Mortality is increased in those developing a PPC, as is morbidity and hospital length of stay.⁸ Around a fifth of patients (14–30%) who develop a PPC will die within 30 days of major surgery compared with 0.2–3% of those with no PPC.⁸ ARISCAT was developed from a prospective, multicenter, observational study of 2464 patients undergoing non-obstetric, in-hospital surgical procedures with general, neuraxial, or regional anesthesia.⁹

ARISCAT used the 2015 European Joint Task Force published guidelines for perioperative clinical outcome definitions for PPCs. It considered respiratory infection, respiratory failure, pleural effusion, atelectasis, pneumothorax, bronchospasm, and aspiration pneumonitis to be the composite measures and defined pneumonia, acute respiratory distress syndrome (ARDS), and pulmonary embolus as individual adverse outcomes.⁸

Using the ARISCAT risk score helps identify the patient groups susceptible to PPCs. In an elective setting it can be used to aid surgical timing, as it contains modifiable factors such as anaemia and respiratory infection within the past month. The disadvantages of using the ARISCAT score relate to how PPCs are defined. There is considerable heterogeneity of definitions used between studies, and most contain subjective elements. These factors limit its routine clinical use.⁸

Risk prediction models

Physiological and Operative Severity Score for the enUmeration of Mortality and Morbidity and Portsmouth variant

Physiological and Operative Severity Score for the enUmeration of Mortality and Morbidity (POSSUM) and Portsmouth POSSUM (P-POSSUM) use 12 physiological variables and six surgical variables to calculate 30-day mortality after surgery (Table 4).

Table 4.

P-POSSUM variables.

Physiological variables	Operative variables
Age Cardiac signs Respiratory history/chest X-ray findings Systolic BP Heart rate Glasgow coma score Haemoglobin White cell count Urea Sodium Potassium ECG	Operative severity Multiple procedures Total blood loss Peritoneal soiling Presence of malignancy Urgency of surgery

POSSUM was developed in 1991 as a scoring system for surgical audit in an English district general hospital. It included emergency and elective procedures for patients undergoing gastrointestinal, hepatobiliary, urological, and vascular surgery. Multivariate logistic regression of physiological, operative, and postoperative variables was performed for the prediction of the 30-day morbidity and mortality rates.¹

P-POSSUM is a newer risk model that was developed in Portsmouth, UK using alternative risk equations, but the same physiological and surgical variables as POSSUM. It was validated in a large cohort in a single centre.¹ Because of the original authors' lack of confidence in the reporting of perioperative complications, P-POSSUM has no morbidity prediction equation.¹ P-POSSUM has the advantage of being comprehensive and well validated.¹ It includes data from elective and emergency procedures. It considers the patient's physiological status, incorporating cardiorespiratory and neurological status at the time of the procedure, along with surgical findings.

Whilst helping to provide a comprehensive picture, the operative findings limit its value in preoperative decision-making. A number of the physiological variables contain subjective elements that may introduce error into the model. It has also been shown to overestimate risk in some groups (see below). Further surgery-specific versions of POSSUM have also been developed including colorectal, oesophagogastric, thoracic, and vascular scores.

National emergency laparotomy audit calculator

The National emergency laparotomy audit (NELA) calculator estimates 30-day mortality after emergency bowel surgery. NELA was commissioned after evidence of a high incidence of death for patients undergoing emergency laparotomy in hospitals across England and Wales, and wide variations in the provision of care.¹⁰ Through the provision of high quality comparative data from all emergency laparotomy providers, NELA aims to improve the quality of care for patients undergoing emergency laparotomy.¹⁰

P-POSSUM was used to predict mortality for NELA patients. Linkage of NELA data to Office of National Statistics mortality data revealed that P-POSSUM overestimated mortality when the predicted risk was greater than 15%.¹¹ This led to the development of a NELA-specific risk calculator, using data gathered from more than 38,000 patients in the first 2 yrs of NELA.¹² Both P-POSSUM and NELA calculators are available on the NELA smartphone app. The new calculator has the advantage of being accurate and specific for emergency laparotomies. It requires much the same data to be inputted as per P-POSSUM, and has the same disadvantages—it is time consuming, requires estimates of intraoperative findings, and has no morbidity component.

Surgical outcome risk tool

The surgical outcome risk tool (SORT) uses six preoperative variables to predict 30-day mortality in non-cardiac, non-neurological inpatient surgery. The six variables are: (i) ASA-PS; (ii) urgency of surgery; (iii) surgical specialty; (iv) severity of surgery; (v) cancer; and (vi) age.²

SORT was developed after the 2011 publication of the National Confidential Enquiry into Patient Outcome and Death (NCEPOD) ‘Knowing the Risk’ report that assessed perioperative care.¹³ A key recommendation was that a mortality risk assessment should be made explicit to patients before surgery, and documented on the consent form.² Post hoc analysis of the ‘Knowing the Risk’ data set was conducted, with data from 16,788 patients being analysed to develop the SORT risk prediction calculator.²

SORT is a simple to use risk prediction model containing readily collected preoperative data. It includes data from emergency and elective cases across a wide range of surgical specialties, and was generated from a broad representation of size and types of hospital across England, Wales, and Northern Ireland.² It is available as an online calculator (www.sortsurgery.com) and as a smartphone app. However, it has a relatively limited scope in focusing only on 30-day mortality, having no morbidity component. The model development did not include day case, obstetric, neurosurgery, cardiac or transplant surgery. BMI and haemoglobin and creatinine concentrations were also not included in the analysis because of missing data, and these factors can influence outcome.

American College of Surgeons national surgical quality improvement project universal surgical risk calculator

The American College of Surgeons national surgical quality improvement project (ACS NSQIP) universal surgical risk calculator uses 21 preoperative factors to predict 14 outcomes within 30 days of surgery: mortality, serious complication, any complication, pneumonia, cardiac complication, surgical site infection, urinary tract infection, venous thromboembolism/blood clot, renal failure, ileus, anastomotic leak, readmission, return to operating room, and discharge to post-acute care facility (Fig. 1).¹⁴

Fig 1.

American College of Surgeons National Surgical Quality Improvement Programme online surgical risk calculator. (A) Data input form showing the required variables. (B) Results screen showing the 14 predicted outcomes, with the colour indicating whether these are below, at, or above the average for the whole dataset. Reproduced by permission of the American College of Surgeons.

The ACS NSQIP collects data on more than 150 variables, including preoperative risk factors, intraoperative variables, and 30-day postoperative mortality and morbidity outcomes for patients undergoing major surgical procedures in both the inpatient and outpatient setting in the private sector within the USA.¹⁴

The ACS NSQIP universal surgical risk calculator was developed in 2013 using data from 1.4 million patients from 393 ACS NSQIP hospitals.¹⁵ It aims to provide accurate, patient-specific risk information to guide both surgical decision-making and informed consent. It was further updated in 2016 to include additional complications and was recalibrated to ensure ongoing accuracy.¹⁶

In many ways the ACS NSQIP calculator approaches the ideal. It uses only preoperative variables to give comprehensive patient and procedure-specific morbidity and mortality risks. It gives valuable information to patients before operation to aid their decision-making. It can be accessed at https://riskcalculator.facs.org/RiskCalculator/index.jsp.

The comprehensive nature of the calculator comes at a cost. It is time consuming and expensive. It requires dedicated data collectors known as surgical clinical reviewers, who capture the data using a variety of methods including medical chart abstraction.¹⁴ The data are derived from the private sector within the USA. This raises questions regarding comparability to other countries, especially with regards to patient characteristics, perioperative practice, and standards of care.

Cardiac surgery risk prediction models

Risk prediction models are also used to predict outcomes after cardiac surgery. A variety of tools exist, including the European system for cardiac operative risk evaluation (EuroSCORE), the Society of Thoracic Surgeons algorithms, and the Parsonnet score.¹⁷ Of these, EuroSCORE is the most widely studied and utilised.¹⁷ The full details of these models is beyond the scope of this article, and these have been reviewed in an earlier article.¹⁸

Cardiac risk prediction models focus on preoperative predictions of mortality. However, the ability to predict only operative mortality is not an adequate method of determining surgical outcome, as morbidity is much more common in this patient cohort.¹⁷ This is in keeping with many of the risk prediction models explored above.

Summary

All risk assessment tools have limitations; no perfect score (Table 5) or model (Table 6) exists. Many contain subjective elements, and there is a fine balance between ease of use, accessibility, and accuracy. Risk assessment tools containing only preoperative variables are easier to use and can be used in the surgical decision-making process, but often have no morbidity component. Risk prediction models give patient-specific risk estimates but are more complex to use.

Table 5.

Summary of risk scores.

Risk score	Outcome	Advantages	Disadvantages
ASA	Population-based mortality	Simple to use Correlates with outcome in number of settings	Subjective Does not consider surgery planned
Lee's RCRI	Postoperative cardiac complications (non-cardiac surgery)	Simple to use Non-invasive Identifies those at increased risk before operation	Developed 20 yrs ago in a single centre from elective patients Uncertain generalisability
ARISCAT	Postoperative pulmonary complications	Identifies those at increased risk, contains modifiable features	No widespread agreement in how PPCs are defined

Table 6.

Summary of risk prediction models.

Risk prediction model	Predicted outcome	Advantages	Disadvantages
P-POSSUM	30-day mortality	Comprehensive—considers patient and surgical factors	Requires estimate of intraoperative findings Contains subjective elements Overestimates mortality in high risk groups
NELA	30-day mortality	Accurate and specific for emergency laparotomy patients	Time consuming Requires estimate of intraoperative findings No morbidity component
SORT	30-day mortality	Simple to use Readily available preoperative data Generated from emergency and elective patients across a wide range of surgical specialties	No morbidity component
ACS NSQIP	30-day mortality 30-day morbidity Return to theatre Readmission Discharge to post acute care facility	Comprehensive patient and procedure specific morbidity and mortality risks	Time consuming Expensive Data from private hospitals in USA—uncertain generalisability

Risk assessment tools should always be used as part of a clinical decision-making process with the patient. No single risk tool can contain all relevant clinical information. However, the importance of accurate risk assessment was highlighted in the third NELA report. The 30- and 90-day mortality of patients without a formal documented risk assessment was equal to those documented as being high risk (7.1% and 10.6%, respectively, vs 7.8% and 12.7%, respectively).¹⁹ Where risks had been documented, patients were more likely to receive subsequent care that met standards. As patient characteristics and associated comorbidities evolve over time, along with enhanced surgical techniques and standards of care, risk assessment tools also need periodic recalibration to ensure ongoing accuracy.

Many of the risk prediction models analysed use 30-day mortality as their end point. Within perioperative medicine there is a move away from focusing on mortality within a limited timeframe and a greater emphasis on morbidity. There are a number of reasons for this. Postoperative morbidity is much more common, and has a large impact on patients' quality of life and their ability to return to full function. The development of postoperative complications has also been shown to decrease long-term survival.²⁰ Incorporating patient-related outcome measures in the future development of risk assessment tools will be vital in providing high quality, relevant information to patients.

Declaration of interest

The authors declare that they have no conflicts of interest.

MCQs

The associated MCQs (to support CME/CPD activity) will be accessible at www.bjaed.org/cme/home by subscribers to BJA Education.

Biographies

John Stones BSc, FRCA is a final year specialty registrar completing advanced training in perioperative medicine and vascular anaesthesia. His clinical interests include anaesthesia for major gastrointestinal and high-risk surgery and quality improvement.

Dave Yates FRCA is a consultant in anaesthesia and critical care medicine at York Hospital. His clinical work involves anaesthesia for major general and vascular surgery, perioperative, and intensive care medicine. He has recently set up one of the UK's first perioperative medicine services. His research interests include prehabilitation for high-risk surgical patients and improving perioperative outcomes using protocolled care.

Matrix codes: 1H02; 2A03; 3J02