ASA Physical Status Determinatison by General Internists and Impact on Cardiac Risk Assessment

Abstract

Objectives:

Estimating cardiac risk is important for preoperative evaluation, and several risk calculators incorporate the American Society of Anesthesiologists (ASA) physical status score. The purpose of this study was to determine the concordance of ASA scores assigned by general internists and anesthesiologists and assess whether discrepancies affected cardiac risk estimation.

Methods:

This observational study included military veterans evaluated in a preoperative evaluation clinic at a single center during a 12-month period. ASA scores were recorded by General Internal Medicine residents under the supervision of a General Internal Medicine attending, performing a preoperative medical consultation, and were compared with ASA scores assigned by an anesthesiologist on the day of surgery. ASA scores and Gupta Cardiac Risk Scores incorporating each ASA score were compared.

Results:

Data were collected on 206 patients, 163 of whom had surgery within 90 days and were included. ASA scores were concordant in 60 (37.3%), whereas the ASA scores were rated lower by the general internist in 101 (62.0%) and higher in 2 (1.2%). Interrater reliability was low (κ = 0.08), and general internist scores were significantly lower than anesthesiologist scores (P < 0.01). Gupta Cardiac Risk Scores were calculated for 160 patients, and they exceeded 1% in 14 patients using the anesthesiologist ASA score, compared with 5 patients using the general internist score.

Conclusions:

ASA scores assigned by general internists in this study were significantly lower than those assigned by anesthesiologists, and these discrepancies in the ASA score can lead to substantially different conclusions about cardiac risk.

The American Society of Anesthesiologists (ASA) Physical Status Classification System was developed in the 1940s as a means of objectively categorizing patient-specific operative risk, independent of procedure-specific risk.^1,2 The ASA score has stood the test of time. It remains widely used today and has been shown repeatedly to be predictive of postoperative complications,^3–5 including major adverse cardiovascular events (MACE).^6–8 The ASA score has become an important component in preoperatively estimating cardiac risk, and it has been included as a predictor variable in two widely used algorithms: the Gupta Cardiac Risk Score⁹ and the American College of Surgeons National Surgical Quality Improvement Program⁸ cardiac risk estimation tools. These tools can be useful for delivering guideline-based preoperative care. For example, guidelines issued by the American College of Cardiology and the American Heart Association recommend that preoperative cardiac stress testing can be safely omitted for most patients with a predicted postoperative risk for MACE of <1%.¹⁰

The usefulness of preoperative risk estimation tools depends in large part on the ability of clinicians involved in preoperative assessment, including general internists, to accurately assign ASA scores. Prior studies, however, have raised significant questions about the reliability of ASA scores, even among anesthesiologists.^11,12 Concerns also have been raised about the accuracy of ASA scores assigned by nonanesthesiologists.^13,14 For example, most General Internal Medicine physicians do not use the ASA score in their typical practice and do not receive much instruction or experience in assigning ASA scores during training. The reliability of ASA scores assigned by general internists has not been well studied because there have been few comparisons of ASA scores assigned by anesthesiologists and nonanesthesiologists.^13,15 To our knowledge, no studies have specifically examined whether the ASA assignment by different provider types has an effect on preoperative cardiac risk estimation. The purpose of this study was to compare ASA scores assigned by anesthesiologists and general internists in actual clinical practice and to determine whether differences in ASA score assignments translated to differences in preoperative cardiac risk estimation.

Methods

Study Design, Setting, and Population

Between April 2017 and April 2018, we collected data on a convenience sample of patients undergoing preoperative medical consultations at a single Veterans Affairs Medical Center affiliated with an academic institution. Consults were not required by protocol, but they were available at the request of the referring surgeon or the nurse completing a preanesthesia evaluation. Consults were performed in a general internal medicine clinic by upper-level (second or third year) categorical Internal Medicine residents under the supervision of a board-certified Internal Medicine attending. This study was determined to be quality improvement by the institutional review board at the Birmingham VA Medical Center and therefore not human subjects research. Oversight was thus the responsibility of the quality director, who reviewed and approved all of the aspects of the study. The requirement for individual patient consent was waived.

Data Sources and Study Variables

Data were prospectively collected at the time of consultation on a standardized form, so physicians who filled out the form were aware that they were participating in a quality improvement project. Elements collected included the type of operation to be performed, ASA score (options ranged from 1 to 4), functional status (independent, partially dependent, or dependent), Revised Cardiac Risk Index (RCRI) measures, and estimated exercise capacity. No specific instruction as to how to assign ASA scores was provided, but a sheet with the ASA score definitions and examples provided by the ASA was available as a printout in the workroom, although we did not attempt to track how often this information was used. Similarly, exercise capacity was not based on formal assessment (eg, Duke Activity Status Indicator), but rather informal metrics cited in guidelines (eg, climbing a flight of stairs, walking up a hill signifying >4 metabolic equivalents).¹⁰

At the end of the study, we retrospectively extracted several elements from the electronic medical record. We reviewed the chart to determine whether surgery occurred at any time in the 90 days after the preoperative consultation. If a surgery occurred, then we reviewed anesthesiology provider notes from the day of surgery to determine the ASA score they assigned. Typically, two notes were written by anesthesiology providers from the day of surgery that included an ASA score: one preoperative note from the attending anesthesiologist, and one intraoperative note that was typically completed by an anesthesiology resident or a nurse anesthetist. We recorded both, but considered the ASA score by the attending anesthesiologist to be primary for this study. We also recorded the ASA score from the preoperative anesthesia clinic, where evaluations are conducted with registered nurses from the anesthesia department. Finally, we retrieved the type of surgery, age on the day of surgery, sex, and most recent serum creatinine before surgery for each patient.

Statistical Analysis

The primary outcome was the concordance of ASA scores assigned by the general internists and the attending anesthesiologists. We also calculated the weighted Cohen κ,¹⁶ which is a measure of interrater reliability (IRR) that accounts for chance agreement on a scale of −1 to 1, with 0 representing agreement expected by chance and 1 representing perfect agreement. To determine whether there were statistically significant differences between the ASA assignments between the two groups, we measured the symmetry of the contingency table using the McNemar-Bowker test.¹⁷ We repeated these comparisons for the ASA scores assigned by the attending anesthesiologists with those assigned by the preoperative anesthesia clinic and by the other anesthesiology provider from the day of surgery. Finally, we calculated the Gupta Cardiac Risk Score using the ASA scores assigned by the general internists and the attending anesthesiologists, compared the means of each score using the t test, and compared the proportion of patients with risk scores >1% using each score. All of the calculations were performed using STATA version 15 (StataCorp, College Station, TX).

Results

A total of 206 patients were evaluated in the preoperative medical clinic during the time period, and 165 went on to undergo an operation in the next 90 days. Two additional patients were excluded for missing internal medicine ASA scores, leaving 163 in the final sample for comparing ASA scores. The characteristics of those included are shown in Table 1. The average age was 66.6, 93.9% were male, and 101 underwent orthopedic surgery procedures. Included patients were evaluated in the internal medicine clinic a mean of 26.8 days before their surgery (range 1–81).

Table 1.

Baseline patient characteristics, N = 163

Characteristic	N (%)
Age, y, mean (SD)	66.6 (8.5)
Male sex	153 (93.9)
Operation type
Orthopedics	101 (62.0)
Urology	38 (23.3)
All others	24 (14.7)
RCRI score
0	103 (63.2)
1	42 (25.8)
≥2	18 (11.0)
RCRI factors
Cerebrovascular disease	13 (8.0)
Coronary artery disease	38 (23.3)
Congestive heart failure	13 (8.0)
Diabetes, requiring insulin	17 (10.4)
Serum creatinine >2 mg/dL	3 (1.8)
High-risk operation	2 (1.2)
Exercise capacity, N = 158
<4 METS or unknown	28 (17.7)
4–10 METS	100 (63.3)
≥10 METS	30 (19.0)
Functional status, N = 162
Independent	149 (92.0)
Partially dependent	11 (6.8)
Dependent	2 (1.2)
Most recent serum creatinine, mg/dL
<1.5	154 (94.5)
>1.5	9 (5.5)

METS, metabolic equivalents; RCRI, Revised Cardiac Risk Index; SD, standard deviation.

The ASA scores as determined by general internists and attending anesthesiologists are shown in Table 2. The scores were concordant for 60 patients (36.8%). The general internist score was lower than the anesthesiologist score for 101 (62.0%) and higher for two (1.2%). The Internal Medicine score was two categories lower for 11 (6.7%). The weighted Cohen κ was 0.08 and the McNemar-Bowker test χ² was 95.21 (P < 0.01).

Table 2.

ASA scores assigned by general internists and anesthesiologists

Anesthesiologist ASA scores	General internist ASA scores				Total
	ASA-1	ASA-2	ASA-3	ASA-4
ASA-1	0	0	0	0	0
ASA-2	6	10	2	0	18
ASA-3	9	73	50	0	132
ASA-4	0	2	11	0	13
Total	15	85	63	0	163

ASA, American Society of Anesthesiologists.

Comparisons of the attending anesthesiologist and other anesthesiology provider ASA scores are shown in Table 3. Comparing the attending anesthesiologist with the other day of surgery anesthesia provider, scores were concordant for 145 patients (89.0%). The weighted Cohen κ was 0.64 and the McNemar-Bowker test χ² was 1.53 (P = 0.67). Comparing the attending anesthesiologist with the preoperative anesthesia clinic nurse, scores were concordant for 139 patients (85.3%). The weighted Cohen κ was 0.61 and the McNemar-Bowker test χ² was 2.25 (P = 0.34).

Table 3.

Concordance of ASA scores among providers

	Attending anesthesiologist vs general internist	Attending anesthesiologist vs other day of surgery anesthesia provider (CRNA or resident)	Attending anesthesiologist vs preoperative anesthesia clinic (RN)
Concordant scores, %	36.8	89.0	85.3
Weighted κ	0.08	0.64	0.61
McNemar-Bowker test, χ2 (P value)	95.21 (<0.01)	1.53 (0.67)	2.25 (0.34)

ASA, American Society of Anesthesiologists; CRNA, certified registered nurse anesthetist; RN, registered nurse.

The Gupta Cardiac Risk Score was able to be calculated in 160 patients (2 patients underwent eye surgery, which is not an included surgery type, and 1 was missing functional status). The mean Gupta risk was 0.60% (standard deviation 0.51) using the attending anesthesiologist ASA score and 0.30% (standard deviation 0.34) using the general internist ASA score (P < 0.01). The Gupta risk was >1% for 14 patients (8.8%) using the attending anesthesiologist ASA score and 5 patients (3.1%) using the general internist ASA score.

Discussion

In this study of military veterans evaluated in a preoperative medical consultation clinic before a variety of surgical procedures, we found that general internists rated patients’ ASA scores significantly lower than the anesthesiologists did on the day of surgery. This lower ASA score has the potential to affect preoperative management, as reflected by fewer patients having predicted cardiac risk of >1%, a threshold cited in current US guidelines on preoperative cardiac evaluation.¹⁰

The IRR of ASA scores between general internists and anesthesiologists in our study (κ = 0.08) rates as “slight” using a widely cited classification system,¹⁸ and is lower than κ scores identified in previous studies. A systematic review¹⁹ published in 2016 identified 13 studies, 8 of which reported IRR of ASA scores using a κ statistic: 2 had fair IRR (κ range 0.21–0.40), 3 had moderate reliability (κ range 0.47–0.53), and 3 had good reliability (κ range 0.61–0.82). Although that review did not specify the training background of raters in each of the included studies, it appears that only two included nonanesthesiologists (one included nurses who routinely assigned ASA scores for a registry of trauma victims²⁰ and one included nurses who routinely evaluated patients in an anesthesia preoperative evaluation clinic),²¹ which may explain why our study found lower IRR. In a more recent study of 101 patients undergoing total joint arthroplasty, however, Knuf et al¹⁵ compared ASA scores assigned in a medicine preoperative clinic, a preoperative anesthetic unit, and on the day of surgery by the anesthesia provider conducting the anesthetic care. They found low IRR between the medicine clinic and day of surgery scores (κ= 0.156), with the medicine clinic scores significantly lower (McNemar P = 0.025), whereas IRR between the preoperative anesthetic unit and day of surgery provider was high (κ = 0.863), which is generally consistent with our findings.

Our finding that the differences in ASA scores translated to different cardiac risk predictions should be interpreted with some caution given our relatively small sample size. Our results do not provide a precise estimate of the proportion of patients who would likely receive additional cardiac testing as a result of the ASA score discrepancies, but rather illustrate the simple point that an ASA discrepancy of even a single point could lead to different care for otherwise similar patients. For example, a totally independent 70-year-old individual with serum creatinine ≥1.5 mg/dL undergoing an orthopedic procedure would have a predicted Gupta Cardiac Risk Score of <1% with an ASA score of 2 (0.32%), but >1% with an ASA score of 3 (1.26%). Given the risk threshold approach of current US guidelines,¹⁰ this could determine whether a patient undergoes a preoperative cardiac stress test.

Overall, our findings have several potential implications. First, our results showing that general interests systematically underestimate ASA is an additional argument in favor of anesthesiologists serving a larger role in preoperative evaluation and risk assessment. Preoperative evaluation clinics are increasingly being run by anesthesia departments, which has numerous potential benefits, including reducing operating room delays and cancellations²² and reducing unnecessary preoperative testing.²³

Second, if general internists are to remain involved in preoperative evaluation, then additional interventions could be considered to increase their accuracy in assigning ASA scores. Internal Medicine residency programs could include more time in training internists to assess ASA scores. Alternatively, exposure to ASA-approved examples of correct ASA scoring has been shown to increase the proportion of correct assignments, particularly among nonanesthesiologists.¹³

Finally, the present approach to preoperative cardiac care could be reconsidered. A strict risk-threshold approach to preoperative management, especially when it relies on risk estimation systems that incorporate relatively coarse categories and subjective inputs such as the ASA score, will likely result in a significant variation of the final output (ie, whether a patient undergoes a preoperative cardiac stress test). Alternatively, more objective risk scoring systems could be used (eg, the RCRI, which admittedly has its own limitations),⁹ or as the Canadian guidelines recommend, biomarker-based preoperative risk stratification using brain natriuretic peptide.²⁴ For example, in our study, a similar proportion of patients had elevated risk (ie, risk for MACE >1%)²⁵ using a cutoff of ≥2 RCRI risk factors (11.0%) the Gupta Cardiac Risk Score calculated with the anesthesiologist ASA score (8.8%).

This study has several strengths. First, whereas most studies assessing the reliability of ASA scores by multiple raters have relied on hypothetical case vignettes, our study included the evaluation of actual patients.¹⁹ Second, we included patients across a variety of surgeries, which improves generalizability. Finally, although the variability of cardiac risk estimates using different methods has been evaluated,²⁶ our study is the first of which we are aware that specifically examines how variations in ASA lead to variations in cardiac risk estimation.

This study also has several limitations. First, as a single-site study that included military veterans exclusively, our findings may not be generalizable to other locations or settings. Second, Internal Medicine residents were involved in the clinical care in the preoperative medicine clinic, which could result in less accurate ASA score assignment and therefore lower IRR when compared with the ASA scores assigned by anesthesiologists. Internal Medicine attendings, however, were involved in the care of each of these patients, although they also may have little training and experience assigning ASA scores. Third, because ASA scores were assigned by Internal Medicine attendings and anesthesiologists on different days, it is possible that medical conditions changed during that period. More likely, different contextual factors (ie, seeing a patient in street clothes in an outpatient clinic vs wearing a hospital gown and on a gurney in a preoperative holding area) affected this assessment. Fourth, the Gupta Cardiac Risk Score contains another subjective measure—the functional status; however, because this is not routinely recorded in the medical record, we were unable to account for whether general internists and anesthesiologists differ in their assessment of functional status. Finally, this study is not able to identify why scores differed or to what degree the anesthesiologists’ ASA score more accurately reflected the patients’ actual physical status.

Conclusions

We found general internists performing a preoperative medical consultation rated ASA scores significantly lower than attending anesthesiologists on the day of surgery, which led to discrepancies in predicted cardiac risk. The low concordance of scores is concerning and has potential implications for preoperative evaluation. If general internists are to remain involved in preoperative evaluation, then efforts may be needed to increase their accuracy in assigning ASA scores, although arguably, anesthesiologists should continue increasing their primary role of preoperative evaluation and risk assessment.

Key Points.

• Interrater reliability was low (κ = 0.08) for American Society of Anesthesiologists (ASA) scores assigned by general internists and anesthesiologists.

• ASA scores assigned by general internists were significantly lower than ASA scores assigned by anesthesiologists (P < 0.01).

• Discrepancies in the ASA score can lead to clinically meaningful differences in predicted cardiac risk.