Abstract
Background:
The Agency for Healthcare Research and Quality Patient Safety Indicators (PSI) are quality improvement indicators used to determine hospital performance and, increasingly, to rank surgical programs. The American College of Surgeons National Surgical Quality Improvement Program and the Society for Vascular Surgery Vascular Quality Improvement databases are also frequently used to compare outcomes, but definitions of complications vary between the systems and the optimal system for tracking complications in complex endovascular repair remains unclear. Herein we assess the three outcome tracking systems and their ability to capture complications after fenestrated endovascular abdominal aortic aneurysm repair (FEVAR) and open aortic aneurysm repair in a large complex aortic program.
Methods:
Demographic and operative data for patients undergoing repair of juxtarenal or pararenal aortic aneurysms between 2004 and 2018 via both open and FEVAR approaches at the Johns Hopkins Medical Institutions were compiled in a prospectively maintained retrospective database. Postoperative complications were defined according to a surgeon-defined system, the Society for Vascular Surgery Vascular Quality Initiative, the American College of Surgeons National Surgical Quality Improvement Program, and the Agency for Healthcare Research and Quality PSI data dictionaries and were compared between surgical approaches as well as eras before and after the introduction of FEVAR. Complication rates between the classification systems were compared using proportion testing and the strength of the correlation between the systems was evaluated with Spearman’s rank test.
Results:
Of 145 patients, 60 (41.4%) underwent FEVAR and 85 (58.6%) underwent open aortic aneurysm repair. The introduction of fenestrated technology was associated with a decrease in the overall number of complications from 37.2% to 20.6% by surgeon-defined classification system (P = .036). The VQI identified the most complications (39.9% of the entire cohort and 25% of FEVAR cases), followed by the NSQIP (29.0% and 33.3%, respectively) and PSI (4.1% and 5%). The two clinically focused databases were found to correlate well with a surgeon-designed classification system, as well as each other (Spearman ρ ≥ 0.735) but not with PSI (ρ < 0.23). Proportion testing demonstrated the rate of complications identified by PSI to be significantly less than either VQI or NSQIP (P < .001). Specifically, PSI did not effectively identify renal complications (1.4% vs 9.0% by NSQIP and 27.3% by VQI definitions; P < .001).
Conclusions:
The introduction of FEVAR is associated with an overall decrease in complications in this study. The clinically relevant VQI and NSQIP databases show good concordance in capturing complications; however, PSI did not correlate with either and captured significantly fewer complications. These data highlight the value of high scrutiny classification systems to track postoperative complications and suggest that PSI are insufficient to rank complex aortic programs with high levels of FEVAR use.
Keywords: Endovascular repair, Fenestrated endovascular aortic repair, Abdominal aortic aneurysm, Patient Safety Indicator
The introduction of endovascular technology has revolutionized aortic repair. Endovascular stent grafting (EVAR) has become the standard of care for both elective and emergent aneurysm repair,1 and the introduction of fenestrated and branched technology (fenestrated endovascular abdominal aortic aneurysm repair [FEVAR]) has extended the benefits of endovascular surgery to patients with increasingly anatomically complex aneurysms involving the renal and visceral arteries.2,3 It is now well-documented that the endovascular approach is associated with reductions in morbidity and mortality,2,4,5 but in the long term these stent grafts require close screening and often require ancillary procedures to address endoleaks and other device failures.4,6
Unlike open aortic repair (OAR), a strong volume-outcomes relationship in EVAR has not been established7 or investigated in complex EVAR, because these procedures have to this point been performed at specialized centers.8 There is, therefore, ongoing interest in identifying modifiable factors that impact the initial success and durability of these procedures as they increase in prevalence.7
The Society for Vascular Surgery Vascular Quality Initiative (SVS-VQI)9 and American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP)10 are largely used as quality improvement rather than performance metrics. In contrast, the Agency for Healthcare Research and Quality Patient Safety Indicators (AHRQ-PSI)11 are a collection of short-term quality metrics used by the Centers for Medicaid and Medicare Services to quantify performance and allocate Medicare funding. They have been discussed as a basis for the regionalization of care for certain complex procedures, including aortic repair.12,13 Although the VQI is a vascular-specific database and even the more general NSQIP database has added some vascular-targeted metrics for specific procedures, the PSI is a universal system, not designed with any specification for vascular procedures. It has been shown previously to function poorly with respect to accurately identifying complications in vascular procedures, including OAR.14,15
It remains unclear what characteristics define the optimal system for assessing complications that occur following complex endovascular repair. Herein we examine and compare three major outcomes tracking systems (VQI, NSQIP, and PSI) in their ability to detect complications at a single institution aortic program both before and after the introduction of complex endovascular (FEVAR) techniques.
METHODS
After institutional review board approval, we retrospectively analyzed a prospectively maintained database of aortic repairs performed at the Johns Hopkins Medical Institutions between 2004 and 2018. All patients undergoing FEVAR or OAR of a juxtarenal or pararenal aneurysm were included. Aneurysms were defined as juxtarenal or pararenal based on their close proximity to or direct involvement of the renal arteries, respectively. Aneurysms with significant extension to the visceral segment that would necessitate supraceliac clamping were not included. This patient cohort was selected to best represent the population whose aneurysmal extent would be relevant to the application of fenestrated stent grafts. Open repairs were further subdivided into two groups, based on whether they occurred before the introduction of fenestrated technology at our institution (January 2013) or after. FEVAR was the preferred approach after its introduction if patient anatomy was suitable (diameter, length, and angle of the aortic neck; distal landing zone; and caliber of the femoral artery access vessels). Any open repairs performed emergently were not included in this analysis. Patients were defined as symptomatic if they presented with abdominal or back pain thought to be associated with the aneurysm or new renal failure associated with renal artery involvement with the expanding aneurysm. Symptomatic patients were most often offered open repair given the time required for a customed FEVAR device to be ordered, created, and delivered. Connective tissue disease patients were also recommended for OAR.
The primary end point was the occurrence of any complication. Complications were initially defined by a surgeon-designed classification system created by a group of vascular surgeons at Johns Hopkins specifically for the population of patients undergoing repair of juxtarenal and pararenal aortic aneurysms in either an open or endovascular approach. The definitions were crafted based on existing literature and clinical experience with postoperative complications felt to be clinically significant. Complications were then reclassified according to the specifications used by the ACS-NSQIP, SVS-VQI, and AHRQ-PSI databases (Table I). These data definitions are available publicly for the PSI and NSQIP databases, and the thoracic and complex EVAR data dictionary was used for VQI. For reclassification, the electronic medical record of all included patients was manually retrospectively reviewed to allow for the determination of whether or not a complication would be counted by each database definition. Hemorrhagic, Cardiac, respiratory, renal, wound, visceral and limb ischemia, neurologic, and thromboembolic complications were included in the analysis because all were both clinically relevant and captured by the majority of the databases, with a few exceptions. For example, the PSI system does not capture several major organ systems (neurologic, extremity, gastrointestinal) and notably the VQI does not capture venous thromboembolism.
Table I.
Definitions of complications, by organ system, as defined by our surgeon-created system, the SVS-VQI, the ACS-NSQIP, and AHRQ-PSI systemsa
| Complication | Surgeon designed | SVS-VQI | ACS-NSQIP | AHRQ-PSI | Composite (most stringent) |
|---|---|---|---|---|---|
| Hemorrhagic | Return to the OR or additional procedure for control of hemorrhage | Quantifies RBC transfusions within 72 hours of the OR Return to the OR or additional procedure for control of hemorrhage |
Quantifies RBC transfusions within 72 hours of the OR Return to the OR or additional procedure for control of hemorrhage |
Return to the OR or additional procedure for control of hemorrhage | Quantifies RBC transfusions within 72 hours of the OR Return to the OR or additional procedure for control of hemorrhage |
| Cardiac | Cardiac arrest Myocardial infarction with EKG changes or cardiologist’s diagnosis or procedure (angiography or PCI) Any new onset dysrhythmia requiring ICU readmission or procedure (cardioversion) |
Cardiac arrest Myocardial infarction with elevated troponin or EKG changes Any new onset dysrhythmia requiring medication or cardioversion Any new onset heart failure |
Cardiac arrest Myocardial infarction characterized by troponins 3× upper limit of normal, EKG changes or cardiologist’s diagnosis |
Not captured | Cardiac arrest Myocardial infarction with EKG changes or cardiologist’s diagnosis or procedure (angiography or PCI) Any new onset dysrhythmia requiring ICU readmission or procedure (cardioversion) Any new onset heart failure |
| Respiratory | Pneumonia Unplanned reintubation Ventilator support >72 h Respiratory distress requiring ICU readmission |
Pneumonia Unplanned reintubation Captures ventilator support >12 h and >24 h |
Pneumonia Unplanned reintubation Ventilator support >48 h |
Unplanned reintubation Ventilator support >96 h |
Pneumonia Unplanned reintubation Captures ventilator support >12 h and >24 h Respiratory distress requiring ICU readmission |
| Renal | Creatinine increase of 3× baseline or to a level of >3 mg/dL with a change of at least 0.5 mg/dL New hemodialysis |
Creatinine increase of >0.5 mg/dL New hemodialysis |
Creatinine increase of > 2 mg/dL New hemodialysis |
New hemodialysis | Creatinine increase of >0.5 mg/dL New hemodialysis |
| Wound | Surgical site infection Dehiscence, superficial or deep |
Surgical site infection Dehiscence, superficial or deep |
Surgical site infection Dehiscence, superficial or deep |
Any dehiscence requiring return to OR | Surgical site infection Dehiscence, superficial or deep |
| Thromboembolic | Any DVT or PE | Not captured | Any DVT or PE | Any DVT or PE | Any DVT or PE |
| Bowel ischemia | Any ischemic colitis, medically or surgically managed | Any ischemic colitis, medically or surgically managed | Any ischemic colitis, medically or surgically managed | Not captured | Any ischemic colitis, medically or surgically managed |
| Neurologic | Any cerebrovascular accident Any spinal cord ischemia |
Any cerebrovascular accident | Any cerebrovascular accident | Not captured | Any cerebrovascular accident Any spinal cord ischemia |
| Extremity | Limb ischemia requiring procedural or operative intervention | Any documented limb ischemia, medically or surgically managed | Limb ischemia requiring procedural or operative intervention | Not captured | Any documented limb ischemia, medically or surgically managed |
ACS-NSQIP, American College of Surgeons National Surgical Quality Improvement Program; AHRQ-PSI, Agency for Healthcare Research and Quality Patient Safety Indicators; DVT, deep venous thrombus; EKG, electrocardiogram; ICU, intensive care unit; OR, operating room; PCI, percutaneous coronary intervention; PE, pulmonary embolus; RBC, red blood cell; SVS-VQI, Society for Vascular Surgery Vascular Quality Initiative.
The final column delineates the all-inclusive and most stringent definition from all systems as the composite system for defining complications.
To examine the ability of each system to capture complications, a composite classification system was created incorporating all possible complications and, when definitions varied, the most stringent version of each organ system definition (Table I). The use of this method identified a set of parameters that caught 100% of patients who incurred a complication in any of the four system (surgeon designed, PSI, VQI, or NSQIP).
The associations of individual patient characteristics with outcomes were measured with analysis of variance testing for continuous variables and Pearson’s χ2 testing for categorical variables. Values were accepted as significant if P ≤ .05. Proportion testing was used to compare complication rates between the individual outcome classification systems, and the strength of correlation between the systems was determined using Spearman’s rank correlation test, reported in the form of a correlation coefficient (ρ). For the generated correlation coefficient, the coefficient was interpreted as indicative of a strong correlation if ρ > 0.59 and indicative of a weak correlation if ρ < 0.40; any intermediate value was interpreted as moderate. All data analysis was performed using Stata Version 15.1 (StataCorp LP, College Station Tex).
RESULTS
Of 145 patients who underwent repair of juxtarenal or pararenal aortic aneurysms, 60 (41.4%) underwent FEVAR and 85 (58.6%) underwent OAR. Of the patients undergoing OAR, 42 (49.4%) underwent repair after the introduction of FEVAR at our institution. The demographics, comorbidities and operative data of these groups are summarized in Table II. The cohorts were, overall, similar with a few notable exceptions. The FEVAR group tended to be older (average 75.3 years vs 67.0 years in OAR group after FEVAR introduction and 70.1 years in OAR group before FEVAR introduction; P < .001), were less likely to be symptomatic (P = .023) or to have connective tissue disease (P = .006), to have smaller aneurysms (P = .016), and to have lower operative estimated blood loss (P < .001) than the OAR group. When comparing the overall patient populations by era (compared with the P value P era in Table II), the patients in the era after the introduction of FEVAR had smaller aneurysms (P = .030), were more likely to have congestive heart failure (P = .021), and had a lower estimated blood loss (P = .044). When comparing only the OAR populations between the two eras (P open), the only notable shifts in the patient factors were an increase in patients with congestive heart failure (P = .012) and connective tissue disease (P = .038).
Table II.
Demographic and operative information for the study cohort
| OAR pre (n = 43) | OAR post (n = 42) | FEVAR (n = 60) | P value | P value open | P value era | |
|---|---|---|---|---|---|---|
| Male sex | 28 (65.1) | 27 (64.3) | 48 (80.0) | .135 | .936 | .308 |
| Race | ||||||
| White | 38 (88.4) | 36 (85.7) | 52 (86.7) | .934 | .715 | .732 |
| Nonwhite | 5 (11.6) | 6 (14.3) | 8 (13.3) | – | – | – |
| Age, years | 70.1 ± 6.8 | 67.0 ± 9.4 | 75.3 ± 9.6 | <.001 | .088 | .287 |
| AAA size, cm | 6.4 ± 1.3 | 6.2 ± 1.0 | 5.8 ± 0.6 | .016 | .490 | .030 |
| Baseline sCr, mg/dL | 1.11 ± 0.37 | 1.10 ± 0.37 | 1.21 ± 0.64 | .213 | .840 | .581 |
| Symptomatic | 11 (25.6) | 9 (21.4) | 4 (6.7) | .023 | .652 | .057 |
| HLD | 34 (79.1) | 32 (76.2) | 52 (86.7) | .367 | .750 | .643 |
| HTN | 36 (83.7) | 36 (85.7) | 55 (91.7) | .439 | .799 | .359 |
| DM | 8 (18.6) | 5 (11.9) | 11 (18.3) | .616 | .391 | .684 |
| PVD | 6 (14.0) | 5 (11.9) | 14 (23.3) | .236 | .778 | .481 |
| CAD | 25 (58.1) | 17 (40.5) | 29 (48.3) | .263 | .103 | 0151 |
| COPD | 12 (27.9) | 9 (21.4) | 18 (30.0) | .620 | .489 | .859 |
| CHF | 1 (2.3) | 8 (19.0) | 8 (13.3) | .050 | .012 | .021 |
| Connective tissue disease | 0 | 4 (9.5) | 0 | .006 | .038 | .188 |
| Smoking history | 30 (69.8) | 36 (85.7) | 48 (80.0) | .189 | .078 | .091 |
| CKD | 10 (23.3) | 7 (16.7) | 19 (31.7) | .217 | .441 | .776 |
| ESRD | 0 | 0 | 1 (1.7) | .490 | – | .515 |
| Suprarenal clamp | 40 (93.0) | 34 (81.0) | – | .097 | .097 | .097 |
| Clamp time, minutes | 34.1 ± 15.7 | 36.2 ± 11.9 | – | – | .540 | – |
| EBL, mL | 1836 ± 1338 | 2179 ± 1740 | 619 ± 839 | <.001 | .330 | .044 |
| Fluoro time, minutes | – | – | 57.5 ± 29.9 | – | – | – |
| Contrast, mL | – | – | 139 ± 54.7 | – | – | – |
AAA, Abdominal aortic aneurysm; CAD, coronary artery disease; COPD, chronic obstructive pulmonary disease; CHF, congestive heart failure; CKD, chronic kidney disease; DM, diabetes mellitus; EBL, estimated blood loss; ESRD, end-stage renal disease; FEVAR, fenestrated endovascular abdominal aortic aneurysm repair; HLD, hyperlipidemia; HTN, hypertension; OAR post, after the introduction of fenestrated technology; OAR pre, before the introduction of fenestrated technology; PVD, peripheral vascular disease; sCr, serum creatinine.
Values are frequency (%) for discrete variables and means ± standard deviation for continuous variables. P values, bolded where significant (P ≤ .05), represent comparison between all three groups (P), between the two open repair groups before (OAR pre) and after (OAR post) the introduction of fenestrated technology (P open), and between all patients before (OAR pre) and after (OAR post and FEVAR) the introduction of fenestrated technology (P era).
A comparison of outcomes between the groups is fully demonstrated in Table III. There was no difference in mortality between the groups or the eras. Patients undergoing FEVAR had a significantly shorter length of stay when compared with the OAR groups (4.0 days vs 8.7 days for OAR after the introduction of FEVAR and 9.4 days for OAR before the introduction of FEVAR; P < .001). By surgeon-designed and VQI definitions, the introduction of FEVAR decreased the overall number of complications for the patient population receiving either type of complex aneurysm repair (37.2% before FEVAR introduction vs 20.6% after [OAR and FEVAR] by surgeon definitions [P = .036]; and 55.8% before the introduction of FEVAR vs 36.3% after the introduction of FEVAR by VQI definitions [P = .029]). When examining system-specific complications, the introduction of FEVAR decreased respiratory complications for all-comers in both the surgeon-designed and PSI systems (16.3% for OAR before FEVAR introduction vs 4.9% after the introduction of FEVAR [OAR and FEVAR] by surgeon definitions [P = .023]; and 7.0% before the introduction of FEVAR vs 1.0% after the introduction of FEVAR by PSI system [P = .044]). By NSQIP and VQI definitions, FEVAR patients had lower rates of respiratory complications than OAR patients (0% of FEVAR patients vs 14.1% of all OAR patients by NSQIP definitions [P = .009]; and 0% in FEVAR patients vs 9.4% in OAR by VQI definitions [P = .049]). The VQI system alone identified a significant decrease in renal complications among the FEVAR group (16.7% vs 34.1%; P = .044).
Table III.
Postoperative outcomes and complications by classification system in open aortic (OAR) and fenestrated endovascular abdominal aortic aneurysm repair (FEVAR) patients
| OAR pre (n = 43) | OAR post (n = 42) | FEVAR (n = 60) | P value | P value open | P value era | |
|---|---|---|---|---|---|---|
| Surgeon designed | ||||||
| Mortality | 0 | 0 | 1 (1.7%) | .232 | – | .515 |
| Any complication | 16 (37.2) | 7 (16.7) | 14 (23.3) | .083 | .033 | .036 |
| Hemorrhage | 0 | 0 | 0 | – | – | – |
| Cardiac | 4 (9.3) | 1 (2.4) | 4 (6.7) | .410 | .175 | .316 |
| Respiratory | 7 (16.3) | 5 (11.9) | 0 | .008 | .563 | .023 |
| Renal | 6 (14.0) | 3 (7.1) | 5 (8.3) | .513 | .308 | .255 |
| Wound | 3 (7.0) | 0 | 2 (3.3) | .211 | .081 | .131 |
| DVT/PE | 1 (2.3) | 0 | 0 | .303 | .320 | .122 |
| Bowel ischemia | 1 (2.3) | 1 (2.4) | 6 (10.0) | .139 | .987 | .274 |
| Neurologic | 0 | 1 (2.4) | 3 (5.0) | .306 | .309 | .188 |
| Extremity | 1 (2.3) | 1 (2.4) | 0 | .489 | .987 | .526 |
| SVS–VQI | ||||||
| Any complication | 24 (55.8) | 18 (42.9) | 19 (31.7) | .050 | .232 | .029 |
| Hemorrhage | 4 (9.3) | 2 (4.8) | 5 (8.3) | .063 | .414 | .612 |
| Cardiac | 5 (11.6) | 1 (2.4) | 4 (6.7) | .242 | .096 | .144 |
| Respiratory | 4 (9.3) | 4 (9.5) | 0 | .049 | .972 | .195 |
| Renal | 15 (34.9) | 14 (33.3) | 10 (16.7) | .044 | .709 | .092 |
| Wound | 3 (7.0) | 0 | 2 (3.3) | .211 | .081 | .131 |
| DVT/PE | – | – | – | – | – | – |
| Bowel ischemia | 1 (2.3) | 1 (2.4) | 6 (10.0) | .139 | .987 | .274 |
| Neurologic | 0 | 0 | 3 (5.0) | .114 | – | .256 |
| Extremity | 1 (2.3) | 1 (2.4) | 0 | .489 | .987 | .526 |
| ACS–NSQIP | ||||||
| Any complication | 15 (34.9) | 7 (16.7) | 20 (33.3) | .119 | .054 | .283 |
| Hemorrhage | 3 (7.0) | 2 (4.8) | 5 (8.3) | .799 | .665 | .964 |
| Cardiac | 2 (4.7) | 1 (2.4) | 2 (3.3) | .844 | .571 | .595 |
| Respiratory | 6 (14.0) | 6 (14.3) | 0 | .009 | .964 | .101 |
| Renal | 3 (7.0) | 2 (4.8) | 8 (13.3) | .304 | .665 | .601 |
| Wound | 3 (7.0) | 0 | 2 (3.3) | .208 | .081 | .126 |
| DVT/PE | 1 (2.3) | 0 | 0 | .303 | .303 | .122 |
| Bowel ischemia | 0 | 1 (2.4) | 6 (10.0) | .049 | .309 | .080 |
| Neurologic | 0 | 0 | 3 (5.0) | .120 | – | .256 |
| Extremity | 1 (2.3) | 1 (2.4) | 0 | .480 | .986 | .274 |
| AHRQ–PSI | ||||||
| Any complication | 3 (7.0) | 2 (4.8) | 1 (1.7) | .385 | .665 | .257 |
| Hemorrhage | 0 | 0 | 0 | – | – | – |
| Cardiac | – | – | – | – | – | – |
| Respiratory | 3 (7.0) | 1 (2.4) | 0 | .101 | .320 | .044 |
| Renal | 0 | 1 (2.4) | 1 (1.7) | .623 | .309 | .355 |
| Wound | 0 | 0 | 0 | – | – | – |
| DVT/PE | 1 (2.3) | 0 | 0 | .303 | .320 | .122 |
| Bowel ischemia | – | – | – | – | – | – |
| Neurologic | – | – | – | – | – | – |
| Extremity | – | – | – | – | – | – |
| New HD | 1 (2.3) | 1 (2.4) | 1 (1.7) | .960 | .987 | .888 |
| UTI | 1 (2.3) | 0 | 2 (3.3) | .503 | .320 | .888 |
| RTOR | 4 (9.3) | 3 (7.1) | 1 (1.7) | .212 | .717 | .195 |
| ICU readmission | 5 (11.6) | 1 (2.4) | 3 (5.0) | .185 | .096 | .079 |
| 30-day readmission | 4 (9.3) | 1 (2.4) | 1 (1.7) | .126 | .175 | .043 |
| sCr max, mg/dL | 1.71 ± 1.02 | 1.90 ± 1.66 | 1.65 ± 1.47 | .688 | .535 | .865 |
| LOS (days) | 9.4 ± 8.9 | 8.7 ± 6.4 | 4.0 ± 3.9 | <.001 | .678 | .005 |
ACS-NSQIP, American College of Surgeons National Surgical Quality Improvement Program; AHRQ-PSI, Agency for Healthcare Research and Quality Patient Safety Indicators; DVT, deep vein thrombosis; HD, hemodialysis; ICU, intensive care unit; LOS, length of stay; OAR post, after the introduction of fenestrated technology; OAR pre, before the introduction of fenestrated technology; PE, pulmonary embolism; RTOR, return to operating room; sCr, serum creatinine; SVS-VQI, Society for Vascular Surgery Vascular Quality Initiative; UTI, urinary tract infection.
Values are frequencies (%) or mean ± standard deviation. P values, bolded where significant (P ≤ .05), represent comparisons between all three groups (P), between the two open repair groups before (OAR pre) and after (OAR post) the introduction of fenestrated technology (P open), and between all patients before (OAR pre) and after (OAR post and FEVAR) the introduction of fenestrated technology (P era).
Table IV compares the rates of complications between the different systems for FEVAR patients. PSI identified a complication in only one FEVAR patient (1.7%), which was attributable to a patient who required new dialysis after the procedure. The NSQIP and VQI identified a complication in 33.3% and 31.7% of FEVAR patients, respectively. Overall these systems show strong concordance both with each other and with the surgeon-defined classification system. For incurring any complication, concordances were ρ ≥ 0.735 among these three systems; for specific complications, concordances were all strong with ρ ≥ 0.674 (Table V). Total complication rates as defined by PSI, however, did not correlate with any other system, with all correlation coefficients weak (ρ ≤ 0.236; Table IV). On proportion testing, PSI identified significantly fewer complications in FEVAR patients than any of the other systems (P < .001). When broken down into components, PSI particularly underperformed compared with the VQI and the NSQIP with respect to detecting both hemorrhagic (P = .022 for both) and renal complications (P = .004 vs VQI and P = .015 vs NSQIP).
Table IV.
Summary of complications captured in each of the respective classification systems for fenestrated endovascular abdominal aortic aneurysm repair (FEVAR) patients
| Surgeon designed |
VQI |
NSQIP |
PSI |
P value |
Spearman’s ρ |
|||||
|---|---|---|---|---|---|---|---|---|---|---|
| n = 60 FEVAR patients | Versus surgeon designed | Versus VQI | Versus NSQIP | Versus surgeon designed | Versus VQI | Versus NSQIP | ||||
| Any complication | 14 (23.3%) | 19 (31.7) | 20 (33.3) | 1 (1.7) | <.001 | <.001 | <.001 | 0.236 | 0.191 | 0.184 |
|
| ||||||||||
| Hemorrhagic | 0 | 5 (8.3) | 5 (8.3) | 0 | – | .022 | .022 | – | – | – |
|
| ||||||||||
| Cardiac | 4 (6.7) | 4 (6.7) | 2 (3.3) | NC | – | – | – | – | – | – |
|
| ||||||||||
| Respiratory | 0 | 0 | 0 | 0 | – | – | – | – | – | – |
|
| ||||||||||
| Renal | 5 (8.3) | 10 (16.7) | 8 (13.3) | 1 (1.7) | .094 | .004 | .015 | 0.432 | 0.291 | 0.332 |
|
| ||||||||||
| Wound | 2 (3.3) | 2 (3.3) | 2 (3.3) | 0 | .154 | .154 | .154 | – | – | – |
|
| ||||||||||
| DVT/PE | 0 | NC | 0 | 0 | – | – | – | – | – | – |
|
| ||||||||||
| Bowel ischemia | 6 (10.0) | 6 (10.0) | 6 (10.0) | NC | – | – | – | – | – | – |
|
| ||||||||||
| Neurologic | 3 (5.0) | 3 (5.0) | 3 (5.0) | NC | – | – | – | – | – | – |
|
| ||||||||||
| Extremity | 0 | 0 | 0 | NC | – | – | – | – | – | – |
|
| ||||||||||
| Death | 1 (1.7) | 1 (1.7) | 1 (1.7) | 1 (1.7) | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 |
ACS-NSQIP, American College of Surgeons National Surgical Quality Improvement Program; AHRQ-PSI, Agency for Healthcare Research and Quality Patient Safety Indicators; DVT, deep vein thrombosis; NC, not captured; PE, pulmonary embolism; SVS-VQI, Society for Vascular Surgery Vascular Quality Initiative.
P values reflect proportion testing against the PSI system. Spearman’s ρ values reflect rank correlation testing against the PSI system. Values are number (%).
Table V.
Spearman’s ρ values reflecting rank correlation testing between institutional and Society for Vascular Surgery Vascular Quality Initiative (SVS-VQI) complications, institutional vs American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) complications, and NSQIP vs SVS-VQI complications by organ system for fenestrated endovascular abdominal aortic aneurysm repair (FEVAR) patients
| NSQIP | Versus SVS-VQI | |
|---|---|---|
| Any complication | 0.735 | |
| Hemorrhagic | – | |
| Cardiac | 0.695 | |
| Respiratory | – | |
| Renal | 0.877 | |
| Wound | 1.000 | |
| DVT/PE | – | |
| Bowel ischemia | 1.000 | |
| Neurologic | 1.000 | |
| Extremity | – | |
| Surgeon designed | Versus SVS-VQI | Versus ACS-NSQIP |
| Any complication | 0.774 | 0.780 |
| Hemorrhagic | – | – |
| Cardiac | 1.000 | 0.695 |
| Respiratory | – | – |
| Renal | 0.674 | 0.769 |
| Wound | 1.000 | 1.000 |
| DVT/PE | – | – |
| Bowel ischemia | 1.000 | 1.000 |
| Neurologic | 1.000 | 1.000 |
| Extremity | – | – |
DVT, Deep vein thrombosis; NC, not captured; PE, pulmonary embolism.
Table VI summarizes the complications in the OAR and FEVAR groups using the composite (ie, most stringent) definitions and the percentage of these complications captured by each of the classification systems. Overall, the NSQIP and VQI performed particularly well with capturing FEVAR complications, with 88% to 95% capture of both total complications and individual patients with complications. The VQI had a much higher capture rate of complications in the OAR group than the NSQIP (85%-98% vs 48%-51%), mostly owing to its capture of subtle renal complications (100% vs 17%), which were less common in the FEVAR group. The PSI system captured 12% of patients with complications in the OAR group and even fewer in the FEVAR group (5%), a trend that continued when examining total complication capture, which was only 3% for PSI in both categories.
Table VI.
Number of complications captured by each system for open complex aortic repair (OAR) and fenestrated endovascular abdominal aortic aneurysm repair (FEVAR) compared with the composite standard classification (the all-inclusive and most stringent version of each organ system definition)
| Composite |
Surgeon designed |
SVS-VQI |
ACS-NSQIP |
AHRQ-PSI |
||||||
|---|---|---|---|---|---|---|---|---|---|---|
| OAR (n = 85) | FEVAR (n = 60) | OAR | FEVAR | OAR | FEVAR | OAR | FEVAR | OAR | FEVAR | |
| Patients with any complication | 43 | 21 | 23 (53) | 14 (67) | 42 (98) | 19 (90) | 22 (51) | 20 (95) | 5 (12) | 1 (5) |
|
| ||||||||||
| Total complications | 67 | 33 | 35 (52) | 21 (64) | 57 (85) | 31 (94) | 32 (48) | 29 (88) | 2 (3) | 1 (3) |
|
| ||||||||||
| Hemorrhagic | 6 | 5 | 0 (0) | 0 (0) | 6 (100) | 5 (100) | 5 (100) | 5 (100) | 0 (0) | 0 (0) |
|
| ||||||||||
| Cardiac | 6 | 4 | 5 (83) | 4 (100) | 6 (100) | 4 (100) | 3 (50) | 2 (50) | – | – |
|
| ||||||||||
| Respiratory | 16 | 0 | 12 (75) | 0 (100) | 8 (50) | 0 (100) | 12 (75) | 0 (100) | – | – |
|
| ||||||||||
| Renal | 30 | 10 | 9 (30) | 5 (50) | 30 (100) | 10 (100) | 5 (17) | 8 (80) | 1 (3) | 1 (10) |
|
| ||||||||||
| Wound | 3 | 2 | 3 (100) | 2 (100) | 3 (100) | 2 (100) | 3 (100) | 2 (100) | 0 (0) | 0 (0) |
|
| ||||||||||
| DVT/PE | 1 | 0 | 1 (100) | 0 (100) | – | – | 1 (100) | 0 (100) | 1 (100) | 0 (100) |
|
| ||||||||||
| Bowel ischemia | 2 | 6 | 2 (100) | 6 (100) | 2 (100) | 6 (100) | 1 (100) | 6 (100) | – | – |
|
| ||||||||||
| Neurologic | 1 | 3 | 1 (100) | 3 (100) | 0 (0) | 3 (100) | 0 (0) | 3 (100) | – | – |
|
| ||||||||||
| Extremity | 2 | 0 | 2 (100) | 0 (100) | 2 (100) | 0 (100) | 2 (100) | 0 (100) | – | – |
|
| ||||||||||
| UTI | 1 | 2 | 1 (100) | 2 (100) | – | – | 1 (100) | 2 (100) | – | – |
|
| ||||||||||
| Death | 0 | 1 | 0 (100) | 1 (100) | 0 (100) | 1 (100) | 0 (100) | 1 (100) | 0 (100) | 1 (100) |
ACS-NSQIP, American College of Surgeons National Surgical Quality Improvement Program; AHRQ-PSI, Agency for Healthcare Research and Quality Patient Safety Indicators; DVT, deep vein thrombosis; NC, not captured; PE, pulmonary embolism; SVS-VQI, Society for Vascular Surgery Vascular Quality Initiative; UTI, urinary tract infection.
Values are number (%).
Of the 60 FEVAR patients, there was one death (1.7%) within 30 days, and four patients (6.7%) did not have a documented follow-up appointment after discharge from their index hospitalization. At a median length of follow-up of 1.25 years (interquartile range, 0.37-2.69 years) for the remaining 55 FEVAR patients, nine (16.4%) have been diagnosed with endoleaks, six (10.9%) with a type II, two (3.6%) with a type III, and one (1.8%) with a type I and type II endoleak. Three patients (5.5% of the 55 FEVAR patients with follow-up) have required reintervention for endoleaks.
DISCUSSION
Overall, our results emphasize that not all performance measure systems are created equal in terms of their ability to detect and capture clinically relevant outcomes with respect to vascular surgery, particularly low morbidity vascular procedures, such as endovascular aortic repair. The VQI and NSQIP systems correlate well with a surgeon-created system of tracking clinical outcomes—these systems capture complications that clinicians deem important and correlate well with each other, and the VQI demonstrates versatility in its ability to capture subtle renal injury in both open and endovascular complex aortic repair. The PSI system does not show concordance with any of the clinically focused systems and identified only 5% of complications in the FEVAR group when compared with a composite classification system with stringent definitions.
A previous study of the PSI system in renovisceral aortic repair in a large cohort of Medicare patients demonstrated a significant overall decrease of PSI incurrences with the introduction of complex endovascular technology.16 In our cohort, respiratory PSI were the only significantly decreased complications in the era after FEVAR introduction, but a significant decrease in total complications was demonstrated statistically in both the VQI and institutional systems. This finding was due in large part to a decrease in the rate of renal complications, consistent with what has been published previously about the morbidity benefits of complex EVAR.6,17 The significant decrease in the rate of respiratory complications in our cohort was primarily owing to a lack of prolonged intubations and pneumonias in the FEVAR group; previous reports of morbidity benefit of FEVAR have demonstrated largely equivalent pulmonary outcomes, but may have used less stringent definitions of pulmonary complications.5,6,17 All told, our findings support that the introduction of fenestrated technology has allowed a subset of older, more frail patients to access aortic repair.
The PSI system overall captured 1 complication in 60 consecutive FEVARs, which was only 3% of complications captured by other systems. This extremely low capture rate, coupled with the weak concordance with all the clinically focused outcomes tracking systems, calls into serious question the logic of using the PSI as a means of ranking aortic programs. Previous work has highlighted the inaccuracy of PSI as a performance metric in OAR,14 which again is reflected here with a 12% capture of patients with complications in the open aortic cohort; however, this inaccuracy becomes increasingly relevant as low-morbidity endovascular techniques are applied to progressively more complex disease processes in a frail cohort of patients.
Although the use of a general surgery system such as the NSQIP to track complications in FEVAR, a procedure that is highly specialized and less similar to general surgery than OAR, seems likely to be associated with poor performance, this was not the case. The NSQIP system actually performed better in terms of capturing complication in FEVAR cases vs OAR (95% vs 51% of total patients with complications captured). The VQI, a vascular-specific classification system, performed well in both scenarios (90% vs 98%).
In our cohort, renal complications were significantly decreased in the FEVAR cohort by VQI definitions. Previous large series examining the shift in patient populations accessing complex aortic repair with the advent of fenestrated and branched stent grafts has demonstrated these patients tend to have higher rates of chronic kidney disease than open repair patients.5,17 This finding makes particularly important the need for advanced catheter skills to optimize deployment of the body graft and the bridging renovisceral stents and to minimize the contrast load administered.18 Prior studies have shown that the volume of contrast required to perform FEVAR is 28% higher than that for simple EVAR and the amount of contrast administered can vary 10-fold between surgeons even for simple EVAR.18,19 Efforts to correlate the volume of contrast with short-term postoperative renal dysfunction after EVAR have not been successful, but with the knowledge that most significant renal functional decline occurs in the months to years after endovascular repair and that any renal dysfunction after EVAR is associated with increased mortality,19 it is difficult to make the argument that the significant contrast load presented to the kidneys during FEVAR has no importance.
A perfect quality metric in aortic repair does not exist, but in this analysis the VQI system has the greatest granularity with regard to renal dysfunction and captures the most postoperative complications overall. This high capture rate in addition to its concordance with a surgeongenerated classification system indicates its ability to cast a net acquiring the largest amount of clinically relevant complications. Surgeons performing multiplatform complex aortic repairs must be aware of the insufficiency and lack of granularity inherent to the PSI system in which they are being judged and compared. Efforts to regionalize aortic repair and use performance-based compensation should be based on highly sensitive vascular-specific quality metrics that account for the inherent differences between open and endovascular repair. Additionally, because most complication tracking systems are focused on short-term outcomes limited to the initial hospital stay or the first 30 days after a procedure, they may reflect the low perioperative morbidity of endovascular procedures, but not capture the potential long-term reinterventions. The low frequency of short term morbidity and mortality in EVAR also makes it more difficult to use complication tracking systems to compare aortic programs, because the events of interest are rare.15
Limitations inherent to this work include its retrospective nature, which hinders its ability to capture complications that rely on narrative and coding documentation in the electronic medical record. Also notable is that the electronic medical record at our institution changed in the middle of the postimplementation era (July 2016), which introduces the possibility of additional documentation discrepancy to the data. The moderate size of the sample facilitated the close review of the source data, but this element, along with the relatively low number of complications captured in some of the systems, also limits the power of this dataset in performing multivariate analyses and drawing more specific conclusions. This work is strengthened by its primary basis on consecutive data from a high-volume complex aortic referral practice that uses both OAR and FEVAR for juxtarenal and pararenal aneurysms. Single-institution data made it possible for scrupulous examination of the electronic medical record and accurate recoding of the complications in accordance with the unique definitions of each classification system.
CONCLUSIONS
When applying the PSI definitions of complications to a single-institution cohort of complex aortic repairs, PSI identified only one postoperative event among 60 consecutive FEVAR cases. This was a significantly lower rate of identifying clinically relevant events when compared with two other major systems of tracking postoperative complications (VQI and NSQIP), particularly with respect to renal outcomes. Additionally, on correlation testing, the PSI showed very weak concordance with both these other clinically based systems. These findings have particular relevance as advanced, low morbidity endovascular techniques increase in their prevalence and applicability to frail patients with complex aneurysms. Any future performance-based efforts to regionalize care and modify compensation for aortic aneurysm repair must be rooted in a vascular-specific, stringent system of tracking patient outcomes.
ARTICLE HIGHLIGHTS.
Type of Research:
Single-center, retrospective analysis of prospectively collected cohort data
Key Findings:
Of 145 juxtarenal or pararenal aortic aneurysm patients, 41.1% underwent fenestrated endovascular abdominal aortic aneurysm repair (FEVAR) and 58.6% open repair. Introduction of FEVAR decreased complications from 37.2% to 20.6% by a surgeon-defined system. The Society for Vascular Surgery Vascular Quality Improvement database identified the most complications (39.9% of the entire cohort and 25% of FEVAR). The Agency for Healthcare Research and Quality Patient Safety Indicators captured the fewest (4.1% and 5.0%).
Take Home Message:
The introduction of FEVAR is associated with an overall decrease in complications. Patient safety indicators are insufficient to rank complex aortic programs with high levels of FEVAR use.
Footnotes
The editors and reviewers of this article have no relevant financial relationships to disclose per the JVS policy that requires reviewers to decline review of any manuscript for which they may have a conflict of interest.
REFERENCES
- 1.Chaikof EL, Dalman RL, Eskandari MK, Jackson BM, Lee WA, Mansour MA, et al. The Society for Vascular Surgery practice guidelines on the care of patients with abdominal aortic aneurysm. J Vasc Surg 2018;67:2–77. [DOI] [PubMed] [Google Scholar]
- 2.Oderich GS, Greenberg RK, Farber M, Lyden S, Sanchez L, Fairman R, et al. Results of the United States multicenter prospective study evaluating the Zenith fenestrated endovascular graft for treatment of juxtarenal abdominal aortic aneurysms. J Vasc Surg 2014;60:1420–8. [DOI] [PubMed] [Google Scholar]
- 3.Orr NT, Davenport DL, Minion DJ, Xenos ES. Comparison of perioperative outcomes in endovascular versus open repair for juxtarenal and pararenal aortic aneurysms: a propensity-matched analysis. Vascular 2017;25:339–45. [DOI] [PubMed] [Google Scholar]
- 4.Mohamed N, Galyfos G, Anastasiadou C, Sachmpatzidis I, Kikiras K, Papapetrou A, et al. Fenestrated endovascular repair for pararenal or juxtarenal abdominal aortic aneurysms: a systematic review. Ann Vasc Surg 2020;63:399–408. [DOI] [PubMed] [Google Scholar]
- 5.Doonan RJ, Girsowicz E, Dubois L, Gill HL. A systematic review and meta-analysis of endovascular juxtarenal aortic aneurysm repair demonstrates lower perioperative mortality compared with open repair. J Vasc Surg 2019;70:2054–64.e3. [DOI] [PubMed] [Google Scholar]
- 6.Tinelli G, Crea MA, de Waure C, Di Tanna GL, Becquemin JP, Sobocinski J, et al. A propensity-matched comparison of fenestrated endovascular aneurysm repair and open surgical repair of pararenal and paravisceral aortic aneurysms. J Vasc Surg 2018;68:659–68. [DOI] [PubMed] [Google Scholar]
- 7.Zettervall SL, Schermerhorn ML, Soden PA, McCallum JC, Shean KE, Deery SE, et al. The effect of surgeon and hospital volume on mortality following open and endovascular repair of abdominal aortic aneurysms. J Vasc Surg 2017;65:626–34. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Haulon S, Barilla D, Tyrrell M, Tsilimparis N, Ricotta JJ II. Debate: whether fenestrated endografts should be limited to a small number of specialized centers. J Vasc Surg 2013;57:875–82. [DOI] [PubMed] [Google Scholar]
- 9.Society for Vascular Surgery Vascular Quality Initiative. Data analysis. Updated Jan 2020. Available at: www.vqi.org/data-analysis/. Accessed January 2020.
- 10.American College of Surgeons. User guide for the 2018 ACS NSQIP Participant Use Data File (PUF). Released October 2019. Available at: www.facs.org/-/media/files/quality-programs/nsqip/nsqip_puf_userguide_2018.ashx. Accessed January 2020.
- 11.Agency for Healthcare Research and Quality. Patient safety indicators technical specifications. Version v2019 (ICD 10-CM/PCS), July 2019. Available at: www.qualityindicators.ahrq.gov/Modules/PSI_TechSpec_ICD10_v2019.aspx. Accessed January 2020.
- 12.Cima RR, Lackore KA, Nehring SA, Cassivi SD, Donohue JH, Deschamps C, et al. How to best measure surgical quality? Comparison of the Agency for Healthcare Research and Quality Patient Safety Indicators (AHRQ-PSI) and the American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) postoperative adverse events at a single institution. Surgery 2011;150:943–9. [DOI] [PubMed] [Google Scholar]
- 13.Kubasiak JC, Francescatti AB, Behal R, Myers JA. Patient safety indicators for judging hospital performance. Am J Med Qual 2017;32:129–33. [DOI] [PubMed] [Google Scholar]
- 14.Sorber R, Giuliano KA, Hicks CW, Black JH III. Patient safety indicators are an insufficient performance metric to track and grade outcomes of open aortic repair. J Vasc Surg 2020. May 19. [Epub ahead of print]. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Kaafarani HM, Borzecki AM, Itani KM, Loveland S, Mull HJ, Hickson K, et al. Validity of selected patient safety indicators: opportunities and concerns. Am Coll Surg 2011;212:924–34. [DOI] [PubMed] [Google Scholar]
- 16.Rose J, Evans C, Barleban A, Bandyk D, Wilson SE, Chang DC, et al. Comparative safety of endovascular aortic aneurysm repair over open repair using patient safety indicators during adoption. JAMA Surg 2014;149:933. [DOI] [PubMed] [Google Scholar]
- 17.Rao R, Lane TR, Franklin IJ, Davies AH. Open repair versus fenestrated endovascular aneurysm repair of juxtarenal aneurysms. J Vasc Surg 2015;61:242–55. [DOI] [PubMed] [Google Scholar]
- 18.Westin GG, Rockman CB, Sadek M, Ramkhelawon B, Cambria MR, Silvestro M, et al. Increased ischemic complications in fenestrated and branched endovascular abdominal aortic repair compared with standard endovascular repair. J Vasc Surg 2020;72:36–43. [DOI] [PubMed] [Google Scholar]
- 19.Zarkowsky DS, Hicks CW, Bostock IC, Stone DH, Eslami M, Goodney PP. Renal dysfunction and the associated decrease in survival after elective endovascular aneurysm repair. J Vasc Surg 2016;64:1278–85. [DOI] [PMC free article] [PubMed] [Google Scholar]