Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2016 Apr 1.
Published in final edited form as: JAMA Surg. 2015 Apr 1;150(4):325–331. doi: 10.1001/jamasurg.2014.1819

What is High Risk Surgery? Development of a List of High Risk Operations for Patients Age 65 and Older

Margaret L Schwarze 1, Amber E Barnato 2, Paul J Rathouz 3, Qianqian Zhao 3, Heather B Neuman 1, Emily R Winslow 1, Gregory D Kennedy 1, Yue-Yung Hu 4, Christopher M Dodgion 1, Alvin C Kwok 5, Caprice C Greenberg 1
PMCID: PMC4414395  NIHMSID: NIHMS677524  PMID: 25692282

Abstract

Importance

No consensus exists regarding the definition of “high risk” surgery in older adults. An inclusive and precise definition of high risk surgery may be useful for surgeons, patients, researchers and hospitals.

Objectives

To develop a list of “high risk” operations.

Design

1) Retrospective cohort study; and 2) Modified Delphi procedure.

Setting

All Pennsylvania acute care hospitals (Pennsylvania Health Care Cost Containment Council [PHC4], 2001–2007) and a nationally-representative sample of U.S. acute care hospitals (Nationwide Inpatient Sample [NIS], HCUP, AHRQ 2001–2006).

Patients

Admissions 65 and older to PHC4 hospitals and admissions 18 and older to NIS hospitals.

Methods

We identified ICD-9 CM procedure codes associated with >1% inpatient mortality in PHC4. We used a modified Delphi technique with 5 board certified surgeons to further refine this list by excluding non-operative procedures and operations that were unlikely to be the proximate cause of mortality and were instead a marker of critical illness (e.g., tracheostomy). We then cross-validated this list of ICD-9CM codes in the NIS.

Main Outcomes Measures

1) Delphi consensus of at least 4/5 panelists; 2) proportion agreement in the NIS.

Results

Among 4,739,522 admissions 65 and older in PHC4, 2,569,589 involved a procedure, encompassing 2,853 unique procedures. Of 1,130 procedures associated with a crude inpatient mortality of at least 1%, 264 achieved consensus as high risk operations by Delphi. The observed inpatient mortality in the NIS was ≥ 1% for 227/264 (86%) of the procedures in patients age 65 and older. The pooled inpatient mortality rate for these identified high risk procedures performed on patients age ≥65 was double the inpatient mortality for correspondingly identified high risk operations for patients less than 65 (6% vs. 3%).

Conclusions

We developed a list of procedure codes that can be used to identify “high risk” surgical procedures in claims data. This list of “high risk” operations can be used to standardize the definition of high risk surgery in quality and outcomes-based studies and design targeted clinical interventions.

Introduction

High risk surgery is not well defined, but surgeons “know it when they see it.” Surgery can be high risk due to patient specific factors or operation specific factors;1,2 however, teasing out these comingled contributors can be challenging. There is little debate that open repair of an abdominal aortic aneurysm (AAA) is high risk surgery. However, this operation is almost exclusively performed on older patients, most of whom have pre-existing cardiovascular disease or risk factors for vascular disease. As such, the operation is high risk partly due to the characteristics of the patients on whom it is routinely performed. Nonetheless, the operation itself has inherent risks given the need for laparotomy, and the cardiac stress engendered by aortic cross-clamping.

Some investigators have characterized high risk surgery by identifying operations that are associated with significant inpatient mortality. Although these lists identify operations a surgeon might characterize as high risk, the collection of operations is contaminated by operations associated with caring for patients with critical illness, such as tracheostomy, ventriculostomy and wound debridement.3–5 Others have focused more on patient factors, attempting to identify a high risk group of patients who have any surgical procedure. 2,6 Using another approach, Birkmeyer and colleagues have examined surgical quality and safety for over 15 years using a specific group of major cardiovascular and cancer operations with high operative morbidity or mortality (AAA repair, carotid endarterectomy, coronary artery bypass grafting (CABG), aortic valve repair (AVR), pancreatectomy, esophagectomy, gastrectomy, and lung resection).7,8 This strategy more precisely identifies high risk surgery, covers 54 ICD9-CM codes and a large number of operations performed annually (344,766).9,10 However, the list is limited, excluding many operations that are typically considered high risk such as thoracic aneurysm repair, organ transplantation and neurosurgical procedures. Furthermore, the list contains procedures that are primarily performed electively. Currently, there is no general consensus about a broader definition of high risk surgery.

An inclusive and precise definition of high risk surgery may be useful for multiple purposes: 1) surgeons can use this information to characterize the nature of a proposed operation with patients and their families, 2) researchers can use this information to evaluate trends, outliers and successful treatment of postoperative complications (rescue) and 3) hospitals can use this information to estimate procedure specific mortality based on patient age and admission acuity. We sought to expand the list of high risk operations beyond the historically utilized list of elective major cardiovascular and oncologic operations for patients 65 and older. We then evaluated this more inclusive list using a second data set to determine the boundaries of our confidence in this list and used age and admission acuity to examine how this list might change with these variables.

Methods

To develop a list of high risk procedures we first identified operations associated with a 1% or greater mortality in a locally available data base. We then sorted these procedures into “high risk operations” and “not high risk operations” using a modified Delphi process to identify procedures that were the likely proximate cause of mortality and not simply associated with high mortality. We then used a second data set to determine the level of confidence that the identified “high risk operations” had a mortality of 1% or greater. (Figure)

Figure 1.

Figure 1

Open in a new tab

Flow diagram of the strategy used to identify procedures that are high risk surgery.

N signifies the number of surgical procedures identified with ≥1% inpatient mortality in the HCUP NIS.

Initial Identification of ICD-9 CM procedure codes with 1% or greater mortality

We chose a threshold of 1% mortality to categorize operations as “high risk” because operative risk has historically been equated to operative death11,12 and in the contemporary era 1% or greater mortality signifies substantial operative risk.10 We used the Pennsylvania Health Care Cost Containment Council (PHC4) data file that records all non-VA hospital admissions in Pennsylvania. We sampled all admissions among patients 65 and older between April 1, 2001 and December 31, 2007 and identified procedures by ICD-9CM codes associated with an average crude in-hospital mortality of 1% or greater. The PHC4 data comprise a convenience sample similar in size to a 5% sample of Medicare beneficiaries.

Grouping procedure codes

To facilitate review of the procedures, we then applied a modification of the Health Care Cost and Utilization Project (HCUP) Clinical Classifications Software (CCS) to organize these ICD-9CM surgical codes into cohesive groups. The HCUP CCS incorporates 3,900 ICD-9-CM codes from January 1980 to September 2011 and 9,000 CPT procedure codes from January 1992 to January 2011 in to 244 clinically meaningful categories.13–15 However, these categorizations were not designed to account for surgical complexity or risk in the groupings and needed to be adapted for our purposes. CCS categories composed of exclusively non-operative procedures (e.g. home health services, chemotherapy, laboratory, etc.) were excluded from review as they were not surgically relevant. Three surgical reviewers independently examined the CCS categorization of the remaining 2,071 ICD-9 procedure codes. We eliminated the CCS-defined categories that were clinically ambiguous, and their surgical ICD-9 codes were assigned to existing or newly created categories. For example, category 99 (other OR (operating room) GI therapeutic procedures) was largely heterogeneous and was eliminated. Codes were incorporated into existing categories such as exploratory laparotomy (89), and new categories were created for biliary minor (99.1), biliary major (99.2), pancreatic (99.3), liver minor (99.4), and liver major (99.5). Of the 2,071 ICD-9 codes reviewed, 300 (14.5%) codes required adjudication by a board-certified surgeon because at least one of the primary reviewers disagreed with the new classification. Our revised classification included 219 original CCS categories that did not contain surgical procedures or captured surgical complexity and risk well and an additional 50 new categories as described above (SAS code for new CCS groupings available from the authors upon request.)

Modified Delphi Process

In order to eliminate procedures that were markers of severity of illness (ie tracheostomy) rather than the proximate cause of death, we utilized a modified Delphi Process. The first author presented the procedures identified in the PHC4 organized within the new CCS groups to 5 other members of the team, all of whom are fellowship-trained, academic, board certified surgeons (MLS, HBN, ERW, GDK, CCG) who have been practicing from between five and ten years. The procedures were then iteratively sorted to separate the procedures that should be included on a list of high risk operations from those that should not be included on such a list. The specific goal was to include procedures that had both an operative mortality of 1% or greater and were likely the proximate cause of the patient’s death for “all-comers age 65 and older”. With each round, brief statements of the reasoning for excluding any procedure from the high risk surgery list or for including a procedure on the high risk surgery list that had been excluded in prior iterations was provided. After each Delphi round, we collected and collated responses. For each subsequent round a list of all of the procedures was represented to allow the panel to consider their responses in light of the anonymous annotations that described respondents’ reasoning from the previous round. This strategy was used for each subsequent round for a total of four rounds at which time there was minimal change in consensus ratings.

Cross-validation of procedure list: HCUP-NIS

We then calculated the crude inpatient mortality of the procedures meeting consensus in the Nationwide Inpatient Sample (NIS) from the Healthcare Cost and Utilization Project (HCUP), Agency for Healthcare Research and Quality (AHRQ) from January 1, 2001 to December 31, 2006. Each year, the NIS captures all admissions for patients 18 years and older in a stratified nationally-representative 20% sample of U.S. acute care hospitals.16 Specifically, we identified all admissions with one of the procedures identified in the Delphi process listed in the principal procedure field, then estimated the survey-weighted population total number of admissions and the crude inpatient mortality proportion (and 95% confidence interval) for each procedure over the 2001 to 2006 timeframe.16 We then transformed estimated proportions to the logit scale and performed one-sided hypothesis tests for whether inpatient mortality for a given procedure was less than 1% versus the alternative that is greater than 1%. We report one sided p-values from these tests with type I error rate set at 0.05. For our primary analysis, we limited our cohort to patients age 65 and older and then stratified our analysis by admission acuity (emergent and urgent versus elective). Finally, to determine whether or not this list could be extrapolated to patients under the age of 65, we repeated the analysis in the cohort of patients under age 65.

For each cohort, we estimated the survey-weighted total number of annual procedures performed nationally, and the survey-weighted mortality of this pooled group of procedures.

IRB

University of Pittsburgh authors conducted PHC4 analysis under a data use agreement with PHC4 with approval of the University of Pittsburgh IRB. University of Wisconsin authors conducted NIS analysis under a data use agreement with AHRQ with approval of the University of Wisconsin IRB.

Results

List Development: PHC4

Among 4,739,522 admissions 65 and older in PHC4 between April 1, 2001 and December 31, 2007, 2,569,589 involved a procedure, encompassing 2,853 unique (surgical and non-surgical) procedures. There were 1,130 distinct ICD-9CM procedures associated with an average crude inpatient mortality of 1% or greater. These procedures accounted for 40% of the procedures identified during the study period.

Results of Delphi Process

There was a 100% response rate to all rounds of the Delphi process. After four iterations, there was complete consensus (five out of five) for 219 procedures and near-complete consensus (four out of five) for 45 additional operations that were believed to be both surgical operations (operations performed in an operating room by a surgeon) and the likely proximate cause of the associated mortality. These 264 operations were used for subsequent analysis. This broad collection of operations includes cardiac and thoracic surgery, and neurosurgical, vascular, gastrointestinal and urologic operations.

Cross-validation of list: NIS

Among patients age 65 and older in the HCUP Nationwide Inpatient Sample between January 1, 2001 and December 31, 2006, there were 832,452 sample admissions for patients age 65 and older with one of the 264 ICD-9 procedure codes identified by our Delphi process listed as the principal procedure. The crude inpatient mortality in the NIS was significantly greater than or equal to 1% for 227 of the 264 (86%) operations identified by the Delphi process from the PH4C sample. (eTable 1) The weighted mortality estimates ranged from 1% (CI: 1–2%, p = 0.004) for creation of esophagogastric sphincter (ICD-9 = 44.66) to 68% (CI: 57 – 79%, p < 0.0001) for implantation of external ventricular assist device (ICD-9 = 37.65). Of the 37 procedures for which mortality was not statistically significantly greater than 1% in the NIS sample, 13 procedures did not occur in the NIS data set during the study period, suggesting these procedures are rarely performed or the specific procedure code is rarely used in patients age 65 and older. The remaining 24 procedural codes were identified in the NIS but the one-sided p-value was greater than 0.05, i.e. the mortality rate was not reliably greater than or equal to 1%. We confirmed that 37 of the 44 procedures where there was a 4 out of 5 consensus with the Delphi technique (as opposed to 5/5 consensus) had a mortality of ≥1% in the NIS.

Using the weighted national estimate, this final group of 227 ICD-9 operative procedure codes account for a total of 4,019,773 operations performed on patients 65 years and older between 2001 and 2006, or 669,962 primary operations per year. For the six-year study period, the estimated number of operations identified for each major category was: urologic 165,148, hepatobiliary 60,439, non-hepatobiliary gastrointestinal (including esophageal) 1,341,757, non-cardiac vascular 736,042, cardiac 1,332,626, thoracic 163,841 and neurosurgical 192,750.

Surgical Acuity in Patients 65 and Older

In order to determine the impact of admission acuity on the high risk procedure list, we examined the 264 procedures identified by the Delphi process according to the admission acuity. The observed inpatient mortality was significantly greater than or equal to 1% for patients age 65 and older during emergent or urgent admissions for 224 of the 264 procedures identified by the Delphi process in the PH4C. (eTable 2) Two-hundred fourteen of these procedures were included in the collection of 227 procedures identified as significantly high risk for all types of admissions in patients age 65 and older, and 10 procedures (unilateral adrenalectomy (7.22), endarterectomy (38.10), arm vessel replacement (38.43), abdominal vein resection/replacement (38.47), occlusion of leg vein (38.89), high gastric bypass (44.31), proctopexy (48.76), bile duct excision (51.69), transureteroureterostomy (56.75), repair of ureter (56.89)) were uniquely high risk for patients 65 and older during emergent or urgent admissions only. Using weighted national estimates, there were 1,646,535 high risk operations performed on patients age greater than or equal to 65 during emergent-urgent admissions, accounting for 45% of these procedures performed across all admission types.

The observed inpatient mortality was significantly greater than or equal to 1% for patients age 65 and older during elective admission for 163 of the 264 procedures identified in the PHC4. (eTable 3) Using the weighted national estimate, these procedures accounted for 1,812,512 operations in patients age 65 and older over the 6 year study period. The pooled in-hospital mortality rate across this group of procedures performed during elective admissions was 3% in comparison to the pooled inpatient mortality of 10% for the 224 procedures in the emergent/urgent setting. Figure 2 displays the distribution of mortality estimates across all identified procedures and across the subgroups of procedures restricted by the acuity of the admission.

Figure 2.

Figure 2

Open in a new tab

Pooled inpatient mortality for each collection of high risk operations.

Nationwide Inpatient Sample, Age less than 65

In patients less than 65 years old, we identified a total of 934,673 operations in the NIS 20% sample associated with the final 264 procedure codes determined by our Delphi process. The observed inpatient mortality for patients less than 65 in the NIS for operations was significantly greater than or equal to 1% for 154 of the 264 (58%) of procedures. The pooled in-patient mortality for these procedures in younger patients was 3%, significantly lower than the 6% pooled inpatient mortality for high risk operations for patients age ≥ 65 across all admissions. Using the weighted national estimates, approximately 2,709,026 operations were performed for these 154 procedure codes in patients less than 65 years old during the 2001–2006 study period. This translates to 451,504 high risk operations annually on patients less than 65 years of age. When examining the NIS population as a whole, younger patients account for just about half (49%) of the procedures performed for these selected 154 high risk operations. (eTable 4)

Discussion

Using two population based cohorts and the Delphi method, we identified a list of 227 high risk operations in patients age 65 and older. We found that older patients are recipients of more than 650,000 of these procedures annually. The pooled mortality rate for high risk procedures performed on patients age 65 and older was double the pooled inpatient mortality for the procedures on this list with a mortality of ≥1% for patients less than 65.

Our results have impact for surgeons, researchers, policy makers and patients. For surgeons, conceptualizing an operation as high risk, particularly for an older patient, can be used to assist the decision making calculus. By expanding this characterization beyond 8 – 14 primary procedures and excluding operations that are simply markers of critical illness and unlikely to be the proximate cause of death, our inclusive list defines a large cohort of operations for which the risks and benefits of surgery should be carefully evaluated to assure that trade-offs between surgical treatment and non-surgical options have been fully considered. For some operations on this list, such as lung transplantation and aortic valve replacement, such considerations are readily apparent. However, for other procedures, such as open right hemicolectomy which carries 2% inpatient mortality for patients age 65 and older during an elective admission, recognition and classification of this operation as high risk may prompt more careful deliberation and enhance consideration for non-surgical options. Although this list does not precisely link specific risk factors with a defined subset of isolated surgical complications, it characterizes “high-risk surgery” to yield a group of procedures that should trigger more consideration in terms of the risks and benefits particularly for patients 65 and older.

For researchers and policy makers, expansion of the list of high risk operations adds to the methodological capacity to study surgical safety and quality in order to judge the effects of interventional strategies for a large number of operations that carry considerable surgical mortality. This robust collection of operations with significant risk can be used to define a larger cohort of at-risk patients in order to assess the effects of quality and safety improvement efforts targeted at surgical outcomes in general (for example pre-habilitation or readmission reduction programs).

For patients age 65 and older, the designation of an operation as high risk may present an opportunity for the patient to pause and consider the value of surgery, alternative treatments or prepare for the real potential of an unwanted outcome. Although mortality statistics should be accurately and specifically disclosed to patients preoperatively, interpretation of numerical information is often difficult for patients to assimilate17,18 and patients may struggle to associate the quantitative description of a 1 or 2% operative mortality with an understanding that the operation is high risk. Notably, during all types of admissions the number of operations considered high risk for patients 65 and older is considerably larger than the number of high risk operations for younger patients (227 vs. 154), and the pooled mortality is twice as high for older patients. Although these estimates of mortality are not stratified by co-morbid conditions or by the less easily ascertained but likely more relevant designation of frailty,19,20 the widespread use of surgery in older Americans demonstrated in this cohort and the substantially higher mortality associated with these procedures suggests the significant relevance of age as a contributor to surgical outcomes.

Our findings have important strengths and limitations. By using two comprehensive inpatient discharge data sources we were able to assess the results of our Delphi process and increase the precision of the mortality estimates. However, administrative data sets are vulnerable to coding errors21 and both data sets capture in-hospital mortality only that does not capture all mortality in the early postoperative period. Although we used current data available at the time of our study, these data will likely change over time with improvements in clinical care and will certainly need to be reconsidered with the advent of ICD-10. Furthermore, our analysis was restricted to patient age and the acuity of admission to describe factors associated with operative mortality. We did not specifically risk stratify patients by known single organ-system risk predictors22–24 or physician-determined risk scores, for example ASA classification.25,26 Also, measures of clinical frailty, a potentially more accurate risk-predictor,19,27 are not readily accessible in the discharge data used in this study. As such, our list provides a general notion of operative intrinsic risk and additional risk calculators (for example, ACS NSQIP surgical risk calculator)28 designed for precise risk prediction are necessary to tailor individual risk discussions. Our results suggest that caution should be used in applying this list to patients under age 65 as the list is more limited (only 154 procedures) but that extrapolation may be useful in certain circumstances. Furthermore, our strategy was to develop the list for patients age 65 and older and then examine this list for patients less than age 65. There may be some procedures for patients age less than 65 that were not picked up by this strategy such that the list for younger patients is incomplete. Although we separated operations performed during elective admissions from those performed during emergent and urgent admissions, this designation does not allow us to account for operations that were performed emergently during elective admissions and as such, the elective surgery list most likely includes some operations that were performed emergently. Finally, although our modified Delphi process was completely anonymous respondents may have felt pressure to conform to the group process29,30 which could potentially overstate the degree of consensus we achieved.

Conclusion

We developed a list of 227 operations that carry a 1% or greater operative mortality for patients age 65 and older. These “high risk” operations are performed on more than half a million patients age 65 and older annually. Our results provide a standard that can be used to define high risk surgery that can be employed in clinical and research settings.

Supplementary Material

eTables

Acknowledgments

The authors conducted the analysis under a data use agreement with the Pennsylvania Health Care Cost Containment Council (PHC4). The following statement is provided and required by the Pennsylvania Health Care Cost Containment Council (PHC4): PHC4 has provided this data in an effort to further PHC4’s mission of educating the public and containing health care costs in Pennsylvania. PHC4, its agents and staff, have made no representation, guarantee, or warranty, expressed or implied, that the data—financial, patient, payor and physician specific information—are error-free, or that the use of the data will avoid differences of opinion or interpretation, or disputes with those who use published reports or purchased data. PHC4, its agents and staff, will bear no responsibility or liability for the results of the analysis, or consequences of its use.

The authors would like to acknowledge the HCUP Data Partners that contribute to the Nationwide Inpatient Sample and the State organizations involved: http://www.hcup-us.ahrq.gov/db/hcupdatapartners.jsp

Funding: Dr. Schwarze receives funding from the Greenwall Faculty Scholars Program. The project described was supported by the Clinical and Translational Science Award (CTSA) program, previously through the National Center for Research Resources (NCRR) grant 1UL1RR025011, and now by the National Center for Advancing Translational Sciences (NCATS), grant 9U54TR0000427, and Dr. Barnato is receives funding from R01AG035112. These funding sources had no role in the design and conduct of the study; collection, management, analysis, or interpretation of the data; and preparation, review, or approval of the manuscript for publication.

Dr. Schwarze had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Footnotes

Potential Conflicts of Interest: The authors have no potential conflicts of interest to report.

References

  • 1.Khuri SF, Henderson WG, DePalma RG, Mosca C, Healey NA, Kumbhani DJ. Determinants of long-term survival after major surgery and the adverse effect of postoperative complications. Ann Surg. 2005 Sep;242(3):326–341. doi: 10.1097/01.sla.0000179621.33268.83. discussion 341–323. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Pearse RM, Harrison DA, James P, et al. Identification and characterisation of the high-risk surgical population in the United Kingdom. Crit Care. 2006;10(3):R81. doi: 10.1186/cc4928. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Weiser TG, Semel ME, Simon AE, et al. In-hospital death following inpatient surgical procedures in the United States, 1996–2006. World J Surg. 2011 Sep;35(9):1950–1956. doi: 10.1007/s00268-011-1169-5. [DOI] [PubMed] [Google Scholar]
  • 4.Semel ME, Lipsitz SR, Funk LM, Bader AM, Weiser TG, Gawande AA. Rates and patterns of death after surgery in the United States, 1996 and 2006. Surgery. 2012 Feb;151(2):171–182. doi: 10.1016/j.surg.2011.07.021. [DOI] [PubMed] [Google Scholar]
  • 5.Kwok AC, Semel ME, Lipsitz SR, et al. The intensity and variation of surgical care at the end of life: a retrospective cohort study. Lancet. 2011 Oct 15;378(9800):1408–1413. doi: 10.1016/S0140-6736(11)61268-3. [DOI] [PubMed] [Google Scholar]
  • 6.Hamel MB, Henderson WG, Khuri SF, Daley J. Surgical outcomes for patients aged 80 and older: morbidity and mortality from major noncardiac surgery. J Am Geriatr Soc. 2005 Mar;53(3):424–429. doi: 10.1111/j.1532-5415.2005.53159.x. [DOI] [PubMed] [Google Scholar]
  • 7.Reames BN, Ghaferi AA, Birkmeyer JD, Dimick JB. Hospital Volume and Operative Mortality in the Modern Era. Ann Surg. 2013 Dec 23; doi: 10.1097/SLA.0000000000000375. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Birkmeyer JD, Siewers AE, Finlayson EV, et al. Hospital volume and surgical mortality in the United States. N Engl J Med. 2002 Apr 11;346(15):1128–1137. doi: 10.1056/NEJMsa012337. [DOI] [PubMed] [Google Scholar]
  • 9.Finks JF, Osborne NH, Birkmeyer JD. Trends in hospital volume and operative mortality for high-risk surgery. N Engl J Med. 2011 Jun 2;364(22):2128–2137. doi: 10.1056/NEJMsa1010705. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Birkmeyer JD, Lucas FL, Wennberg DE. Potential benefits of regionalizing major surgery in Medicare patients. Eff Clin Pract. 1999 Nov-Dec;2(6):277–283. [PubMed] [Google Scholar]
  • 11.Neuman MD, Bosk CL. What we talk about when we talk about risk: refining surgery’s hazards in medical thought. Milbank Q. 2012 Mar;90(1):135–159. doi: 10.1111/j.1468-0009.2011.00657.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Moyer CA, Key JA. Estimation of Operative Risk in 1955. JAMA. 1956;160(10):853–855. doi: 10.1001/jama.1956.02960450035008. [DOI] [PubMed] [Google Scholar]
  • 13.HCUP Clinical Classifications Software (CCS) for ICD-9-CM. [Accessed April 22, 2011];Healthcare Cost and Utilization Project (HCUP). 2000–2003. www.hcup-us.ahrq.gov/toolssoftware/ccs/ccs.jsp.
  • 14.HCUP Clinical Classifications Software for Services and Procedures. [Accessed April 22, 2011];Healthcare Cost and Utilization Project (HCUP) 2004 http://www.hcup-us.ahrq.gov/toolssoftware/ccs_svcsproc/ccssvcproc.jsp.
  • 15.Elixhauser A, Steiner C, Palmer L. Clinical classifications software (CCS) Rockville, MD: U.S. Agency for Healthcare Research and Quality; 2011. [Google Scholar]
  • 16.Agency for Healthcare Research and Quality R, MD. HCUP Nationwide Inpatient Sample (NIS) Healthcare Cost and Utilization Project (HCUP). 2001–2006. www.hcup-us.ahrq.gov/nisoverview.jsp. [PubMed]
  • 17.Cykert S, Dilworth-Anderson P, Monroe MH, et al. Factors associated with decisions to undergo surgery among patients with newly diagnosed early-stage lung cancer. JAMA. 2010 Jun 16;303(23):2368–2376. doi: 10.1001/jama.2010.793. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Sudore RL, Yaffe K, Satterfield S, et al. Limited literacy and mortality in the elderly: the health, aging, and body composition study. J Gen Intern Med. 2006 Aug;21(8):806–812. doi: 10.1111/j.1525-1497.2006.00539.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Robinson TN, Wu DS, Sauaia A, et al. Slower Walking Speed Forecasts Increased Postoperative Morbidity and 1-Year Mortality across Surgical Specialties. Ann Surg. 2013 Oct;258(4):582–590. doi: 10.1097/SLA.0b013e3182a4e96c. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Makary MA, Segev DL, Pronovost PJ, et al. Frailty as a predictor of surgical outcomes in older patients. J Am Coll Surg. 2010 Jun;210(6):901–908. doi: 10.1016/j.jamcollsurg.2010.01.028. [DOI] [PubMed] [Google Scholar]
  • 21.Finlayson E, Birkmeyer JD. Research based on administrative data. Surgery. 2009 Jun;145(6):610–616. doi: 10.1016/j.surg.2009.03.005. [DOI] [PubMed] [Google Scholar]
  • 22.Lee TH, Marcantonio ER, Mangione CM, et al. Derivation and prospective validation of a simple index for prediction of cardiac risk of major noncardiac surgery. Circulation. 1999 Sep 7;100(10):1043–1049. doi: 10.1161/01.cir.100.10.1043. [DOI] [PubMed] [Google Scholar]
  • 23.Goldman L, Caldera DL, Nussbaum SR, et al. Multifactorial index of cardiac risk in noncardiac surgical procedures. N Engl J Med. 1977 Oct 20;297(16):845–850. doi: 10.1056/NEJM197710202971601. [DOI] [PubMed] [Google Scholar]
  • 24.L’Italien GJ, Paul SD, Hendel RC, et al. Development and validation of a Bayesian model for perioperative cardiac risk assessment in a cohort of 1,081 vascular surgical candidates. J Am Coll Cardiol. 1996 Mar 15;27(4):779–786. doi: 10.1016/0735-1097(95)00566-8. [DOI] [PubMed] [Google Scholar]
  • 25.Menke H, Klein A, John KD, Junginger T. Predictive value of ASA classification for the assessment of the perioperative risk. Int Surg. 1993 Jul-Sep;78(3):266–270. [PubMed] [Google Scholar]
  • 26.Wolters U, Wolf T, Stutzer H, Schroder T. ASA classification and perioperative variables as predictors of postoperative outcome. Br J Anaesth. 1996 Aug;77(2):217–222. doi: 10.1093/bja/77.2.217. [DOI] [PubMed] [Google Scholar]
  • 27.McAdams-DeMarco MA, Law A, Salter ML, et al. Frailty and early hospital readmission after kidney transplantation. Am J Transplant. 2013 Aug;13(8):2091–2095. doi: 10.1111/ajt.12300. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Surgical Risk Calculator. [Accessed October 13, 2013];ACS NSQIP Surgical Risk Calculator. 2013 Available at: http://www.riskcalculator.facs.org/PatientInfo/PatientInfo.
  • 29.Witkin BR, Altschuld JW. Planning and conducting needs assessment: A practical guide. Thousand Oaks, CA: Sage Publications, Inc; 1995. [Google Scholar]
  • 30.Hsu C, Sandford BA. The Delphi Technique: Making Sense of Consensus. Practical Assessment Research & Evaluation. 2007;12(10):1–8. [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

eTables
NIHMS677524-supplement-eTables.docx (106.2KB, docx)

RESOURCES