Table 1.

Detailed information on each outcome measure

Score name	Objective of instrument	Included variables	Range of score	Weight of variables	Length of follow-up	Data used to calibrate	Relevance of variables	Validation	Examples of application
Days Alive and Out of Hospital21 (Canada)	Easily calculated quality measure of patient outcomes after surgery. Associated with patient characteristics, surgical complexity, in-hospital complications and longer-term outcomes.	Mortality Duration of hospital stay Readmission within 30 days.	0–30 Higher score indicates worse outcome as reflects longer duration of hospital stay. Patient death automatically scores as 0 Every point on scale considered important.	All variables combined with equal weighting.	Looks at listed variables between date of the index surgery and 30th postoperative day.	Association with characteristics: Patient (age, sex, co-morbidities). Hospital (academic status, total bed number, surgical volume at each institution). Surgical (procedure types, procedure duration).	Duration of hospital stay is a surrogate for quality and speed of recovery after surgery. Readmission is a surrogate for postoperative complications.	Validated in sample of 540 072 patients. Construct validity assessed. Hierarchical multivariable quantile regression model used to assess association of patient, surgical and hospital characteristics with score.	Perioperative clinical trials. Major elective non-cardiac surgical procedures.
I-FEED22 (Canada)	An outcome measure for postoperative ileus (prolonged absence of bowel function after surgery).	Severity of limitation of oral intake, vomiting, physical examination, symptom duration. 0–2 considered normal 3–5 considered intolerance 6+ considered dysfunction	0–6+ Higher score indicates worse GI dysfunction.	Intake: 3 points Nausea: 3 points Emesis: 3 points Exam: 3 points Duration of symptoms: 2 points.	Daily scores generated up to hospital discharge or day 7.	Confirmed four main hypotheses, association of score with: Longer time to GI motility Longer length of hospitalization More complication patient-reported recovery.	Classification developed by expert consensus to account for clinically relevant aspects of GI recovery, factors that influence management and levels of dysfunction associated with increased complications and cost.	Validated in sample of 128 patients. Construct validity for score to measure the construct of postoperative GI recovery was tested according to the four main hypotheses (GI motility, length of stay, complications, patient-reported quality of recovery).	Laparascopic colorectal surgery
Hospital stay, readmission and mortality rate3,23–25 (United States)	Easily calculated quality measure of patient outcomes after surgery.	Mortality Duration of hospital stay Readmission within 30 days Elective or emergent status	0–10 Higher score indicates worse outcome. Every point on scale considered important.	LOS (6 categories): 5 points Mortality rate: 5 points Readmission: 1 point	30 days	Correlation between the hospital-level complication rate and HARM scores used for internal validation.	Previous studies have shown the value of each individual HARM component as a measure of quality.	Validated in a sample of 81 622 colectomy discharges; of which 44% were emergent. Logistic regression showed that the complication rate was significantly associated with each HARM component. Reliability and validity assessed through bootstrapping correlation coefficients.	Colorectal surgery
Surgical complication OUTcome26 (United States)	Easily calculated quality measure of patient outcomes after surgery with focus on the severity of postoperative complications.	Complication severity Mortality 0 indicates no complications 100 indicates death	Minimum score 0, no upper limit Higher scores indicate worse outcome.	Points based on severity (‘grade’) of complication. No pre-set guidelines for each grade, that is points assigned to a particular grade vary by type of complication.	Score generated during each day of hospitalization if a complication meeting criteria arose.	Not described	Classification developed by expert consensus. A panel of surgeons assigned a SCOUT severity score for each grade of complication.	Trialled in a sample of 9000 general and vascular surgical cases. Statistical validation methods unclear.	General and vascular surgery
Mortality, transfer, length of stay27 (German)	Easily calculated quality measure of patient outcomes after surgery.	Postoperative mortality rate. Postoperative transfer to another hospital. Duration of hospital stay above predefined duration.	Positive (MTL+) or negative (MTL−). Positive MTL indicates worse outcomes.	Any one of the variables occurring results in an overall positive score.	Length of follow-up varies, but 30 days or 22 days suggested.	Association with characteristics: Patient (such as ASA, complications, age, tumour-dependent factors).	MTL+ has a high correlation with existing patient risk factors and strongly correlated with occurrence of postoperative complications.	Trialled in sample of 14,978 patients undergoing colorectal resection. MTL rates calculated and compared to well established single outcome measures using multivariate regression analysis. For each outcome measure, postoperative complications were tested regarding predictability	Colorectal cancer surgery
Textbook Outcome28 (Netherlands)	Composite quality measure of clinical process indicators that measures if a series of predefined desirable short-term health outcome indicators are met	Different positive short-term health outcomes based on procedure.	Positive or negative. Positive TO indicates better outcomes.	All variables must occur to result in an overall positive score.	Not applicable. Review of existing database.	Indicators of good clinical outcome selected after literature review. Justification for each selection provided in Appendix S1.	Unsure about extent that medical complexity and co-morbidity rate variation influence TO scores.	Trialled in sample of 45,848 patients undergoing range of surgical procedures. To assess impact of clinical indicators where the total TO was not met (TO = 0), the specificity of each indicator was determined. A pairwise comparison between TO score on hospital level and score per indicator was performed per treatment.	Gastrointestinal diagnoses requiring endoscopic intervention. Can be adapted for any procedure.
Postoperative Morbidity Index29 (United States)	A measure used to estimate both the overall frequency and severity of complications in a postoperative population.	Complication severity and number of patients. Score is sum of complication severity weights divided by total number of patients.	0–1.00 Higher score indicates worse outcome. Score of zero indicates that no patient had a postoperative complication. Score of 1.00 indicates that every procedure in the series resulted in a postoperative death. Every point on scale considered important	Each complication individually weighted based on severity. Severity calculated by expert consensus.	Not applicable. Review of existing database	Relevant complications selected through the validated and commonly used ACS-NSQIP system.	Severity of complications assigned using the validated Accordion Severity Grading System.	Trialled in sample of 655 patients undergoing distal pancreatectomy. Standard deviation of score was calculated as the weighted mean of standard deviations of institutional score values. Patient severity weight (0–1.00) was used as the dependent variable in regressions seeking correlates to score. Categorical variables were tested using two-sided independent sample t tests and ANOVA. For continuous variables, univariate linear regression was used.	Distal pancreatectomy
Therapeutic Intervention Scoring System30 (Germany)	A comprehensive outcome measure for postoperative patients in surgical ICU. Intermediate TISS score available for postoperative patients not in ICU.	Comprehensive list of variables including basic activities of care, ventilatory and renal support, cardiovascular support, and neurologic, interventions and metabolic support.	Varies	Each component in the score given equal weighting.	Scores calculated during hospital stay until discharge.	Initial simplified TISS28 developed based on analysis of 10 079 ICU records	An increased level of therapeutic activities at the end of ICU stay is associated with worse hospital outcome; 21.4% of patients with TISS of 20 or greater on discharge died subsequently during hospital stay.	Trialled in sample of 1808 patients in a surgical ICU. Some statistical tests not applied to avoid arbitrary significant results based on the large number of cases rather than on clinically relevant differences. Stepwise logistic regression analysis applied to evaluate score on the day of admission for prediction of hospital mortality rate.	Primarily used in an ICU setting where connected monitors can collect data.
Patient Quality Score31 (United States)	A comprehensive outcome measure assessing adherence to a comprehensive set of perioperative process-based Quality Indicators (QIs).	Comprehensive outcome list examining process-based QIs and complications: Prophylactic antibiotics, postoperative euglycaemia, prophylactic venous thromboembolism therapy, central venous line, urinary catheter, postoperative ambulation, medication list, pressure ulcer risk assessment, oral intake documentation, surgical safety checklist.	1–100% The patient quality score was calculated for each patient as the number of QIs passed divided by the number of QIs for which each patient was eligible.	Unclear	Not applicable. Review of existing database.	A Delphi consensus survey was used to determine QIs most relevant to the patient population. Inter-rater agreement was assessed for each QI using per cent agreement and the AC1 statistic.	Not described	Trialled in sample of 273 patients undergoing abdominal surgery. A Poisson regression used to test for association between patient quality score and occurrence of complications, which was adjusted for other patient characteristics. Poisson regression revealed that as quality score increased, incidence of postoperative complications decreased. Sensitivity analysis revealed that association was likely driven by postoperative ambulation QI.	Elective major abdominal operations.
DIMICK et al. score32 (United States)	A composite quality measure that incorporates information from multiple quality indicators to optimally predict ‘true’ risk-adjusted morbidity rate for each operation.	Morbidity rate, including morbidity rate with other related procedures. Duration of hospital stay. Re-operation rate	Hospitals ranked based on composite quality measures into 1-star, 2-star or 3-star rating.	The weight on each quality indicator is determined for each hospital to minimize the expected mean squared prediction error, using an empirical Bayes methodology. Weight based on the hospital-level correlation of each quality indicator with the mortality rate, and the reliability with which each indicator is measured.	Not applicable Review of existing database.	Calculated the correlation of each individual quality indicator with the mortality rate and calculated the average reliability of the standardized mortality rate and complication ratios for each procedure.	Adding risk-adjusted morbidity rates with ‘other’ procedures enhanced the reliability of hospital performance assessment. The ability to ‘borrow’ signals from these other operations reflects the presence of shared structure and process that lead to better outcomes.	Validated in a sample of patients undergoing aortic valve replacement. Estimated random-effect logistic models of mortality rate at the patient level, controlling for the same patient covariates. The random-effect logistic model used. Constructed an R-squared statistic for the 2002 to 2003 forecast equal to the amount of variation being predicted by the composite quality measure as percentage of all hospital-level variation.	Ventral hernia repair. Colon resection. Lower extremity bypass surgery. Abdominal aortic aneurysm repair. Aortic valve replacement.

GI, gastrointestinal; LOS, length of (hospital) stay; HARM, Hospital stay, Readmission, and Mortality; SCOUT, Surgical Complication OUTcome; MTL, Mortality, Transfer, Length-of-stay; TO, Textbook Outcome; NSQIP, National Surgical Quality Improvement Program; TISS, Therapeutic Intervention Scoring System; I-FEED, Intake, response to nausea treatment, Emesis, Exam, and Duration; ACI, first-order agreement coefficient; DIMICK, Dimick et al. 2013.