Patient-Specific Surgical Outcomes Assessment Using Population-Based Data Analysis for Risk Model Development

Abstract

Optimal surgical planning and decision making surrounding surgical interventions requires patient-specific risk assessment which incorporates patient pre-operative clinical assessment and clinical literature. In this paper, we utilized population-based data analysis to construct surgical outcome predictive models for spinal fusion surgery using hospital, patient and admission characteristics. We analyzed population data from the Nationwide Inpatient Sample (NIS) –a nationally representative database– to identify data elements affecting inpatient mortality, length of stay, and disposition status for patients receiving spinal fusion surgery in the years 2004–2008. In addition to outcomes assessment, we want to make the analytic model results available to clinicians and researchers for pre-operative surgical risk assessment, hospital resource allocation, and hypothesis generation for future research without an individual patient data management burden. Spinal fusion was the selected prototype procedure due to it being a high volume and typically inpatient procedure where patient risk factors will likely affect clinical outcomes.

Introduction

Clinical information on surgical procedure outcomes such as inpatient mortality, hospital length of stay and the anticipated discharge disposition location of the patient provides clinicians, patients, families and facilities actionable data to optimally plan for a patient’s surgical procedure. These three outcomes are associated with the patient peri-operative evaluation and risk assessment and have been frequently evaluated in the literature. For example, patient race is sometimes considered an access barrier for healthcare services which can be reflected in extended patient length of stays; which may subsequently trigger discharge delays. These discharge delays as reflected in the time between when a patient is medically ready for discharge to another form of care (e.g. nursing facility) and when the patient is actually discharged¹ may reflect suboptimal care planning and delivery. If advanced knowledge of the patient’s disposition expectations were available, it might influence surgery scheduling to facilitate more efficient care delivery. With available planning data on patient risk factors, clinicians might also advise patients to reschedule their surgery around needed rehabilitation or skilled nursing facility availability that better meets their recovery requirements. Inpatient mortality has been selected by the Agency of Healthcare Research and Quality (AHRQ) as an Inpatient Quality Indicator (IQI) making it a focus of hospital quality efforts². Information availability ultimately may also influence the clinician’s decision to offer surgery or the patient’s decision to pursue a surgical procedure if the patient has high-risk medical co-morbidities³.

Even when clinical risk factors are known, the quality and availability of published data relevant to a particular patient may be insufficient to guide definitive procedure outcome assessment⁴. While randomized clinical trials are considered the highest quality of clinical evidence, in many cases the reported outcomes of surgical interventions do not necessarily apply to an individual patient within the population due to limitations of study design, implementation issues, and patient selection criteria in the studied populations⁵. Ultimately, clinicians need to answer the question that patients pose: “For me in particular, what are the expected outcomes of this surgical intervention?”⁵. To address this question, the provider must manage the “individual knowledge” conundrum pointed out by Pantazi⁶ in optimization of their own patient-specific decision making. This need can be addressed by using population based analysis made possible by the recent exponential growth in digital clinical data. However, this work has historically been challenging due to the lack of both the available clinical data and corresponding computing power⁷.

It is our hypothesis that the knowledge gained from analyzing nationally representative data can greatly benefit care delivery and outcome⁸. Population-based data sets have high degrees of generalizability and robust tools are available to examine and identify quality of careconcerns⁹ and can enhance practice guideline development¹⁰. The application of population data to improve quality of care seems appealing, but has many challenges especially when raw population data is queried for risk assessment. In Table 1, we have identified several challenges which we encountered using a population-based clinical decision support system for outcome assessment¹¹.

Table 1.

Challenges of Population based Systems for Outcomes Assessment

Challenge	Description	Impact
Cost and Workflow Interruption	Infrastructure and Maintenance Cost: Substantial computing analysis power is needed to provide timely response to users’ inquiries especially when several data elements are used to generate patient-specific surgical outcome assessment. During system maintenance, the system may fail to perform its primary function. This can be due to database updates or other maintenance work.	Adds extra financial, technical, and managerial burden to the clinical setting. Interrupts clinical workflow which may create substantial concerns for patient safety, workflow efficiency, and user satisfaction.
Data Issues	Data Deficiency and Inconsistency: Data quality issues (missing or inconsistent data) may degrade the accuracy of procedure outcome assessment. The population data may not have sufficient granularity for a clinician’s inquiry about a specific type of cases. Limitations on raw data disclosure due to data agreements and HIPAA regulations	A patient-specific procedure outcome assessment may not be possible. Limits the ability of cohort identification

In this paper, we utilized outcome predictive modeling to overcome the challenges identified above and created an estimate of surgical outcomes optimally tailored to individual groups of patients and hospital sites using significant data elements that have been cleansed for potential data and security issues. Outcome predictive modeling is a process by which population data is analyzed to create statistical or probabilistic models to determine the likelihood of future outcomes¹². Predictive modeling creates access to underlying clinical data patterns which do not exactly reflect the underlying patient data thereby masking patient identities to address data agreements and HIPAA rules. Light-weight outcome predictive models can assess surgical risks with minimal computing power and workflow interruptions as compared to traditional clinical data marts/data warehouses which typically require directly access to underlying data for analysis.

We applied population-based analysis using NIS data within the context of spinal surgical procedures to create patient-specific outcome predictive models to assess inpatient mortality, length of stay, and disposition status. Spinal surgeries present substantial clinical management challenges and represent a high volume procedure which is typically completed as an inpatient procedure with potential for substantial surgical complications. In addition, the use of spinal-fusion surgery in the U.S. is rapidly increasing¹³. Despite this trend, there is a paucity of literature addressing perioperative outcomes, particularly for specific patient comorbidities¹⁴. Spinal surgery has high financial costs and is often associated with increased bleeding and severe postoperative pain¹⁵. It is also one of the most invasive orthopedic surgeries performed on children¹⁶. Given the significance of the procedure, this clinical area was chosen as a clinical prototype for this study.

Background

Surgical risk assessment is generally part of the patient’s preoperative assessment to facilitate medical management prior to surgical interventions, which represent a highly stressful and impactful healthcare interaction. With 45 million procedures occurring each year, there is a large and growing need for pre-procedural evaluations¹⁷. Preoperative evaluations have been shown to reduce perioperative delays and address medical problems, especially new problems that effect surgical timing and potential surgical cancellations^18–20. Preoperative assessments also provide clinical interventions to optimize medical conditions prior to and after a surgical intervention and help avoid cancellations, which can have substantial economic impacts^21,22. In addition, the outpatient preoperative evaluation can reduce inpatient care costs^23,24.

Assessment prior to surgery focuses on several key areas including cardiopulmonary status and hematologic considerations as well as medication management to assess risk and manage patient co-morbidities^25,26. Once the patient assessment is completed, consultation results need to be communicated to surgical and anesthetic providers²⁷ to address surgical and anesthetic concerns and to optimize planning for the procedure²⁸.

The preoperative evaluation is challenging since it is a multispecialty assessment with general practitioners, anesthesia, surgery, nursing, and pharmacy providing input into the surgical planning process²⁸. In many cases the primary care provider completes the preoperative evaluation and communicates management recommendations to the surgical team for management in addressing or optimizing pre-existing or new medical problems. Given the challenge of understanding and utilizing clinical preoperative needs, the creation of additional clinical risk identification tools can better represent patient risk and facilitate procedural planning. It has been previously noted that the overall surgical outcome is positively correlated with a facility’s ability to manage complications when they occur²⁹. If the surgical team is informed of patient-specific potential risks, aggressive clinical monitoring and other interventions can help identify potential events quickly to help reduce the risk of adverse outcomes for the patient.

Methods

Data Source

Population data were obtained from NIS, which is the largest all-payer inpatient care database in the United States maintained by the Healthcare Cost and Utilization Project (HCUP) and sponsored by the Agency for Healthcare Research and Quality (AHRQ)³⁰. It contains data from approximately 8 million hospital stays each year sampled from approximately 1,000 hospitals to approximate a 20-percent stratified sample of U.S. community hospitals. Inpatient stay records in the NIS database include clinical and resource use information typically available from discharge abstracts. This study was exempt from institutional review board approval since the data utilized were sufficiently de-identified.

Inclusion Criteria

A total of 356,857discharges were identified from NIS for patients that underwent primary fusion spinal surgical procedures with International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) codes: 81.00 - 81.09, 81.30- 81.39, 81.62 – 81.64, 84.51, 84.52, 84.62 - 84.64in the U.S. from 2004 through 2008. The ICD 9 CM codes were identified after consultation with medicine and surgical procedure experts and through a review of surgical outcome literature^31,32.

Data Preprocessing

The list of independent data variables (Table 2) abstracted from NIS included patient demographic and co-morbidity information in addition to hospital and admission characteristics.

Table 2.

The list of independent data variables abstracted from NIS for population based analysis

Patient Characteristics:
Demographics:	Age, Sex (male, female), Race (white, black, Hispanic, others)
Number of co-morbidities:	0, 1, 2, ≥3
Co-morbidities:	Acquired Immune Deficiency Syndrome (AIDS), Alcohol Abuse, Deficiency Anemia, Rheumatoid Arthritis, Blood Loss Anemia, Congestive Heart Failure, Pulmonary Disease, Coagulopathy, Depression, Diabetes, Drug Abuse, Hypertension, Hypothyroidism, Liver Disease, Lymphoma, Fluid and Electrolyte Disorders, Metastatic Cancer, Neurological Disorders, Obesity, Paralysis, Peripheral Vascular Disorders, Psychoses, Pulmonary Circulation Disorders, Renal Failure, Solid Tumor Without Metastasis, Valvular Disease, Weight Loss.
Hospital Characteristics:
Annual Volume:	Very low, Low, Medium, High, Very High
Region:	Northeast, Midwest, South, West
Location:	Rural, Urban
Bed-size:	Small, Medium, Large
Teaching Status:	Yes, No
Admission characteristics:
Weekend Admission:	Yes, No
Elective Admission:	Yes, No
Discharge Year:	2004, 2005, 2006, 2007, 2008

The co-morbidities measures reported by Elixhauser et al³³ were computed using AHRQ comorbidity software³⁴. In addition, we calculated the number of comorbidities per admission as it might influence inpatient mortality. Patient race was categorized into white, black, Hispanic and other. The annual hospital volume was calculated and included as a variable based on prior studies suggesting its usefulness in predicting surgical outcomes^35,36. Hospital volume was specifically determined using the unique hospital identifier for each institution provided by NIS. We categorized the annual hospital volume into very low (<40 cases), low (40 to 99 cases), medium (100 to 199 cases), high (200 to 399), and very high volumes (>400). A number of binary meta-variables and derived variables were also included to facilitate the analysis step. Missing values of any of the studied data elements (<1% of the data) were replaced using an expectation-maximization (EM) analysis for continuous variables and frequencies analysis for categorical variables³⁷.

Predictive Model Construction

All data analyses were performed using SAS software (version 9.2; SAS Institute, Inc, Cary, NC). Descriptive statistics for all variables were conducted. One predictive model was created per assessed outcome. The inclusion method to incorporate variables into the predictive model is carried out in a forward stepwise manner, testing for the statistical significance with variable inclusion at each stage requiring p-value to enter of 0.05. Finally, these models were validated internally and validation measures reported. The literature shows that internally validated clinical decision support systems are beneficial when deployed clinically³⁸. For the validation step, we used the SAS PROC SURVEY method to randomly divide the set of identified admissions into two mutually exclusive datasets: the training set (80% of admissions, n = 285,485) and the validation set (20% of admissions, n = 71,372). Both sets were generated once and used to create and validate the three models.

Inpatient Mortality Predictive Model

We used logistic regression (SAS procedure proc logistic) to predict inpatient mortality (i.e. the probability of patient death during the hospitalization where the surgical procedure was performed). The training set used to create the model has 693events (i.e. deaths) which is large enough to meet the recommended value for minimum Events per Variable (EPV) to maintain our statistical model^39,40.

Length of Stay Predictive Model

We used multivariate linear regression modeling (SAS procedure proc reg) to estimate the length of stay for patients admitted for spinal fusion surgery. The length of stay is a continuous variable measured in days with NIS providing a calculated patient length of stay per admission by subtracting the admission date from the discharge date. The exploratory analysis prior to the modeling step indicated that patient length of stay is not normally distributed. It follows a distribution that is skewed to the right (mean=3.9, median=3). Thus, it was transformed using its natural logarithm to provide an appropriately normal distribution for modeling as indicated by a quantile-quantile plot.

Disposition Status Predictive Model

We use SAS procedure proc logistic with option link=glogit to fit a multinomial logistic regression model to identify the variables associated with disposition to nursing facility or Home Health Care (HHC) using discharge to home (routine) as the reference level. The NIS database also includes patients who were discharged to short-term hospital or discharged Against Medical Advice (AMA). We excluded admissions pertaining to those two types as they represent <0.6% of the training set used to create the disposition status predictive model.

Results

The inpatient mortality rate for the entire cohort was 0.3%. The mean and median patient length of stay were3.9and 3 days respectively, 77.4% of patients were discharged to home, 12.5% discharged to another care facility, and 9.5% were discharged to Home Health Care. Age followed a normal distribution (Shapiro-Wilk test, p<0.01, mean=52 y, StdDev =15 y). Fifty-four percent of patients were women and60.5% were white. About 34.3% of patients had no co-morbidities. The most frequent co-morbidity among the patients was hypertension (40%). The majority of patients had elective admissions (87.3%) and were treated at hospitals with large bed-size (65.3%). Patients also tend to choose teaching hospitals (55.1%) for spinal fusion. The significant predictors (alpha =0.05)and their association with the inpatient mortality and the log transformation of length of stay are presented in Table 3.

Table 3.

Significant Predictors of inpatient mortality and the log transformed length of stay after Spinal Fusion

Significant Predictors	Training	Validation	Inpatient Mortality		Log-LOS
	n=285,485 (80%)	n=71,372 (20%)	Estimate *	OR [95% CI]	Estimate **	[95%CI]
Age			0.530	1.07 [1.05, 1.08]	0.022	[0.02, 0.03]
Sex (Female)	152,907	40,661	−0.617	0.54 [0.47, 0.62]	0.055	[0.05, 0.06]
Teaching Hospital (Yes)	157,822	38,906	0.575	1.78 [1.52, 2.08]	0.165	[0.160, 0.170]
Weekend Admission (Yes)	8,175	992	0.572	1.77 [1.45, 2.17]	0.427	[0.41, 0.44]
Elective Admission (Yes)	245,870	65,034	−1.321	0.27 [0.23, 0.31]	−0.435	[−0.44, −0.43]
Race
White	171,661	44,352	−0.204	0.82 [0.71, 0.94]	−0.045	[−0.05, −0.04]
Black	16,168	3,515	Not Significant		0.029	[0.02, 0.04]
Patient Co-morbidities
AIDS	152	34	1.883	6.58 [2.33, 18.57]	Not Significant
Congestive Heart Failure	4,329	1,246	1.110	3.04 [2.42, 3.80]	0.297	[0.27, 0.32]
Pulmonary Disease	38,968	10,427	0.404	1.50 [1.24, 1.81]	0.541	[0.05, 0.06]
Coagulopathy	3,307	851	1.236	3.44 [2.69, 4.41]	0.347	[0.33, 0.37]
Depression	30,232	8,430	−0.484	0.62 [0.44, 0.86]	−0.065	[−0.08, −0.06]
Hypertension	111,360	32,295	−0.641	0.53 [0.44, 0.63]	−0.073	[−0.08, −0.06]
Fluid and Electrolyte Disorders	15,574	4,562	1.082	2.95 [2.46, 3.55]	0.517	[0.50, 0.53]
Metastatic Cancer	1,720	153	1.493	4.45 [3.40, 5.83]	0.556	[0.52, 0.59]
Neurological Disorders	8,870	2,170	0.853	2.35 [1.83, 3.01]	0.077	[0.06, 0.09]
Paralysis	6,523	1,088	1.067	2.91 [2.34, 3.61]	0.440	[0.42, 0.46]
Peripheral Vascular Disorders	3,892	1,246	0.380	1.46 [1.04, 2.06]	0.076	[0.06, 0.09]
Pulmonary Circulation Disorders	865	211	1.684	5.39 [3.76, 7.73]	0.424	[0.38, 0.46]
Renal Failure	3,169	880	1.302	3.68 [2.83, 4.77]	0.182	[0.15, 0.20]
Solid Tumor Without Metastasis	1,256	218	0.911	2.49 [1.66, 3.74]	0.278	[0.24, 0.31]
Weight Loss	1,565	252	1.429	4.18 [3.24, 5.39]	0.724	[0.69, 0.76]
Number of co-morbidities
0	100,960	21,259	−1.382	0.25 [0.18, 0.35]	−0.319	[−0.34, −0.29]
1	83,152	20,751	−0.417	0.66 [0.53, 0.82]	−0.195	[−0.21, −0.18]
2	55,257	15,550	Not Significant		−0.084	[−0.09, −0.07]
Hospital Annual Volume
Very low	9,120	2,025	Not Significant		0.105	[−0.01, 0.03]
Very High	116,075	31,022	−0.243	0.78 [0.67, 0.91]	−0.085	[−0.08, −0.07]
Hospital Region
Midwest	68,491	15,830	−0.354	0.70 [0.59, 0.84]	−0.043	[−0.05, −0.04]
Hospital Bed Size
Small	38,583	9,487	−0.356	0.70 [0.51, 0.90]	−0.031	[−0.04, −0.02]
Large	186,418	46,668	0.281	1.32 [1.10, 1.56]	0.097	[0.09, 0.10]
Discharge Year
2007	57,443	15,577	−0.353	0.70 [0.58, 0.85]	−0.067	[−0.07, −0.06]
2008	70,307	14,480	−0.278	0.76 [0.64, 0.90]	−0.098	[−0.10, −0.09]

OR: Odds Ratio, CI: Confidence Interval,

^*the logit estimates of the Logistic regression,

^**parameter estimate of linear regression

Table 4 presents the significant results (alpha =0.05) of the multinomial logistic regression for patient disposition after primary spinal fusion procedure.

Table 4.

Significant Predictors of Patient Disposition after Spinal Fusion Surgery

Significant Predictors	Disposition to a Care Facility versus Disposition to Home			Disposition to HHC versus Disposition to Home
	Coefficient*	OR	95%CI	Coefficient*	OR	95%CI
Age	0.081	1.08	[1.08, 1.09]	Not Significant
Sex	0.393	1.48	[1.45 1.52]	0.221	1.25	[1.22, 1.28]
Weekend Admission (Yes)	0.856	2.35	[2.21, 2.51]	0.288	1.32	[1.22, 1.42]
Elective Admission (Yes)	−0.986	0.37	[0.36, 0.39]	−0.187	0.83	[0.80, 0.87]
Hospital Location (Urban)	0.473	1.60	[1.50, 1.71]	Not Significant
Teaching Hospital (Yes)	0.171	1.19	[1.16, 1.22]	−0.210	0.81	[0.79, 0.83]
Race
White	Not Significant			0.182	1.20	[1.17, 1.24]
Black	0.406	1.50	[1.42, 1.58]	0.403	1.49	[1.41, 1.57]
Hispanic	0.255	1.29	[1.21, 1.38]	0.605	1.82	[1.72, 1.93]
Patient Co-morbidities
AIDS	0.524	1.69	[1.10, 2.60]	Not Significant
Alcohol Abuse	0.460	1.58	[1.46, 1.72]	Not Significant
Deficiency Anemia	0.550	1.73	[1.66, 1.81]	0.557	1.73	[1.64, 1.81]
Rheumatoid Arthritis	0.173	1.19	[1.12, 1.27]	0.111	1.12	[1.04, 1.20]
Chronic Blood Loss Anemia	0.720	2.05	[1.86, 2.27]	0.662	1.94	[1.74, 2.16]
Congestive Heart Failure	0.423	1.53	[1.42, 1.64]	0.242	1.27	[1.16, 1.38]
Coagulopathy	0.661	1.94	[1.78, 2.11]	0.403	1.49	[1.35, 1.64]
Diabetes	0.525	1.69	[1.55, 1.84]	0.274	1.31	[1.18, 1.45]
Drug Abuse	0.551	1.73	[1.57, 2.00]	0.269	1.30	[1.17, 1.44]
Hypertension	−0.189	0.83	[0.80, 0.86]	−0.103	0.91	[0.87, 0.95]
Lymphoma	0.344	1.41	[1.18, 1.69]	Not Significant
Fluid and Electrolyte Disorders	0.859	2.36	[2.26, 2.47]	0.580	1.78	[1.69, 1.88]
Metastatic Cancer	0.375	1.46	[1.29, 1.65]	0.701	2.00	[1.75, 2.29]
Neurological Disorders	0.566	1.76	[1.66, 1.87]	0.150	1.16	[1.09, 1.24]
Obesity	0.238	1.27	[1.21, 1.33]	Not Significant
Paralysis	1.773	5.89	[5.50, 6.31]	0.733	2.08	[1.91, 2.26]
Peripheral Vascular Disorders	Not Significant			0.140	1.15	[1.05, 1.25]
Psychoses	0.693	2.00	[1.85, 2.17]	0.271	1.31	[1.20, 1.43]
Pulmonary Circulation Disorders	0.480	1.62	[1.38, 1.89]	0.361	1.43	[1.19, 1.72]
Renal Failure	0.280	1.32	[1.22, 1.44]	0.142	1.15	[1.04, 1.27]
Weight Loss	0.990	2.69	[2.37, 3.06]	0.409	1.49	[1.27, 1.76]
Number of co-morbidities
0	−0.842	0.43	[0.39, 0.48]	−0.562	0.57	[0.51, 0.63]
1	−0.464	0.63	[0.59, 0.68]	−0.295	0.75	[0.69, 0.81]
2	−0.192	0.83	[0.79, 0.87]	−0.113	0.90	[0.85, 0.95]
Hospital Annual Volume
Very Low	0.244	1.28	[1.19, 1.37]	−0.202	0.82	[0.76, 0.88]
Low	0.147	1.16	[1.11, 1.22]	−0.212	0.81	[0.77, 0.85]
High	0.148	1.16	[1.13, 1.20]	Not Significant
Hospital Region
North	0.706	2.03	[1.95, 2.11]	0.808	2.22	[2.14, 2.31]
Midwest	−0.204	0.82	[0.79, 0.85]	−0.382	0.67	[0.66, 0.71]
Hospital Bed-size
Small	−0.469	0.63	[0.60, 0.65]	−0.314	0.73	[0.71, 0.76]
Medium	−0.173	0.84	[0.82, 0.87]	−0.200	0.82	[0.80, 0.85]
Discharge Year
2004	0.342	1.41	[1.36, 1.46]	0.164	1.17	[1.13, 1.22]
2005	0.284	1.33	[1.28, 1.38]	Not Significant
2006	0.145	1.16	[1.12, 1.20]	0.105	1.11	[1.07, 1.15]
2007	0.155	1.15	[1.13, 1.21]	0.177	1.19	[1.15, 1.23]

OR: Odds Ratio, CI: confidence Interval, LOS: Length of Stay

^*Coefficient = logistic regression coefficient.

The multivariate analysis for patient comorbidities (Table 3) demonstrated that inpatient mortality was significantly lower for patients with zero or one co-morbidity (OR=0.25 and OR=0.66). Pulmonary circulation disorders and AIDS comorbidities showed the highest impact on inpatient mortality when compared to other co-morbidities; the difference in log-odds for inpatient mortality is expected to change by 1.7 and 1.9 respectively, given the other variables in the model are held constant. Interestingly, the analysis showed significantly higher inpatient mortality for hospitals with large bed-size (OR=1.3) as well as teaching hospitals (OR=1.8) which may be partly due to patient selection since larger facilities are likely to have the capacity to manage higher risk patients.

Table 3 also shows the length of stay is prolonged for patients with co-morbidities as weight loss (+70%),metastatic cancer (+60%), fluid and electrolyte disorders (+52%), and paralysis(+40%). In addition, patients without any co-morbidities have a 31.9% reduction in the length of stay. The analysis also demonstrates that elective admissions tend to be shorter when compared to non-elective admissions given the other variables in the model are held constant. The same is also valid for patients discharged in years 2007 and 2008.

The results in Table 4 describe a series of logistic regressions, each of which models post-hospital disposition type - Care Facility or Home Health Care (HHC) or discharge to home. Table 4 shows that the majority of comorbidities significantly affect the likelihood of discharge to another care facility or to Home Health Care. Paralysis was found to be the co-morbidity with the highest impact on disposition to another care facility (OR=5.89) or to Home Health Care (OR=2.08) as compared to home dispositions. The results also show that fewer patients are disposed to another care facility in recent years (e.g. 2007, OR=1.15) than earlier years (e.g. 2004, OR=1.41) which may reflect improvements in surgical management or changes in practice with the implementation of clinical pathways.

Model Validation

Using the 20% validation data set, we assessed each model’s performance on unseen data. For the inpatient mortality model, this was done by examining the discrimination performance as measured by the area under the receiver operating characteristic (ROC) curve⁴² which is found to be 91.8%. For the length of stay model, the training and validation data sets had R² of 0.25; final coefficients can therefore be produced from a full data set (R² 0.25). Residuals were normally distributed, and there was no evidence of heteroskedasticity and the scatter plot of predicted and residual values adequately fit the data (R² [adjusted] = 0.26) which confirms that model assumptions had been satisfied. Finally, for the multinomial regression model of the patient disposition status was validated using the Lowess smooth of the deviance residuals versus predicted probabilities plot. The Lowess smooth is found to be approximately a horizontal line with zero intercept that suggests a goodness of the model fit.

Discussion

Population based analysis for this project utilizes data originating from healthcare providers (discharge data) to provide insight into surgical procedure outcome risk at the patient-level. The results can then be presented to healthcare providers caring for patients in the pre-operative setting to provide improved guidance and counseling. An overview of the closed-loop health information flow chart is illustrated in Figure 1.

Figure 1.

The closed loop health information flow chart

We utilized the test case of spinal fusion surgery to demonstrate the utility and validity of this approach in guiding practitioners in the pre-operative setting. We found that outcome predictive modeling was effective in identifying clinically significant variables that affect outcomes following inpatient surgical procedures using an underutilized data source (NIS). This approach also has the advantage of being a data derivative in that it does not obtain original data from a direct query of discharge data as was described in Table 1 with our first generation POET system.

As a use case, our findings support observations reported previously in the surgery literature that decreased length of stay and decreased mortality are strongly associated with high-volume hospitals⁴². Similarly, co-morbidity data and risk stratification are commonly used and must be incorporated into prognostic models of survival used to guide preoperative patient selection or preoperative patient counseling, such as outcomes used for prognostication associated with primary spinal surgery in the setting of spinal metastasis^35,43. As an additional point of face validity, inpatient mortality of 0.3% in our analysis was comparable to that reported by a national study examining selected spinal fusion procedures (0.25%)⁴⁴. While factors that are associated with different disposition status are often not examined with single institution studies including in the setting of spinal surgery^45,46, disposition status is a valuable factor for optimal surgical planning and may be beneficial to clinicians, facilities, and the patient’s family in proper preoperative planning and counseling for anticipated outcomes. In addition to identifying a number of co-morbidities associated with home healthcare and disposition to another care facility, both the elderly and African Americans were observed to also be more likely to require home healthcare or disposition to another care facility. While elderly patients expectedly require greater resources to make a sufficient recovery, the higher rate of African American home healthcare utilization and care facility use support the large body of literature associated with racial disparities and represent an area for early social work and case management to best utilize family and social support in the establishment of post-operative surgery and recovery plans¹.

We did not consider possible interaction effects among the set of studied data elements since the number of variables increased dramatically. This is another challenge of population-based analysis that we will consider in future work. The interpretation of the data is limited since the underlying data is retrospective and limited to the inpatient clinical setting with limited generalizability to other settings. Given the modeling was established with spinal surgery results, there is also limited ability to generalize the resulting models to other procedures, however, the population based approach could be replicated with other selected procedure types using the same analytic methods.

Despite the potential value to providers and patients, population-based analysis in support of patient-specific surgical outcomes assessment is not a commonly used decision-support tool to guide patients and clinicians regarding likely outcomes after surgical interventions. This can be due to the large number of data elements that are needed for the analysis to create patient-specific evaluations which requires significant data preprocessing, analysis and modeling time. Also, this data should typically be reviewed with an experienced clinician to give adequate context and interpretation to the information and to help identify correlations that may deviate the results of the outcome predictive modeling task.

Our next steps are to expand this work to a more generalizable and scalable framework and interface for providers. Additional evaluation of the approach will be completed by comparing the population-based model used in this study with actual clinical outcomes of patients. The use of other large scale datasets which provide complementary information such as NSQIP (National Surgical Quality Improvement Program)⁴⁷ and VASQIP (Veterans Affairs National Surgical Quality Improvement Program)⁴⁸ databases, two large surgical outcome registries used nationally and confined to the Veteran’s Administration respectively, or a medical chart review of patients with spinal surgeries could provide additional risk variable data needed to validate the model with actual clinical outcomes as this methodology is deployed in the clinical environment to aid preoperative assessment. With final validation, the computationally light-weight risk assessment model has substantial advantages in reducing requirements for computational power and data management overhead. We believe that the creation of our validated data model for the clinical risk of spinal surgery provides the first step in creating actionable data for future incorporation into a clinical system for surgical planning that could potentially be used by patients and families for self-assessment and procedure planning. Future work will include validation of risk models for other surgical procedures. Also, this creates an opportunity to utilize and implement both patient and provider-oriented assessments. Ultimately, the use of population-based analyses with large data sets provides an important mechanism to provide personalized clinical risk assessments and decision support data when there are knowledge gaps in existing knowledge bases (e.g., literature and guidelines) and a lack of specific information to help apply the literature to a particular patient.

Conclusion

This study implements and describes the analytic framework and model creation for a system aimed at providing timely and relevant information to inform patient-specific risk assessments in the pre-operative setting. Use of the NIS database with a use case of spinal fusion surgery provided robust models for outcomes following spinal surgery. Our findings support the expansion and ongoing iteration of this approach to improve information availability to patients and providers and to provide clear expectations and optimal preoperative counseling to patients about surgical procedure outcomes.