Assessing clinical discharge data preferences among practicing surgeons

Abstract

Background

It is believed that many postoperative patient readmissions can be curbed via optimization of a patient’s discharge from hospital, but little is known about how surgeons make the decision to discharge a patient. This study explored the criteria that surgeons preferentially value in their discharge decision-making process.

Materials and Methods

All surgical faculty and residents at a U.S. academic medical center were surveyed about the relative importance of specific criteria regularly used to make a discharge decision. Demographic and professional information was collected about each surgeon as well. A Kruskal-Wallis and Fisher’s exact test were used to describe one-way analysis of variance between groupings of surgeons. Ordered logit regressions were used to analyze variations across multiple different subgroups. Factor analysis was used to further characterize statistically relevant groupings of criteria.

Results

88 (49%) of the invited surgeons responded to the survey. Respondents reported statistically less reliance on common laboratory tests and patient demographics when making discharge decisions preferring vital signs, perioperative factors, and functional criteria. Surgeon-specific factors that influenced discharge criteria preferences included years of clinical education and gender. Factor analysis further identified subtle variations in preferences for specific criteria groupings based on clinical education, gender, and race.

Conclusions

Surgeons use a wide-range of clinical data when making discharge decisions. Typical measures of patient condition also appear to be used heterogeneously with a preference for binary rather than continuous measures. Further understanding the nature of these preferences may suggest novel ways of presenting discharge-relevant information to the clinical decision-makers to optimize discharge outcomes.

Background

Over 20% of patients are rehospitalized within 30 days of a hospital discharge, and it has been estimated that the costs for these readmissions are over $17 billion per year.¹ With the passage of the Patient Protection and Affordable Care Act of 2010, hospital readmission rates are rapidly becoming one of the critical health care quality metrics for U.S. hospitals.^2,3 Not only have increased mortality rates and increased health care costs been associated with hospital readmission, but hospitals and physicians are under increasing pressure to reduce hospital readmission rates due to pay-for-performance reimbursement schemes that refuse payment for related readmissions.^1,4,5 Given the threat of hospital readmissions to both the patient and the provider, reducing hospital readmissions is becoming a universally desirable goal. Although surgical readmissions have historically been less well explored in the literature, the current policies do not suggest that operative patients will be excluded from any payment modification scheme.⁶

Surgical readmissions can occur for a variety of reasons that include postoperative complications, aggravation of an underlying comorbidity, poorly coordinated transitions of care, and even unrelated coincidental illness.⁷ It has been suggested that many of these causes of surgical readmission can be curbed via optimization of a patient’s discharge from hospital.⁸ If a surgeon chooses to discharge a patient too early, patients will be at an increased risk of hospital readmission. If patients are discharged beyond the optimal time, precious healthcare resources are wasted on a patient whose care did not require it. It is known that such probabilistic decision-making is the most difficult and often least evidence-based.⁹ While predictive models have made good progress with determining what is the “right” time for discharge, the existing understanding of how best to provide such complementary information to clinical decision-makers is limited.^8,10 Within this context, a better understanding of how surgeons make discharge decisions could potentially lead to improved systems and policies that utilize scarce health care resources more effectively.¹¹ Before focusing on how to change biomedical informatics and related systems to accommodate surgeon behavior, we need to identify the criteria now used by surgeons when making everyday discharges. Currently, there is a dearth of knowledge on criteria used by surgeons to make hospital discharge decisions. Given the vast amount of clinical data available to a surgeon at the moment of discharge decision-making, surgeons must be using some basic heuristic framework to sort essential information from “clinical noise.” A thorough review of the literature demonstrates no prior papers assessing the relative importance of different clinical variables that are subjectively reported for surgical discharge decision-making. For example, do surgeons rely on laboratory tests when deciding whether to discharge a patient? Is the functional status of the patient given primary consideration? With the increasing interest in clinical software that presents data in more clinically useful and meaningfully predictive ways,^10,12 knowing what surgeons are going to want and expect to see will be critical to ensure rapid uptake of any decision-support tools. The following study was therefore undertaken to explore the clinical factors that surgeons preferentially value in their discharge decision-making by evaluating the self-reported responses to a confidential survey completed by surgical residents and faculty at a single U.S. academic medical institution.

Materials and Methods

Survey Instrument

The names and contact information of all current residents, fellows, and faculty at Emory University’s Department of Surgery were provided for use in initiating the study. Each participant was then contacted via automated email invitation to complete an online questionnaire without any form of inducement. Invitations, participant responses, data security, and data aggregation were all managed by an internet-based commercial survey tool operating behind a secured server at the authors’ home institution (Feedback Server 5, Data Illusion, Geneva, Switzerland). Automated reminder emails were targeted to prior non-responders on a weekly basis for up to one month.

The survey was designed by the authors – a number of whom have had formalized training in survey design. The first half of the survey instrument asked surgeons to give their opinion on the relative usefulness of a variety of clinical factors available when making postoperative discharge decisions. Items were thematically grouped into Perioperative Factors (e.g., type of surgery, intraoperative complications), direct clinical data (e.g., Vital Signs, Laboratory Tests, Functional Status), and Patient Demographics (e.g., age, race, socioeconomic status, education level). Respondents were asked to respond with a 5-point Likert scale with 1 indicating the least useful (“almost never”) and 5 indicating the most useful (“almost always”). The second half of the survey collected information on the responding surgeons’ prior experience and training levels, association with a subspecialty, and other demographic data. Any respondent who reported less than 25% of their time spent in clinical practice in surgery was excluded from the study. The entire survey is provided in Appendix 1. The survey was piloted with 3 senior medical students and 3 clinical faculty members who each had prior experience in survey design, but were not directly involved in the initial design of the survey instrument.

The above study design was evaluated by the Emory University Institutional Review Board and determined to be exempt from further review due to its focus on quality improvement and independence from current actual patient care.

Statistical Analysis

Descriptive analysis was first performed to illustrate characteristics of the participating cohort and the aggregated responses of the importance of specific clinical factors when making discharge decisions. The Kruskal-Wallis test was used in one-way analysis of variance between categorical groupings of clinical factors of individual surgeons. Where appropriate, a Bonferroni-adjusted p-value less than 0.0025 was used to confirm statistical significance with multiple testing. In addition, Fisher’s exact test was used to examine differences among surgeon subgroups and the relative importance of individual discharge criteria for each question included in the survey.

Because surgeons’ responses reveal their ordinal rankings of clinical variables ordered logit regressions were used to analyze the information on variations of ranking scores across different subgroups of surgeons (e.g., relative experience, degree of specialization) and thematic categories. This regression analysis preserved the aggregation within categories of clinical factors but disaggregated across observable characteristics of surgeons by estimating the effects on surgeons’ rankings of discharge criteria relative to surgeon-specific factors. Large variations in rankings of variables within each of the five clinical categories prompted the identification of latent underlying factors and the variables with high loadings within each factor via factor analysis. The number of retained factors was based on the criterion of eigenvalues greater than 1 and was confirmed by Bayesian information criterion.

Results

Study Population

88 (49%) out of an invited 178 survey subjects responded to the survey. One respondent was excluded from further analysis because they reported their area of medical expertise to be nonsurgical, and six others were excluded because less than 25% of their time was spent in a clinical setting.

Of the remaining 81 subjects, 64 (79%) were male, 57 (70%) were Caucasian, and 100% received their medical training at U.S. academic medical institutions. The average age of respondents was 39 years. 27 (33%) were residents, 8 were fellows (10%), and 46 (57%) were attending faculty. Surgical subspecialties represented included general and colorectal (35, 43%), cardiothoracic (14, 17%), vascular (2, 3%), transplant (8, 10%), surgical oncology (5, 6%), pediatric (5, 6%), critical care and trauma (4, 5%), plastic (1, 2%), and other (7, 9%).

Surgeons’ Preferences for Specific Categories of Data

Figure 1 graphically shows median Likert scores of surgeons’ responses for each survey variable with overlaid means and standard deviations. Table 1 shows results from the Kruskal-Wallis test for ranks aggregated across respondents and variables within the five listed thematic categories. The rank means ordered from highest to lowest are: vital signs (284) > functional criteria (266) > perioperative factors (240) > patient demographics (119) > lab tests (106). At the aggregated level, surgeons report to value Vital Signs the most, followed by Functional Criteria and Perioperative Factors with the difference between the latter two statistically insignificant. Also, the ranking of each of these three clinical categories is significantly higher than the rankings of both Laboratory Tests and Patient Demographics.

Figure 1. Median Likert Scale Rating, All Surveyed Clinical Factors (n=81).

Means follow variable names; standard deviations in brackets.

Table 1.

One-Way Analysis of Variance by Ranks (Kruskal-Wallis Test) for Clinical Factor Categories

Values reported are absolute differences in rank means of aggregated individuals’ Likert scale scoring.
	Perioperative Factors	Vital Signs	Common Laboratory Tests	Functional Criteria	Patient Demographics
Perioperative Factors	---
Vital Signs	43.86** (0.009)	---
Common Laboratory Tests	134.46** (0.000)	178.32** (0.000)	---
Functional Criteria	25.46 (0.083)	18.40 (0.159)	159.92** (0.000)	---
Patient Demographics	121.33** (0.000)	165.19** (0.000)	13.13 (0.238)	146.79** (0.000)	---
Obs	81	81	81	81	81
Rank Mean†	240	284	106	266	119

p-values in parentheses;

^**p<0.01,

^*p<0.05

^†A high rank indicates a particular clinical category is more preferred than a lower ranked category.

Fisher’s exact test was used to examine differences among surgeon subgroups and the relative importance of individual discharge criteria. Faculty and residents/fellows statistically differed in their reported use of discharge criteria for the following 7 variables: hematocrit/hemoglobin (p=0.001), medications administered (p=0.014), patient’s gender (p=0.020), patient’s body mass index (p=0.017), patient’s race (p=0.030), AST/ALT (p=0.032), and advancing diet appropriately (p=0.034). Weak statistical differences (p<0.10) also existed for: anesthesia mode (p=0.06), potassium (p=0.071) and return of bowel function (robf) (p=0.08). Surgical subspecialists statistically differed from general surgeons in rankings of the following 6 variables: hemoglobin/hematocrit (p=0.002), white blood cell count (p=0.004), blood urea nitrogen/creatinine (p=0.010), advancing diet appropriately (p=0.034), the indication for surgery (p=0.044), and medications administered (p=0.014). Weak statistical differences also existed for: functional status (p=0.084), fall risk (p=0.085), heart rate (p=0.093), anesthesia mode (0.096) and oxygen saturation (0.099).

The one-way analysis of variance reported in Table 1 reveals a significant bifurcation of the five categories of clinical factors into a more important group (Vitals, Functional Criteria, and Perioperative Factors) and a less important group (Patient Demographics and Laboratory Tests). But data analysis at such a high level of aggregation can mask differences within categories that: (a) may be important per se; and (b) may affect across-category ranks. In order to uncover more information about surgeons’ rankings of these discharge criteria, we proceed in steps to disaggregate with respect to identifiable characteristics of surgeons and latent factors within the five categories of clinical factors.

Table 2 reports results from five ordered logit regressions that control for identifiable characteristics of surgeons (e.g., years of education, specialist versus generalist, gender, race) while disaggregating the data with respect to each of the five thematic categories. None of these surgeon characteristics has a significant overall effect on the rankings reported in Table 2. For each thematic category, a surgeon’s probability of overall ranking a category with a mean of 4 or 5 on the five-point Likert scale is: 77% for Vital Signs, 62% for Functional Criteria, 61% for Perioperative Factors, 31% for Patient Demographics, and 24% for Laboratory Tests. Table 2 also demonstrates that specialists and faculty were not significantly different in their ranking of any category compared to others surveyed. In contrast, surgeons with more years of clinical education were less likely to highly rank Laboratory Tests (p=0.028). Female surgeons were more likely to highly rank Functional Criteria (p=0.010). Female surgeons and white surgeons were marginally less likely to highly rank Patient Demographics (p=0.051) and Laboratory Tests (p=0.054), respectively.

Table 2.

Ordered Logit Regression of Responses with Surgeon ID and Specialty specific random intercepts

EQUATION	VARIABLES	Perioperative Factors	Vital Signs	Laboratory Tests	Functional Criteria	Patient Demographics
surgeons’ response	female	0.639 (0.137)	0.707 (0.385)	−0.647 (0.148)	0.698** (0.010)	−0.418* (0.051)
	white	−0.469 (0.214)	0.213 (0.636)	−0.617* (0.054)	−0.001 (0.996)	0.018 (0.872)
	cliniceduc	−0.092 (0.417)	−0.032 (0.881)	−0.302** (0.028)	−0.125 (0.307)	0.037 (0.616)
	faculty	0.659 (0.190)	0.591 (0.526)	0.006 (0.992)	0.256 (0.700)	−0.391 (0.135)
	specialist	−0.272 (0.330)	−0.750 (0.149)	0.048 (0.900)	−0.120 (0.584)	−0.122 (0.558)
	Obs	475	480	1115	630	700
Pred. pop.-averaged probabilities of scores 4,5 95% Confidence Interval		0.608 [.59,.62]	0.773 [.76,.78]	0.238 [.22,.26]	0.618 [.60,.63]	0.313 [.30,.32]

pval in parentheses;

^***p<0.01,

^**p<0.05,

^*p<0.1

Factor Analysis of Surgeons’ Preferences

The initial display of responses in Figure 1 reveals large variations in rankings of variables within each of the five clinical categories. In order to explore the implications of this surgeon response pattern, for each of the five categories used in the preceding data analysis factor analysis was undertaken to identify latent underlying factors and the variables with high loadings within each factor. The 11 subcategories found via factor analysis are reported in Appendix 2. Table 3 reports the results of the ordered logit regression conducted with underlying subcategories and reveals systematic response patterns that are suppressed by aggregating all variables within the original five thematic categories.

Table 3.

Ordered Logit Regression with Latent Factors and Surgeon ID and Specialty specific random intercepts

Categories	Perioperative Factors		Vital Signs	Functional Criteria		Laboratory Tests			Patient Demographics
Underlying Factors	Periop Factors 1	Periop Factors 2	Vitals	Functional Criteria 1	Functional Criteria 2	Lab Tests1	Lab Tests 2	Lab Tests 3	Patient Demographics 1	Patient Demographics 2	Patient Demographics 3
female	1.044 (0.104)	0.561 (0.405)	0.707 (0.385)	0.985*** (0.001)	0.415 (0.302)	−0.912* (0.060)	−1.003 (0.345)	−0.730 (0.265)	−2.019** (0.013)	−0.390 (0.312)	−0.342 (0.456)
white	−0.725 (0.192)	−0.569 (0.452)	0.213 (0.636)	0.330 (0.120)	−0.704** (0.038)	−0.973 (0.133)	−1.402** (0.026)	−0.240 (0.601)	0.408 (0.413)	−0.080 (0.605)	0.045 (0.910)
cliniceduc	−0.441** (0.035)	0.093 (0.752)	−0.032 (0.881)	−0.110 (0.477)	−0.242 (0.146)	−0.493* (0.098)	−0.658 (0.123)	−0.213* (0.098)	−0.355 (0.122)	0.273** (0.017)	−0.174 (0.290)
faculty	1.600 (0.197)	0.593 (0.346)	0.591 (0.526)	−0.194 (0.779)	1.320 (0.183)	0.325 (0.822)	−0.683 (0.751)	−0.405 (0.408)	1.192 (0.251)	−1.131*** (0.008)	−0.203 (0.763)
specialist	0.161 (0.729)	−1.146* (0.099)	−0.750 (0.149)	−0.453* (0.096)	0.264 (0.417)	0.244 (0.725)	0.705 (0.680)	−0.497 (0.176)	−0.021 (0.971)	−0.265 (0.422)	−0.246 (0.580)
Observations	237	238	480	397	233	476	319	320	228	313	159
Pred. pop.-averaged Pr. of {4,5} [95% Conf. Interval]	0.360 [.34, .38]	0.848 [.84,86]	0.773 [.76,.78]	0.700 [.68,.72]	0.451 [.43,.47]	0.172 [.15,.19]	0.137 [.12,.16]	0.489 [.46,.52]	0.083 [.07,.09]	0.376 [.36,.40]	0.498 [.48,.52]

pval in parentheses;

^***p<0.01,

^**p<0.05,

^*p<0.1

In contrast to the similar probability ranking of a 4 or 5 seen in respondents in Table 2, Table 3 (last row) demonstrates that high-level aggregation of thematic categories obscures the effects of lower-level aggregations of variables. While the subcategory Perioperative Factors 2 has a high likelihood of a median ranking of 4 or 5 (85%), Perioperative Factors 1 has a low likelihood (36%). Similar statistically significant variations are seen between Functional Criteria 1 (70%) versus Functional Criteria 2 (45%), Laboratory Tests 3 (48%) versus Laboratory Tests 1 and 2 (17% and 14%), and Patient Demographics 2 and 3 (38% and 50%) versus Patient Demographics 1 (8%).

Table 3 also reveals differences between characteristics of surgeons and the discharge criteria that they think are most important. Surgeons with more years of clinical education rank Perioperative Factors 1 significantly less highly and Patient Demographics 2 significantly more highly. When controlling for other covariates, the estimated odds of a high rank of 4 or 5 for Perioperative Factors 1 decreases by 36% (p=0.035) per year of clinical education while the estimated odds of a high rank for Patient Demographics 2 increases by 32% (p=0.017) per year of clinical education. More years of clinical education also has a marginally significant effect on rankings of Laboratory Tests 1 (p=0.098) and Laboratory Tests 3 (p=0.098). Faculty surgeons are significantly less likely to highly rank Patient Demographics 2 than residents (p=0.008). Specialists rank Perioperative Factors 2 (p=0.099) and Functional Criteria 1 (p=0.096) marginally significantly lower than do generalists. Female surgeons are significantly more likely to rank Functional Criteria 1 highly (p=0.001) but significantly less likely to rank Patient Demographics 1 highly (p=0.013). Those surgeons self-reported as white are significantly less likely than self-reported non-white surgeons to give Functional Criteria 2 and Laboratory Tests 2 a high ranking (p=0.038, p=0.026).

Discussion

The current study is the first in a series of studies performed by the authors’ collaborative research group to explore how surgeons’ decision-making affects postoperative readmissions. By first exploring surgeons’ subjectively reported discharge criteria, we provide a perceptual baseline for our future studies that will compare discharge criteria that are subjectively reported to discharge criteria that outcomes analysis demonstrates to be statistically significant in predicting successful discharges versus 30-day readmissions.

In this study, practicing and trainee surgeons were surveyed to better understand how surgeons believe they make discharge decisions with a particular focus on the relative importance with which they view specific clinical and patient factors. The results reported above show some degree of cross-respondent consensus over a core set of “essential clinical data” providing some support for conventional wisdom that surgeons discharge patients relying on experience and heuristics rather than a methodical calculation using all readily available clinical data.

Although there are clearly favored clinical variables (19 out of 43 variables’ median score ≥4; e.g., successfully advancing diet, procedural complexity), no clear precipice exists between the most relied upon clinical variables and everything else available when viewed in their entirety (Figure 1). However, there are categories of clinical information that surgeons find more useful than others. With a very high degree of significance (p<0.001), surgeons report preferring the contextual procedure-specific and immediate data points (i.e., Vital Signs and Functional Criteria) to lagging indicators such as Laboratory Tests or popular holistic information such as Patient Demographics. This preference for “sentinel” data is logically consistent with clinical practice. For example, in the context of a discharge decision typical measurements of vital signs or the measureable functional ability of a patient would be more indicative of an unresolved need for continued hospitalization than a persistently troubling minor laboratory test value.

The results above also highlight that the thematic categories in which clinicians typically group data hide substantial variation in preference for particular subcategories within each category. For example, surgeons appear to prefer certain functional indicators (e.g., Functional Criteria 1: advancing diet, medication administration needs, return of bowel function, ambulatory status) to other collected data (e.g., Functional Criteria 2: pain score, functional status score, fall risk score). Similarly, there was a far greater preference for the laboratory’s complete blood count test plus measures of renal function (Laboratory Tests 3) than far more numerous laboratory test values available at time of discharge (Laboratory Tests 1 and 2).

Disaggregating the data by clinician characteristics also demonstrates fundamental differences in preferences for discharge decision-making criteria. Data preferences varied between residents and faculty members. Residents reported a greater reliance on laboratory testing data, which was confirmed by the trends associated with surgical education in ordered logit regression analysis. Without further study, it is impossible to determine if this difference is due to the evolution of surgical decision-making as one becomes more experienced or if it is more likely the result of the disproportionate amount of time that a resident’s official role is spent collecting, interpreting, and communicating laboratory data compared to the rest of the surgical team.

Although one would expect to see varying data preferences between surgical subspecialties, the small number of surgeons from a number of representative specialties made it difficult to explore this issue in detail. Other important determinants of discharge data preferences included a surgeon’s gender and race. The significance of these final findings is unclear to us at this point but does mandate further investigation. As the racial and gender diversity of surgical training programs continues to expand,¹³ such differences will be important to better understand and incorporate different approaches to clinical decision-making. Also, “cognitive constraints” such as these are known confounders in clinical decision support tools.¹⁴ Differences such as the user’s race and gender may need to be considered when designing these systems.

Limitations

One clear limitation to our study was that all survey responses were conducted at a single U.S. academic medical institution. Although Emory University School of Medicine’s medical faculty are spread across vastly different clinical environments (e.g., a university hospital, a Veterans Affairs Medical Center, a publicly-owned urban hospital, and an academic-private hybrid hospital model), it is possible that the responses above represent an institutional way of thinking that may not be shared elsewhere. However, this limited data set may also be viewed as a strength in that it provides our institution specific insight into how Emory surgeons use available clinical data to make the discharge decision. Second, the limited size of a single medical institution’s faculty and residency program also limited the power of the statistical analysis. Future studies are planned with larger regional and national populations of surgeons to elicit more data for study of this important question.

A final limitation of the study was the response rate of 49%. Given the time-consuming nature of the survey tool, we feared that further compulsion to respond to the survey would have resulted in less thoughtful responses that could cofound results. On the other hand, there is the possibility that those not responding are the busiest surgeons and thus more likely to have a handful of clinical indicators they heuristically use to handle a heavier clinical case load. However, we believe such a phenomenon to be unlikely given the broad variety of surgical subspecialties and range of clinical training and institutional rank represented in the data above. There were no substantive difference between response rates amongst residents, fellows, or faculty (47% vs. 50% vs. 51%, respectively), and subspecialists’ participation was roughly proportional to division size within the department of surgery.

Conclusion

We believe the information provided here is the first step in creating the modern tools for assisting clinicians to deal with uncertainty with ever increasing amounts of clinical data. For example, even within the logical categorical groupings used above, Figure 1 demonstrates obvious intra-category variability in level of preferences for individual discharge-relevant information. This variation may suggest inherent biases and heuristical behavior as to how surgeons collect and interpret clinical data. Understanding what clinicians think they use to make discharge decisions is as important as what trends behavioral monitoring may demonstrate in how clinicians are actually making these decisions. The analysis above also demonstrates that what data are favored by the clinicians is also influenced by the characteristics of the individual surgeon. Further studies to explore surgeon decision-making must take into account that different subsets of surgeons will approach the same data differently.

Understanding clinicians’ perspectives on discharge is particularly important for designing data analytics with a man-machine interface. A decision support tool that assists a clinician but does not frame recommendations in a way that the clinician sees as contextually appropriate will be ignored.¹⁴ For example, even if algorithmically a patient’s level of education is relevant to his or her chance of readmission, the findings above suggest that the typical surgeon deemphasizes this clinical variable when making a discharge decision. Future decision tools will need to incorporate contextualized means of communication that balance directed recommendations with opportunities for surgeon education about newly recognized correlated risks. Little is known about clinician adoption of such technological assistance and a greater understanding of that process and how clinicians’ existing heuristics may affect it is needed.