Mortality factors in pancreatic surgery: a systematic review. How important is the hospital volume?

Abstract

Background:

How the extent of confounding adjustment impact (hospital) volume-outcome relationships in published studies on pancreatic cancer surgery is unknown.

Methods:

A systematic literature review for studies examining the volume-outcome relationship was conducted. Importance of covariates were assessed by effect size (odds ratio, OR) and statistical significance. The impact of covariate adjustment on hospital (or surgeon) volume effects was analyzed by regression and meta-regression models.

Results:

We identified 87 studies with nearly 1 million patients undergoing pancreatic surgery that included in total 71 covariates for risk adjustment. Of these, 33 (47%) had a statistically significant effects on short-term mortality and 23 (32%) did not, while for 15 (21%) factors neither effect size nor statistical significance were reported. The most important covariates for short term mortality were patient-specific factors. Concerning the covariates, single comorbidities (OR 4.6, 95% CI: 3.3 to 6.3) had the strongest impact on mortality followed by hospital volume (OR 2.9, 95% CI: 2.5 to 3.3) and the procedure (OR 2.2, 95% CI: 1.9 to 2.5). Among the single comorbidities, coagulopathy (OR 4.5, 95% CI: 2.8 to 7.2) and dementia (OR 4.2, 95% CI: 2.2 to 8.0) had the strongest influence on mortality. The regression analysis showed a significant decrease in the association effect size of hospital volume with an increasing number of covariates considered (P = 0.001), while such a relationship was not observed for surgeon volume (P = 0.35).

Conclusions:

This analysis demonstrated a significant inverse relationship between the extent of risk adjustment and the case volume effect, suggesting the presence of unmeasured confounding and overestimation of volume effects. Our results underscore the importance of using clinical and administrative data for research on the volume-outcome relationship.

1. Introduction

A large body of research investigated the association between provider volume and outcome quality for several surgical procedures and different kinds of medical care.^1–4 The central rationale of minimum volume standard is the assumption that an increased procedure volume results in more sophisticated knowledge, better patient management, and more experience in performing these procedures, all of which in turn can improve the outcome and ensures a higher quality of care.^1,5

On the other hand several studies demonstrated comparable outcome quality in low and high-volume centers.^6,7 Two recent population-based studies reported single institution mortality rates up to more than 10%, even for high-volume hospitals.^8,9 These contradictory findings may be explained by differences in patient characteristics, which should be leveled by risk-adjustment. However, the extent to which the outcome-relevant factors (covariates) were taken into account in primary studies and their influence on case volume effects are open questions. Therefore, we conducted a systematic literature review of studies examining the volume-outcome relationship in pancreatic surgery in order to assess confounding effects of various factors related to individual patients and surgical practice. We hypothesized that each factor had differential confounding effect and that there was an inverse relationship between the extent of risk-adjustment and reported strength of case volume effect.

2. Methods

Queried databases were PubMed, Cochrane Central Register of Controlled Trials, Livivo and Medline. In PubMed following search term was used: ((volume-outcome-relationship) OR (volume AND (outcome OR result OR quality))) AND (hospital OR clinic OR center OR surgeon OR operator) AND (pancr*) AND (mortality OR death OR died). Last query was performed as of 2020/09/16. Furthermore, the “related articles” function in PubMed was used and reference sections of articles were screened for additional records.

Inclusion criteria were defined according to PICO as follows: eligible studies had to examine the impact of hospital and/or surgeon volume on short-term mortality (≤ 90 days) for patients (≥ 18 years) undergoing pancreatic resection independently of primary diagnosis. Excluded were studies that performed no risk-adjustment procedure and reported only crude mortality rates. No restriction to language, study and data year, or study type were applied.

Data extraction

As general study characteristics were recorded author name, study period, country, endpoint, data source, data type (administrative vs. clinical), study focus (hospital and/or surgeon volume), including diseases and procedures, as well as number of patients/hospitals/surgeons. Specifically, all covariates used for risk adjustment as well as their effect size, confidence interval and significance were extracted. Finally, the risk-adjusted effects of the procedure volume (at hospital/surgeon level) were recorded. Results of the most complex reported multivariable model were used.

As principal summary measure odds ratios (OR) were used. Other effect measures (for example, hazard ratios and relative risks) were transformed to ORs using established formulas.¹⁰ If necessary, confidence limits were obtained from p-values.¹¹ Categorical groupings of continuous variables (especially for age and comorbidity score) were converted to continuous values by least squares approximation. Continuous volume effects (effects per case) were dichotomized according to leapfrog criteria (≥ 11 vs. < 11 annual procedures).

If possible, the Charlson co-morbidity definitions have been matched to the corresponding Elixhauser definition (for example, renal disease according to Charlson was subsumed under renal failure according to Elixhauser). Comorbidity categories that had no equivalent in the Elixhauser classification system were adopted in their original form.

Since we aimed to demonstrate the importance of covariate adjustment, the ORs of categorical predictors were recalculated as needed, so that the ORs represent the effect difference between the most extreme categories. All adjusting factors were converted so that values of reference categories represent lowest risk of mortality. For example, the reported magnitudes of hospital volume effects represent the difference in odds of mortality between hospitals of the lowest and highest volume category (serving here as reference group).

Exclusion of patients according to a specific covariate (e.g. elective surgery only or limitation to a single resection procedure) was considered as adjustment for this respective factor. Whenever studies used missing data from a covariate as a separate category for risk-adjustment this effect was not considered.

Synthesis of results

The effect sizes of the individual covariates and comorbidities were pooled across studies using meta-analytic procedures. Due to an expected heterogeneity between studies, a random effect model with restricted-maximum likelihood estimation was used. Effect heterogeneity was assessed by I² with values above 50% indicating high heterogeneity. Since raw or aggregated data were often not reported, results were assessed in modified Forrest plots. For each risk factor, study-specific effect sizes were plotted with color-coded significance of the effect and number of cases represented by circle size. For an individual study overview a risk-map was established by plotting the variables of the risk adjustment against the individual studies with concurrent presentation of the effect size and significance level. To correlate the importance of individual risk factors to the risk factor groups, a treemap was created that depicts the pooled effect size of the respective covariates assigned to the corresponding risk category group.

Hospital and surgeon volume effects were specifically analyzed in terms of the extent of risk adjustment, operationalized by the number of total and significant covariates included in the multivariate analyzes. Therefore, regression analyses (linear and quadratic) were performed. Model performances were compared by standard error of the regression. Furthermore, effect of risk-adjustment was assessed by using the number of total and significant covariates as moderators in a meta-regression. Relationship between extent of risk-adjustment and volume effects were graphically presented by scatter plot.

The analyses were performed using R 4.0.3 (R Foundation for Statistical Computing) and two-sided p-values < 0.05 were considered as statistically significant. The implementation and report of the study is based on the recommendations of the PRISMA guideline.¹²

3. Results

Literature review

The literature screening yielded 1683 hits, of which 87 studies from 1984 to 2017 were eligible according to our inclusion criteria, which included a total number of 979,794 patients as summarized in Table 1 (for PRISMA flowchart see supplemental Figure S1; detailed study characteristics are given in supplemental Table S1). The majority of studies were from the United States (N = 56, 64%) or Europe (N = 19, 22%) and based on administrative data (N = 75, 86%).

Tab. 1.

Overview of study characteristics on study and patient level; Numbers of study year and procedure did not add up to 100%, because some studies span over multiple decades and included multiple procedures

	by studies	by patients
Characteristic	N = 871	N = 979,7941
Region
North-America	61 (70.1%)	665,682 (67.9%)
Europe	19 (21.8%)	256,386 (26.2%)
Other	7 (8.0%)	57,726 (5.9%)
Data type
Administrative	75 (86.2%)	820,822 (83.8%)
Clinical	12 (13.8%)	158,972 (16.2%)
Focus
Both	8 (9.2%)	28,998 (3.0%)
Hospital	73 (83.9%)	913,663 (93.3%)
Surgeon	6 (6.9%)	37,133 (3.8%)
Mortality
30 day	23 (26.4%)	202,182 (20.6%)
60 day	1 (1.1%)	13,107 (1.3%)
90 day	8 (9.2%)	75,442 (7.7%)
in-hospital	50 (57.5%)	658,567 (67.2%)
operative	5 (5.7%)	30,496 (3.1%)
Procedure
Radical pancreaticoduodenectomy	72 (82.8%)	813,022 (83.0%)
Total pancreatectomy	41 (47.1%)	676,792 (69.1%)
Proximal pancreatectomy	37 (42.5%)	568,190 (58.0%)
Distal pancreatectomy	38 (43.7%)	665,024 (67.9%)
Radical subtotal pancreatectomy	35 (40.2%)	555,566 (56.7%)
Other partial pancreatectomy	30 (34.5%)	491,139 (50.1%)
Pancreatic resection (no details provided)	11 (12.6%)	73,531 (7.5%)
Study period
before 1990	8 (9.2%)	140,003 (14.3%)
1990–1999	33 (37.9%)	310,703 (31.7%)
2000–2010	58 (66.7%)	758,050 (77.4%)
after 2010	33 (37.9%)	435,300 (44.4%)
No. of covariates	7 (6, 9)	8 (7, 9)

¹n (%); Median (IQR)

Covariates and Covariate Groups

Across the 87 studies, 71 different covariates were considered for risk-adjustment, which can be summarized in seven groups: general patient factors, medical patient factors, disease factors, treatment factors, outcome factors, hospital factors and surgeon factors. Thirty-three covariates showed a significant effect on short-term mortality in at least one study. Studies varied widely with respect to the number of covariates considered (see also supplemental Table S2). The median number of considered covariates was 7 (interquartile range [IQR]: 6 to 9) in studies based on administrative data and 13 (IQR: 8 to 16) in studies based on clinical datasets. Almost all studies included age and gender (98% and 92%, respectively) and comorbidity evaluation (comorbidity scores or single comorbidities, 87%). A detailed overview of all risk factors is presented in the risk-map (supplemental Figure S2).

Seventeen risk factors were considered and reported by more than 5 studies and are summarized in Figure 1. Frequently reported significant covariates were age (46 significant of 47), procedure (19/23), single comorbidities (15/16), comorbidity scores (17/24), diagnosis (15/22), insurance type (12/18) and socio-economic status (6/7) (Supplementary Figure S3 and Table S2). The strongest influence on mortality was found for comorbidities (OR 4.6, 95% CI: 3.3 to 6.3) followed by hospital volume (OR 2.9, 95% CI 2.5 to 3.3). The treemap in Figure 2 summarizes the risk factors according to the covariate group and indicates the relative importance for short-term mortality by area of the respective tile. A significant influence of short term mortality was found in the covariate groups: general patient factors, medical patient factors, surgeon factors, treatment factors and hospital factors. An increased influence of medical patient factors and general patient factors compared to that of hospital factors (62% versus 12%) was detected.

Fig. 1.

Important risk-factors for short-term mortality after pancreatic surgery, sorted by pooled effect size; Only covariates considered in more than five studies and deemed significant at least once are presented; Each point indicate a single study, point diameter represent study sample size, and point color represent significance; Grey rectangles indicate pooled effect size as estimated with random-effects model (OR and 95%-confidence interval limits); Confidence intervals not crossing the dotted line (OR = 1, no effect) indicate a significant effect on short-term mortality; Number of cases varies for each covariate

Fig. 2.

Summary of risk-factors for short term-mortality after pancreatic surgery, grouped according to risk-group Area size and shading are proportional to effect size, with larger area and darker shading indicate higher effects; Only five of seven risk-groups are depicted, as of the two others no single risk-factor was considered by more than five studies

A large number of covariates (52 of 71) was only considered infrequently (by ≤ 5 studies), of which 14 showed a significant effect on short-term mortality. These include body mass index, readmission, American Society of Anesthesiologists class (ASA), tumor size, tumor histology, tumor grading, N stage, length-of-stay, facility type, fistula risk score, risk of mortality, necrosectomy, hospital experience and neoadjuvant radiotherapy. For 15 covariates neither an effect nor a significance was reported.

Single comorbidities represent to the most important isolated risk factor for mortality (Figure 1). Based on the effect sizes of the 15 studies that reported respective measures, the pooled overall effect was estimated. The highest effects, with confidence intervals distinct from 1 were observed for coagulopathy, dementia, renal failure, pulmonary circulation disorders and liver disease (Figure 3).

Fig. 3.

Effects of single comorbidity categories on short-term mortality after pancreatic surgery, sorted by average effect size, as estimated with random-effects model (black rectangle, Odds ratio and 95%-confidence interval) Each point indicate a single study, point diameter represent study sample size, and point color represent significance Confidence intervals not crossing the dotted line (OR = 1, no effect) indicate a significant effect on short-term mortality Co-morbidity definitions according to Elixhauser, except those marked with C (indicating Charlson-definitions); Number of cases varies for each risk factor

Volume effects

The vast majority of studies focused exclusively on hospital volume effects (73 of 87), of which most (89%) reported a significantly increased mortality in the lowest hospital volume group.^1,8,13–75 Six studies reported no significant hospital volume effect, despite a consistent tendency to improved survival in high volume providers.^76–81 Only two studies demonstrated a non-significant reduced mortality in low-volume hospitals.^82,83 Two studies reported hospital volume effects only combined with other characteristics (presence of complications²⁵ and surgeon volume⁸⁴) suggesting that isolated effects for hospital volume could not be specified.

Six studies examined surgeon volume effects, of which two failed significance despite a tendency of reduced mortality for high volume surgeons.^85–90

Furthermore, eight studies simultaneously examined the volume effects of hospital and surgeon. From two studies no effect sizes could be extracted.^84,91 Five studies reported significant effects for both hospital and surgeon volume,^92–96 while one study failed to demonstrate significant effects, although a tendency to decreased mortality was observed for high volume hospitals and surgeons.⁹⁷

Covariate adjustment and volume effects

The hospital volume effect showed a significant negative linear association in relationship to the number of considered covariates (Figure 4). Even after exclusion of two studies with unusual high volume effects (OR > 10), a significant linear relationship was observed (b = −0.14, 95% CI: −0.25 to −0.04; P = 0.007).^13,34 The more covariates are examined, the less influence of the hospital volume was found.

Fig. 4.

Scatterplot of hospital volume effect (lowest vs. highest category, expressed in Odds Ratios) in relation to number of covariates of each study, point diameter represents study sample size, point color indicates type of database; Two studies with ORs > 10 are excluded (Ahola, 2019; Gordon 1999)

Applying a quadratic regression did not improve prediction accuracy, measured by standard error of regression (linear model: 1.42 vs quadratic model: 1.50). Within a sensitivity analysis only the number of significant covariates were examined in relation to the hospital volume effect, which yielded almost identical result (b = −0.20, 95% CI: −0.33 to −0.08, P = 0.002). Incorporating the number of adjusting factors as moderator in a meta-regression also yielded a significant effect (OR = −0.04, 95% CI: −0.07 to −0.01, P = 0.016) and even more pronounced when only significant covariates are considered (OR = −0.06, 95% CI: −0.10 to −0.03, P < 0.001).

There were no significant associations between the extent of the risk adjustment and surgeon volume neither in the linear regression analysis (y = 2.79 – 0.07x; P = .45) nor in the meta-regression (OR = −0.01, 95% CI: −0.27 to 0.25, P = 0.96). Considering only the number of significant covariates the pooled effect sizes resulted in an estimate of −0.04 (95% CI: −0.19 to 0.10, P = 0.56) for each additional considered covariate. Although not significant a notable difference between average effect sizes of studies based on administrative (mean OR = 3.0) and clinical (mean OR = 2.3) databases (mean difference: 0.68, 95% CI: −0.29 to 1.64, P = 0.16) was identified.

4. Discussion

The aim of the present analysis was to systematically identify mortality-related factors and to examine the volume-outcome relationships in published studies on pancreatic surgery. Therefore, we performed a systematic literature review that selected 87 studies on the volume-outcome relationship, including approximately 1 million patients and assessed the impact of risk-adjustment on the reported volume-outcome effect. The analysis yielded two main results.

First, the analysis showed large differences in the extent to which mortality-related covariates were taken into account, with a significant negative correlation between the extent of risk adjustment and hospital volume effects. With increasing numbers of covariates the volume effect decreased. This is an important finding and was described for pancreatic surgery the first time. Since hospitals of the highest and lowest volume group were compared, the reported effects can be considered as an upper limit of volume-related effects. There was no association between the extent of risk adjustment and surgeon volume, although this might be due to the low number of studies (n = 12) and the small differences between the studies (between 6 and 16 covariates).

The second main finding is that the strength of the volume effects tend to differ depending on the data source. Although not significant, studies based on administrative data sets show, on average, higher volume-outcome relationships than those based on clinical data sets. The lack of significance can be attributed to the small number of studies that used clinical data sets. In addition to possible sources of bias in administrative data, the difference may be due to the extent of the data contained and thus the possibility of a comprehensive risk adjustment. Many studies reported differences in patient characteristics between high and low volume hospitals and surgeons with younger, less morbid, higher socio-economic status and more healthy patients in high volume hospitals/surgeons.^{15,26,28,30,41,45,52,69,93} Furthermore, patients’ characteristics are suitable for predicting the probability of treatment at a high-volume hospital or by a high volume surgeon.^89,98–101 According to the differences in patient characteristics, risk adjustment results in attenuated volume effects (positive confounding). A more comprehensive risk adjustment that reduces differences in patients’ characteristics between high- and low-volume providers consequently results in smaller differences. Probably, using clinical data allows more sophisticated risk-adjustment and minimized subtle differences in severity of disease between different hospitals.¹⁰² Large studies with administrative data showed lower mortality rates in about 40% of the hospitals with lower case load than in high volume centers.^8,9 Since it is not feasible to conduct large-scale randomized controlled trials to investigate the volume-outcome relationship results based on administrative and clinical data should give comparable results. Our results underline the significance of clinical research compared to research with administrative data for the control of the health care system.

So far, no extensive summary of the effect of comorbidities on short-term mortality was reported. The presented overview may assist clinicians in their decision to perform surgery as well as researchers to identify and select appropriate variable sets for risk-adjustment. Despite their high prognostic relevance, only ten studies included disease related factors, of which eight are based on clinical datasets.^{19,46,48,53,66,69,72,75,79,83} Another shortcoming is that only two studies considered the length-of-stay,^56,79 although it has been shown that it might bias effects and favors hospitals with shorter length-of-stay.¹⁰³ Since shorter length-of-stay might be a desired outcome and/or quality indicator in the context of in-hospital mortality, the factor should used for adjustment. Given the variable extent to which individual comorbidities have been identified as significant predictors of short-term mortality in relation to summary comorbidity scores, it seems more appropriate to use individual comorbidity categories. In addition, an expansion of Quan-Elixhauser comorbidities to include dementia and cerebrovascular disease as included in the Charlson definition appears to be beneficial for a more accurate risk adjustment.

With a few exceptions, the studies analyzed showed a significantly higher mortality risk in the lowest volume group compared to the highest volume group, both for the effects of hospital and surgeon procedure volume. Only 8 out of 81 studies examining hospital volume effects and 4 out of 14 studies on surgeon volume effects lack a significantly reduced short-term mortality in the highest volume group.^{76–81,85,87,91,97} However, these studies have small sample sizes with < 2000 patients and examined more recent time periods. Only a single study reported a worse outcome in the higher volume hospital group.⁸³

Interestingly, the two studies that failed to show a surgeon volume effect were adjusted for the amount of supervision and training, respectively.^85,87 Findings that are in accordance with other studies support the importance of training and experience.^7,96,104,105

Recent reviews and meta-analyses did not discuss the high variability of studies regarding the risk-adjustment and the database employed, despite differences were described.^106,107 Some reviews recorded the type of the database and emphasized the shortcomings of administrative databases, such as coding inaccuracies, renumeration incentives for upcoding, and an incomplete set of highly outcome-relevant variables.^107–109 However, the impact of the extent of risk-adjustment on reported volume effects were not systematically assessed.

In one meta-analysis, a subgroup-analysis compared studies with and without risk adjustment for severity (tumor stage and adjuvant treatment) and found lower effects in studies with risk adjustment (ORs = 2.0 vs 4.0; high vs. low volume).¹⁰⁸ In a second subgroup analysis within the same meta-analysis, studies with adjustment for acuity of admission showed higher volume effects than unadjusted studies (ORs = 5.0 vs 2.6). However, both subgroup comparisons only included two studies in the adjustment group and showed no significant differences.

It is noteworthy that the present study also has a number of limitations. These include the number of covariates used to compare studies with regard to risk adjustment, which can be regarded as an rough estimate for extent of risk-adjustment and should be extended based on the importance of non-considered factors to increase risk-adjustment quality. In addition, mainly extreme effects were recorded, which should be extended to effects of “intermediate” risk-categories.

In summary, the present analysis demonstrates an effect of procedure volume on mortality across almost all studies analyzed. However, this effect is systematically overestimated, since the scope of the risk adjustment in many studies is too small to enable an unbiased, valid comparison between hospitals. With the compiled risk-map, the present study aims to provide a tool that may facilitate the identification of risk-adjustment variables in further studies.

List of abbreviations:

CI

Confidence interval

IQR

Interquartile range

OR

Odds ratio

PICO

Population, Interventions, Control, Outcome

PRISMA

Preferred Reporting Items for Systematic Reviews and Meta-Analyses