Abstract
Background
Comparison of quality indicators can improve quality of care. However, case-mix adjustment is deemed essential. The aim of this study was to develop and validate case-mix adjustment models and to evaluate the effect of case-mix adjustment for the quality indicators related to complications after breast cancer surgery.
Methods
Multivariable logistic regression with backward selection (P < 0.1) was used to develop case-mix models in patients undergoing breast cancer surgery (all types, breast-conserving surgery, mastectomy with or without immediate reconstruction) in the Netherlands (NABON Breast Cancer Audit). High-impact complications were defined as Clavien Dindo grade ≥3. Model performance was assessed using the area under the receiver operating characteristic curve (AUC), corrected for optimism with bootstrap validation. Observed-to-expected plots were used to visualize the difference between unadjusted and case-mix adjusted hospital performance (hospital shifts).
Results
In total 32 084 patients from 72 hospitals treated in 2021–2022 were included. A between-hospital variation in complication rates was observed for all surgeries (interquartile range 2.4–6.0%), breast-conserving surgery (interquartile range 1.4–3.4%), and mastectomy with (interquartile range 9.4–9.1%) and without reconstruction (interquartile range 3.3–9.7%). Of the considered variables, body mass index, smoking, multifocality and neoadjuvant therapy were weakly associated with complications. However, surgery type was strongly related to complications (AUC 0.70), resulting in noticeable hospital shifts in the quality indicator scores comprising all surgeries. After stratification for surgery type, no evident hospital shifts were observed after case-mix correction.
Conclusion
For valid comparison of complication rates after breast cancer surgery between hospitals, stratification by surgery type is crucial. Subsequently, the evaluated patient and tumour characteristics have a negligible effect on the hospital variation.
This large multicentre registry study evaluates the impact of case-mix adjustment when comparing complication rates after breast cancer surgery across hospitals. The results show that for valid comparisons of complication rates, stratification by surgery type is crucial. Subsequently, the evaluated patient and tumour characteristics have a negligible effect on hospital variation.
Introduction
In Europe, nearly half of the countries have implemented national programmes aimed at improving breast cancer (BC) care1. A key component of these initiatives involves gathering hospital data and presenting them periodically through quality indicators (QIs). Evidence shows that registries, which benchmark hospitals with these QIs, achieve improved quality of care2–4. Hospitals have the opportunity to assess their own performance in comparison to others, enabling those with poorer performance to learn from best practices and improve. When reporting these QIs, it is important that QIs measure actual differences in quality of care, especially when QIs become publicly accessible. This necessitates the use of valid and reliable indicators5.
The Dutch breast cancer quality registry (also called NABON Breast Cancer Audit, NBCA) includes all surgically treated patients with BC in the Netherlands6. Since 2011, data have been gathered and QIs are developed on structures, processes, and outcomes. The Dutch Ministry of Health, Welfare and Sport (VWS) invested from 2018 to 2022 in an outcome-based care programme7. This programme identified ‘complications after surgery’ as an important outcome indicator8. This indicator is currently measured in the Dutch breast cancer quality registry for internal use only.
Transparency regarding high-impact complications after BC surgery is important for improving quality of care and informing patients. Patient and tumour characteristics, known as case-mix factors, can influence the validity of QIs5,9–12. Significant case-mix effects indicate that QIs reflect differences in patient populations rather than in care quality. Vos et al. conducted a study on case-mix adjustment of six BC QIs, highlighting the importance of assessing case-mix impact for each indicator independently5. The specific patient and tumour characteristics of importance depended on the specific structure, process or outcome being measured. Outcome indicators showed evident case-mix impact. Therefore, the aim of this study was to assess whether case-mix played a role for the outcome indicator ‘complications after surgery’.
The aim was to develop and validate case-mix adjustment models and to evaluate the effect of case-mix adjustment on the QI related to complications after BC surgery.
Methods
Study population
All patients registered in the Dutch breast cancer quality registry in 2021 and 2022 were included as this was the first year the QI ‘complications after surgery’ was adequately measured. The inclusion time is the moment of biopsy. The database covers all 72 hospitals that provide BC care in the Netherlands and includes all patients with primary BC and ductal carcinoma in situ (DCIS) who underwent surgery. Lobular carcinoma in situ (LCIS), recurrent BC, phyllodes tumours and non-surgically treated patients were excluded.
Potential case-mix factors
Potential case-mix variables were retrieved from literature and expert opinion (Dutch breast cancer quality board and scientific committee). The potential case-mix variables included patient characteristics (age, body mass index (BMI), sex, smoking), tumour characteristics (T stage, N stage, histologic subtype, differentiation grade, receptor status, multifocality based on pathology, screen detection), and treatment characteristics (type of surgery and systemic therapy).
Quality indicator
The performance indicator ‘complications after surgery’ is defined as the proportion of patients undergoing primary surgery for DCIS or invasive BC that have a complication during admission or within 30 days after surgery. In this study complications with Clavien–Dindo grades 3, 4, or 5 were included, which comprise high-impact complications involving the need for surgical, endoscopic, or radiological intervention, life-threatening complications requiring admission to a monitored care unit, or mortality13,14. The indicator is measured for three groups: all surgeries, breast-conserving surgery (BCS), and mastectomy.
Statistical analysis and model development
Descriptive statistics are presented with frequencies (percentages) or means(s.d.). Complication rates are calculated over 2021 and 2022, and the variation between hospitals is visualized in boxplots. Hospital variation in potential case-mix variables is assessed by medians and interquartile ranges. The i.q.r. represents the range between the first (25th) and third (75th) quartiles of a variable, indicating the spread of the middle 50% of the data.
Multiple imputation with chained equations (MICE) was used to impute missing case-mix values (20 imputations)15,16. Besides the case-mix variables presented, the variables DCIS or invasive status, radiotherapy, adjuvant chemotherapy, endocrine therapy, complications after chemotherapy, and complications after surgery were included to obtain more accurate imputations. In practice, imputing outcomes is sensitive, and therefore, all patients with missing values for the variable ‘complications after surgery’ were removed from the data set after imputation.
For model development, patient data from 2021 and 2022 was used. Multivariable logistic regression was performed for the QI across all different subgroups. Significant case-mix variables were identified with backward selection (P < 0.1). For the continuous variables age and BMI, restricted cubic splines were used to assess if there was a non-linear relationship with the outcome. The impact of individual case-mix variables was evaluated using Wald statistics minus the degrees of freedom (χ2 – df). This enables comparison of the proportional contribution of each predictor to the outcome and to other predictors. The performance of the case-mix models was evaluated using the area under the receiver operating characteristic (ROC) curve (AUC). A value of 0.5 indicates a non-informative model, meaning that there is no influence of case-mix variables, whereas a value of 1 indicates a perfect model; all variation can be explained by case mix. Bootstrap validation with 200 resamples was used to correct the AUC for optimism.
The influence of case-mix adjustment is assessed by plotting unadjusted and case-mix adjusted observed/expected (O/E) ratios. The O/E ratio was calculated by dividing each hospital’s observed complication rate by its expected complication rate. The expected complication rate is determined using the overall average (unadjusted) or predicted probabilities from the case-mix adjustment model (adjusted). An O/E ratio above one indicates that a hospital performed worse than expected and a ratio below one means it performed better than expected.
All analyses were conducted using R, Version 4.2.3.
Results
Patient and tumour characteristics
The study population comprised 32 084 patients, with a mean(s.d.) age of 65(13) years, who either had invasive BC (87%) or DCIS (13%, Table 1). Of these patients, 71% underwent BCS (N = 22 661), 20.4% mastectomy without immediate reconstruction and 8.6% mastectomy with immediate reconstruction (N = 6 654 and N = 2 769; total mastectomy N = 9 423). Patients who underwent a mastectomy had a higher T stage (17% stage T3 or T4 versus 2.3%), had a multifocal tumour more often (34% versus 9.5%), were less likely diagnosed by screening (23% versus 48%) and received neoadjuvant chemotherapy more often (28% versus 19%). In 2021 patient numbers were higher as compared to 2022, due to lower incident numbers in 2020 as a result of the COVID pandemic (2021 N = 16 684, 2022 N = 15 400). Within the entire cohort, three variables had notable rates of missing data, namely smoking (25.1%), BMI (13.4%), and multifocality (5.5%).
Table 1.
Baseline characteristics of patients diagnosed with BC in the Netherlands (2021–2022), categorized by types of surgery relevant to the quality indicator ‘complications after surgery’
| Total | Breast-conserving surgery | Mastectomy | |||
|---|---|---|---|---|---|
| Characteristic | N = 32 084 | N = 22 661 | All N = 9423 |
Without immediate reconstruction N = 6654 |
With immediate reconstruction N = 2769 |
| Age (years), mean(s.d.) Missing | 65(13) 11 | 65(12) 6 | 64(15) 5 | 69(14) 5 | 53(11) 0 |
| BMI (kg/m2), mean(s.d.) Missing | 26.5(5.1) 3164 | 26.7(5.2) 2146 | 26(5) 1018 | 26.6(5.2) 797 | 24.9(4.3) 221 |
| Female | 31 858 (99.3) | 22 647 (99.9) | 9211 (97.8) | 6442 (96.7) | 2769 (100) |
| Smoking | |||||
| Never or previous | 20 149 (84) | 14 246 (84) | 5903 (84) | 4052 (83) | 1851 (85) |
| Current | 3871 (16) | 2736 (16) | 1135 (16) | 812 (17) | 323 (15) |
| Missing | 8064 | 5679 | 2385 | 1790 | 595 |
| Screen detected | 12 633 (40) | 10 553 (48) | 2080 (23) | 1401 (22) | 679 (25) |
| Missing | 864 | 604 | 260 | 197 | 63 |
| T stage | |||||
| cT1 | 16 207 (51) | 13 263 (59) | 2944 (32) | 1975 (30) | 969 (35) |
| cT2 | 8958 (28) | 5580 (25) | 3378 (36) | 2546 (39) | 832 (30) |
| cT3/cT4 | 2135 (6.7) | 526 (2.3) | 1609 (17) | 1293 (20) | 316 (12) |
| cTis | 4375 (14) | 3017 (13) | 1358 (15) | 743 (11) | 615 (23) |
| Missing | 409 | 275 | 134 | 97 | 37 |
| N stage | |||||
| cN0 | 27 547 (86) | 20 306 (90) | 7241 (77) | 4899 (74) | 2342 (85) |
| cN+ | 4340 (14) | 2227 (9.9) | 2113 (23) | 1701 (26) | 412 (15) |
| Missing | 197 | 128 | 69 | 54 | 15 |
| Histologic subtype | |||||
| No special type | 24 667 (79) | 17 942 (81) | 6725 (73) | 4675 (72) | 2050 (76) |
| Lobular | 3700 (12) | 2184 (9.9) | 1516 (16) | 1154 (18) | 362 (13) |
| Other | 2904 (9.3) | 1952 (8.8) | 952 (10) | 665 (10) | 287 (11) |
| Missing | 813 | 583 | 230 | 160 | 70 |
| Differentiation | |||||
| Grade 1 | 6870 (22) | 5604 (26) | 1266 (14) | 877 (14) | 389 (15) |
| Grade 2 | 15 860 (52) | 11 100 (51) | 4760 (53) | 3424 (54) | 1336 (52) |
| Grade 3 | 7893 (26) | 4980 (23) | 2913 (33) | 2055 (32) | 858 (33) |
| Missing | 1461 | 977 | 484 | 298 | 186 |
| Receptor status† | |||||
| HR+, HER2– | 21 370 (82.3) | 15 459 (84.4) | 5911 (78) | 4322 (77) | 1589 (79.9) |
| HER2+ | 2151 (8.3) | 1375 (7.5) | 776 (10) | 567 (10) | 209 (10.5) |
| Triple– | 2434 (9.4) | 1488 (8.1) | 946 (12) | 755 (13) | 191 (9.6) |
| Multifocality | |||||
| Unifocal | 25 300 (83) | 19 408 (90) | 5892 (66) | 4290 (68) | 1602 (62) |
| Multifocal | 5017 (17) | 2042 (9.5) | 2975 (34) | 2009 (32) | 966 (38) |
| Missing | 1767 | 1211 | 556 | 355 | 201 |
| Neoadjuvant chemotherapy | 6841 (21) | 4217 (19) | 2624 (28) | 1745 (26) | 879 (32) |
| Missing | 15 | 5 | 10 | 6 | 4 |
| Year | |||||
| 2021 | 16 684 (52) | 11 568 (51) | 5116 (54) | 3538 (53) | 1578 (57) |
| 2022 | 15 400 (48) | 11 093 (49) | 4307 (46) | 3116 (47) | 1191 (43) |
Values are n (%) unless otherwise indicated. †Patients with Tis disease are excluded, as receptor status is not determined in this group.
Complication rates
The complication rate after surgery in the overall study population was 4.4% (Table S1). Higher complication rates were observed in patients receiving a mastectomy (± immediate reconstruction, 8.9%) than in those undergoing BCS (2.6%). Patients undergoing immediate reconstruction after mastectomy had higher complication rates (13.5%) than those who underwent mastectomy without immediate reconstruction (7%). The majority of patients experienced grade 3 complications (N = 1284; grade 4: N = 13; grade 5: N = 5; Table S2).
Between hospital variation
The interquartile range for occurrence of surgical complications was 2.4–6.0% after all surgeries, 1.4–3.4% after BCS and 5.3–13% after mastectomy (± immediate reconstruction; Fig. S1). In terms of case mix, most variation was seen in the type of surgery (BCS versus mastectomy; Fig. 1). More specifically, within the mastectomy group, there was variability in the uptake of immediate reconstruction (i.q.r. 18–36%). Additionally, considerable variability was observed in the occurrence of screen-detected cancers (i.q.r. 35–43%), differentiation grade (i/q/r/ grade 1: 20–27%; grade 2: 47–55%, grade 3: 22–30%) and smoking status (i.q.r. 14–21%). Minimal variation was seen in sex (i.q.r. 99–100), as few men are diagnosed with BC. Therefore, sex was excluded from model development.
Fig. 1.
Hospital variation (n = 72) in case-mix variables in Dutch patients who underwent breast cancer surgery in 2021 and 2022 (n = 32 084)
The plots show the distribution of median age and BMI, along with the mean percentages of other potential case-mix variables. It is presented in a plot showing the 25th, 50th, and 75th percentiles, along with the minimal and maximal values (range). For the variable ‘immediate reconstruction’ the plot is presented as percentage of all mastectomy patients instead of all surgeries.
Model development, performance, and validation
Age and BMI both showed a linear association with complications, indicating that incorporating splines does not lead to significant improvement in model fit.
Smoking, multifocality and neoadjuvant chemotherapy were associated with the occurrence of surgical complications in most analyses (Table 2), followed by BMI, T and N stage. However, the strength of their association with complications was negligible compared to the strength of association between type of surgery and complications and immediate reconstruction (Fig. S2).
Table 2.
Case-mix adjustment models for complications after surgery, breast-conserving surgery, mastectomy, mastectomy without reconstruction, and mastectomy with reconstruction
| Total | Breast-conserving surgery | Mastectomy | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| N = 32,084* | N = 22,661* | All N = 9423* |
Without immediate reconstruction N = 6654* |
With immediate reconstruction N = 2769* |
||||||
| OR (95% c.i.) | P | OR (95% c.i.) | P | OR (95% c.i.) | P | OR (95% c.i.) | P | OR (95% c.i.) | P | |
| Age (ref: 60:70*) | X | 0.87 (0.82,0.92) | <0.0001 | 1.15 (1.04,1.27) | 0.0051 | Y | Y | |||
| BMI (ref: 25:30†) | 1.22 (1.15,1.29) | <0.0001 | 1.18 (1.10,1.25) | <0.0001 | 1.12 (1.01,1.25) | 0.0371 | ||||
| Smoking (ref: no or quitted) | 1.27 (1.12,1.58) | 0.0010 | 1.25 (1.04,1.53) | 0.0205 | 1.37 (1.01,1.88) | 0.0465 | 1.35 (0.97,1.87) | 0.0748 | Y | |
| Type of surgery (ref: breast-conserving surgery) | 2.50 (2.18,2.57) | <0.0001 | NA | NA | NA | NA | ||||
| Immediate reconstruction (ref: no) | 2.53 (2.15,2.95) | <0.0001 | NA | 5.11 (3.82,6.87) | <0.0001 | NA | NA | |||
| Screen detected (ref: no) | X | 0.75 (0.64,0.87) | 0.0002 | X | X | 0.74 (0.42,0.97) | 0.0365 | |||
|
T stage
cT1 cT2 cT3/4 cTis |
1 1.23 (1.03,1.47) 1.91 (1.46,2.49) 1.26 (1.02,1.55) |
<0.0001 | 1 1.62 (1.26,2.09) 1.88 (1.28,2.77) 0.99 (0.70,1.41) |
0.0004 | 1 1.86 (1.37,2.53) 2.18 (1.37,3.45) 1.22 (0.80,1.89) |
0.0003 | X | |||
| N stage, positive (ref: negative) | 1.30 (1.09,1.55) | 0.0033 | X | 1.38 (1,1.90) | 0.0474 | 1.42 (0.98,2.06) | 0.0643 | X | ||
| Receptor status | X | X | X | X | X | |||||
|
Differentiation
Grade 1 Grade 2 Grade 3 |
X | 1 1.27 (1.05,1.53) 1.24 (0.99,1.55) |
0.0443 | X | X | X | ||||
| Multifocality (ref: no) | 1.30 (1.12,1.50) | 0.0004 | 1.83 (1.55,2.15) | <0.0001 | 1.82 (1.41,2.34) | <0.0001 | 2.26 (1.70,3.00) | <0.0001 | 2.51 (1.68,3.76) | <0.0001 |
| Histologic subtype | 1.18 (1.01,1.40) | 0.0737 | X | X | X | X | ||||
| Neoadjuvant chemotherapy (ref: no) | 0.66 (0.56,0.78) | <0.0001 | 0.52 (0.42,0.64) | <0.0001 | 0.57 (0.41,0.80) | 0.0010 | 0.53 (0.36,0.76) | 0.0006 | ||
| C-statistic | 0.70 (0.69,0.72) | 0.61 (0.59,0.64) | 0.63 (0.60,0.65) | 0.56 (0.53,0.59) | 0.53 (0.50,0.56) | |||||
| Optimism | 0.004 | 0.009 | 0.015 | 0.025 | 0.036 | |||||
| C-statistic after bootstrap validation | 0.70 (0.70,0.71) | 0.60 (0.58,0.63) | 0.61 (0.59,0.63) | 0.54 (0.51,0.57) | 0.50 (0.47,0.53) | |||||
Odds ratios with confidence intervals (OR (95% c.i.) are presented for variables significantly associated with complications after backward selection (P < 0.1). NA, not applicable. X means not included in the model due to P > 0.1. *OR for age is presented as a decade (where one unit increase represents a decade increase). Reference value taken was between 60 and 70 years. †For BMI, one unit increase represents a 5-point increase in BMI. The reference value taken was between a BMI of 25 and 30. BMI, body mass index.
The discriminative ability of the model for all surgeries was good (AUC 0.70), and moderate for mastectomy (± reconstruction, AUC 0.63) and BCS (AUC 0.61). At the hospital level, evident shifts in the observed/expected plots were seen in the QI for all surgeries and the QI scores for mastectomy (± reconstruction) (Fig. 2). In contrast, for patients receiving BCS there was almost no effect of case-mix adjustment. For all developed case-mix models there was a small amount of overfitting (optimism for all models <0.036).
Fig. 2.
Observed–expected plots for all surgically treated patients and stratified for type of surgery
Each dot represents a hospital. On the x-axis the O/E before case-mix adjustment is presented, on the y-axis after case-mix adjustment. As a result, deviation from the diagonal shows the effect of case-mix adjustment. As an example, in the first plot an arrow points to one of the hospitals. This hospital had an O/E rate of 1.29 before case-mix adjustment and 0.91 after case-mix adjustment. This hospital is highly influenced by patient and tumour characteristics and shows a large deviation of the diagonal line.
Further stratification of mastectomy patients was applied based on whether immediate reconstruction was performed, because explanatory analyses showed that immediate reconstruction did significantly affect between-hospital comparisons. After stratification for immediate reconstruction, the AUC was poor (without reconstruction: 0.56, with reconstruction: 0.53), and consequently the influence of case-mix adjustment on hospital shifts was minimal (Fig. 2).
Discussion
This study aimed to develop practical case-mix models for the QI ‘complications after BC surgery’. This QI is applied to all types of surgery, BCS and mastectomy (± immediate reconstruction) in the Dutch breast cancer quality registry. The analyses showed that the type of surgery, and whether immediate reconstruction was performed, were the strongest and most important factors associated with the occurrence of complications. After case-mix adjustment noticeable hospital shifts in outcomes for all types of surgeries and for mastectomy (± reconstruction) were seen. In contrast, the occurrence of complications after BCS was only weakly associated with case-mix variables, meaning negligible influence of case-mix adjustment on QI scores. After further stratifying mastectomy patients based on whether they underwent immediate reconstruction, the former hospital shifts disappeared. This suggests that comparisons can be made without the need for case-mix adjustment by adequate stratification into BCS, mastectomy with immediate reconstruction, and mastectomy without immediate reconstruction.
The importance of type of surgery in relation to complication rates has been highlighted before17,18, with multiple studies confirming comparable complication rates, indicating the lowest complications for BCS and the highest for mastectomy with immediate reconstruction19–22.
Only one study has investigated case-mix adjustment regarding complications after BC surgery before. Berlin et al. examined the influence of case-mix adjustment specifically for the group undergoing immediate breast reconstruction across 11 American hospitals23. Although they reported an impact of case-mix adjustment, they did not specify the strength of the case-mix variables responsible for it, nor did they evaluate the adjustment’s impact on estimated hospital performance. This makes it difficult to compare these results to the current study. Nevertheless, Dutch studies on anastomotic leakage after colon surgery and Clavien–Dindo grade >3 complications after cytoreductive surgeries also showed that adjustment for case-mix variables did not significantly affect hospital outcomes24,25.
Case-mix adjustment for six QIs of the Dutch breast cancer quality registry was previously examined5. Vos et al. demonstrated differences in the impact of case-mix adjustment for each QI. Four of six QIs showed evident impact of case-mix adjustment, comprising two outcome indicators (irradical resection in BCS for invasive BC and breast contour preserving treatment) and two process indicators (MRI before neoadjuvant chemotherapy and radiotherapy for locally advanced BC). This underlines the importance for individual QI evaluation on validity and reliability, to conclude whether case-mix adjustment has added value. Notably, the selection of potential case-mix variables can differ for each QI. For example, type of surgery was not included as a case-mix variable in the study of Vos et al. due to its lack of immediate relevance to the assessed QIs.
Although adjustment for patient and tumour variables had minimal impact on hospital level, statistically significant predictive variables for individual patients were identified, consistent with developed prediction models26,27. The finding that in BCS the likelihood of experiencing a complication decreases with age, whereas in mastectomy it increases with age, is not fully understood. This may be related to the selection of fitter patients for immediate reconstruction post-mastectomy, as immediate reconstruction is not included in the BCS model. Patients receiving neoadjuvant chemotherapy were less likely to have complications; however, the reasons for this observation also remain unclear. There could potentially be a link with the increased use of prophylactic antibiotics in this patient group. Furthermore, an association was found between smoking and the current QI. This effect was not found in the immediate reconstruction group. This is in contrast with a recent published meta-analysis (OR 1.46, 95% c.i. 1.08 to 1.97)28. The reason the current results are not in line with this meta-analysis might be attributed to the fact that plastic surgeons already take into account this risk factor by the choice of treatment (selection bias) – that is a reconstruction is discouraged in smokers. Another possibility is that the sample size was too small to show this association.
A proper case-mix model for outcome QIs after a certain treatment should only include vriables that are not influenced by the choices of a hospital or healthcare provider in the preoperative decisional pathway. Therefore, it is debatable whether type of surgery can be considered as a case-mix factor and should be included in the adjustment model. In the literature, the potential for including the extent of surgery as a case-mix variable is suggested to add value9,25, whereas the surgical approach (for example open versus laparoscopic procedure) may be more closely linked to the choice of a surgical team with accompanying technical capability and therefore preferably excluded9,29. In other terms, the approach of a procedure is more related to quality of care, for which adjustments are undesirable, whereas the extent (for example BCS or mastectomy) could be valuable for case-mix adjustment. Therefore, it was decided to include the type of surgery and, more detailed, immediate reconstruction yes or no in the study. Conversely, it was chosen to incorporate N-stage instead of the type of axillary surgery. As this is a grey area, the decision was further supported by the fact that complications related to axillary surgery are primarily seromas, infections, lymphoedema, and shoulder/arm morbidity, which are generally not classified as Clavien–Dindo grade ≥318.
The primary goal of this study is to stimulate quality of care improvements, but only using valid information to avoid misinformation. Based on the current findings, stratification by type of surgery is recommended rather than adjustment. Considerations that support stratification include the opportunity to shorten the time-consuming process of registering variables, particularly if variables with negligible impact on QIs in a quality registry can be removed. Another important reason that supports stratification, especially for mastectomy patients, is the difference in target groups interpreting the QIs. In the Netherlands, mastectomies are typically performed by surgeons, whereas immediate reconstruction procedures are performed in collaboration with plastic surgeons30. Consequently, stratifying mastectomy patients with and without immediate reconstruction gives better insight for both professions into their respective results. This provides a better opportunity for improvement, aligning with the central aim of quality registries6.
Participation in the Dutch breast cancer quality registry is obligatory for all 72 hospitals delivering BC care in the Netherlands. As a result, a considerable strength of this study is the usage of this nationwide registry including all BC patients treated with curative intent. Data quality of the registry is high, which is related to the use of the Netherlands Cancer Registry (NCR), in which data entry is performed by trained registrars from the Netherlands Comprehensive Cancer Organization (IKNL) in about 75% of the hospitals. Still, in this nationwide data set missing values were noted in the potential case-mix variables smoking (25.1%), BMI (13.4%), and multifocality (5.5%). ‘Multiple imputation, then deletion’ was used, which is the most valid way of reducing bias of missing values and increasing the precision of outcomes when benchmarking hospitals15,31. Another strength is the use of the Clavien–Dindo classification, which aligns with the International Consortium for Health Outcomes Measurement core outcome set for BC32, and therefore provides insight into high-impact complications (Clavien–Dindo grade ≥3). Under-registration in high-impact complications is less common than for minor complications. However, this approach limits the ability to specifically examine the types of complications (for example bleeding, severe infection), which could be valuable for understanding specific case-mix variables, as not all complications share the same risk factors27. Additionally, an important limitation is that the current model selectively includes variables collected in the Dutch breast cancer quality registry. This decision was made carefully considering practical implications. Nevertheless, some important risk factors for complications, like co-morbidities28,33, are not included as they are not measured in the registry.
Based on the current results it is argued that the QI ‘complications after BC surgery’ including all surgically treated patients is not a meaningful indicator, as differences are mostly related to the particular type of surgery instead of quality of care. The authors recommend that the QI is applied to the suggested stratification into type of surgery. As a result, it provides a more valid insight into the performance in time and/or between hospitals, making it more actionable for (plastic) surgeons and omitting the need for case-mix adjustment.
Nevertheless, the need for case-mix adjustment is only giving information on part of the validity of a QI. To obtain a complete overview of the value of the QIs ‘complications after BCS or mastectomy with or without immediate reconstruction’, an additional study conducting a comprehensive evaluation, including aspects such as feasibility and rankability, is needed34. In general, future studies should focus on developing meaningful and valid outcome indicators determining all above-mentioned aspects. When necessary, these indicators should include case-mix adjustment.
Stratification by the type of surgery is crucial for a more valid comparison of complication rates between hospitals after breast cancer surgery. After stratification, the evaluated patient and tumour characteristics have a negligible effect on the hospital variation. The QI in the Dutch national breast cancer registry will be refined based on these findings. Consequently, there will be a greater potential for improvement by more accurately measuring the quality of BC care between hospitals.
Supplementary Material
Acknowledgements
The authors thank all registrar and healthcare professionals for data registration and development of the NABON breast cancer audit (NBCA).
Contributor Information
Elfi M Verheul, Department of Public Health, Centre for Medical Decision Making, Erasmus University Medical Centre, Rotterdam, The Netherlands; Scientific Bureau, Dutch Institute for Clinical Auditing, Leiden, The Netherlands.
David van Klaveren, Department of Public Health, Centre for Medical Decision Making, Erasmus University Medical Centre, Rotterdam, The Netherlands.
Hester F Lingsma, Department of Public Health, Centre for Medical Decision Making, Erasmus University Medical Centre, Rotterdam, The Netherlands.
Elvira Vos, Department of Surgery, Rhode Island Hospital, Warren Alpert Medical School of Brown University, Providence, Rhode Island, USA.
Marije J Hoornweg, Department of Plastic and Reconstructive Surgery, Netherlands Cancer Institute/Antoni van Leeuwenhoek Hospital, Amsterdam, The Netherlands.
Sabine Siesling, Department of Research and Development, Netherlands Comprehensive Cancer Organization (IKNL), Utrecht, The Netherlands; Department of Health Technology and Services Research, Technical Medical Centre, University of Twente, Enschede, The Netherlands.
Linetta B Koppert, Department of Surgery, Erasmus MC Cancer Institute, Rotterdam, The Netherlands.
NBCA consortium:
Alwine A Hellingman, A M Moorman, Anne Brecht Francken, Bert van der Vegt, Carla Meeuwis, Carlijn T I de Betue, Carolien H M van Deurzen, Claudette E Loo, Caroline M E Contant, Cristina Guerrero Paez, D M den Hoed, Daniel Henneman, Dominique J P van Uden, Tanja G Frakking, Els Van Dessel, Enja J Bantema-Joppe, Ernst J P Schoenmaeckers, Ester Siemerink, Floris P R Verbeek, Gea A Gooiker, Henriette Schuttevaer, Hinne A Rakhorst, Ingrid Kappers, Ingrid van den Hoven, James Van Bastelaar, Janneke Verloop, José H Volders, Joan B Heijns, Joyce Meijer, Karin J Beelen, Klaartje van Engelen, Leonienke F C Dols, Linda de Munck, Marjan van Hezewijk, M R F Bosscher, Marian B E Menke-Pluijmers, Margrethe Schlooz-Vries, Marieke E Straver, Martinus A Beek, Maud Bessems, Marije C Gordinou de Gouberville, Milou H Martens, Miriam L Hoven-Gondrie, Marie-Jeanne T F D Vrancken Peeters, Patricia Jansen, Peter A Neijenhuis, Rhodé M Bijlsma, Robert-Jan Schipper, Ramon R J P van Eekeren, Thomas Schok, Tim C van Sprundel, Tim H C Damen, Titia E Lans, Vivianne C G Tjan-Heijnen, and Yvonne L J Vissers
Author contributions
Elfi Verheul (Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Software, Validation, Visualization, Writing—original draft, Writing—review & editing), David Van Klaveren (Conceptualization, Formal analysis, Investigation, Methodology, Software, Supervision, Validation, Visualization, Writing—review & editing), Hester Lingsma (Conceptualization, Methodology, Supervision, Validation, Writing—review & editing), Elvira Vos (Conceptualization, Methodology, Supervision, Validation, Writing—review & editing), Marije Hoornweg (Conceptualization, Supervision, Validation, Writing—review & editing), Sabine Siesling (Conceptualization, Methodology, Resources, Supervision, Validation, Writing—review & editing), and Linetta Koppert (Conceptualization, Data curation, Methodology, Resources, Supervision, Validation, Writing—review & editing).
Collaborators
Participants of the NBCA consortium (NABON Breast Cancer Audit); Alwine A. Hellingman (Department of Surgery, Tergooi Medical Centre), A. M. (Yvette) Moorman (Department of Surgery, Medical Centre Leeuwarden, Leeuwarden), Anne Brecht Francken (Department of Surgery, Isala Klinieken, Zwolle), Bert van der Vegt (Department of Pathology, University of Groningen, University Medical Centre Groningen, Groningen), Carla Meeuwis (Department of Radiology, Rijnstate Hospital, Arnhem), Carlijn T. I. de Betue (Department of Surgery, Jeroen Bosch Hospital, Den Bosch), Carolien H. M. van Deurzen (Department of Pathology, Erasmus MC Cancer Institute, Erasmus University Medical Centre, Rotterdam), Claudette E. Loo (Department of Radiology, Netherlands Cancer Institute–Antoni Van Leeuwenhoek, Amsterdam), Caroline M. E. Contant (Department of Surgery, Maasstad Hospital, Rotterdam), Cristina Guerrero Paez (Dutch Breast Cancer Society (BVN), Utrecht), D. M. (Irma) den Hoed (Department of Surgery, Elizabeth Tweesteden Hospital, Tilburg), Daniel Henneman (Department of Surgery, Treant Healthgroup and Chirurgen Coöperatie Noord Nederland), Dominique J. P. van Uden (Department of Surgery, Canisius Wilhelmina Hospital, Nijmegen), Tanja G. Frakking (Department of Surgery, Rivas Zorggroep Beatrix Hospital, Gorinchem), Els Van Dessel (Department of Surgery, ZorgSaam Hospital, Zeeuws-Vlaanderen), Enja J. Bantema-Joppe (Department of Radiation Oncology, Radiotherapy Institute Friesland, Friesland), Ernst J. P. Schoenmaeckers (Department of Surgery, Meander Medical Centre, Amersfoort), Ester Siemerink (Department of Medical Oncology, Ziekenhuisgroep Twente (ZGT), Hengelo), Floris P. R. Verbeek (Department of Surgery, Groene Hart Hospital, Gouda), Gea A. Gooiker (Department of Surgery, Northwest Clinics, Alkmaar, The Netherlands), Henriette Schuttevaer (Department of Surgery, Martini Hospital, Groningen), Hinne A. Rakhorst (Department of Plastic, Reconstructive, and Hand Surgery, Medisch Spectrum Twente and Ziekenhuisgroep Twente), Ingrid Kappers (Department of Surgery, Tjongerschans ziekenhuis, Heerenveen), Ingrid van den Hoven (Department of Surgery, Anna Hospital, Geldrop), James Van Bastelaar (Department of Surgery, Zuyderland Medical Centre, Heerlen), Janneke Verloop (Department of Research and Development, Netherlands Comprehensive Cancer Organisation (IKNL)), José H. Volders (Department of Surgery, Diakonessenhuis, Utrecht), Joan B. Heijns (Department of Medical Oncology, Amphia Hospital, Breda), Joyce Meijer (Department of Research and Development, Netherlands Comprehensive Cancer Organization (IKNL)), Karin. J. Beelen (Department of Medical Oncology, Rijnstate Hospital, Arnhem), Klaartje van Engelen (Department of Human Genetics, Amsterdam UMC and University of Amsterdam, Amsterdam), Leonienke F. C. Dols (Department of Surgery, Amphia Hospital, Breda), Linda de Munck (Department of Research and Development, Netherlands Comprehensive Cancer Organization (IKNL)), Marjan van Hezewijk (Department of Radiation Oncology, Radiotherapiegroep, Arnhem), M. R. F. (Frederiek) Bosscher (Department of Surgery, Ommelander Hospital, Groningen), Marian B. E. Menke-Pluijmers (Department of Surgery, Albert Schweitzer Hospital, Dordrecht), Margrethe Schlooz-Vries (Department of Surgery, Radboud University Medical Centre, Nijmegen), Marieke E. Straver (Department of Surgery, Haaglanden Medisch Centrum, Den Haag), Martinus A. Beek (Department of Surgery, Isala klinieken, Zwolle), Maud Bessems (Department of Surgery, Jeroen Bosch Hospital, Den Bosch), Marije C. Gordinou de Gouberville (Department of Surgery, Wilhelmina Hospital, Assen and Chirurgen Coöperatie Noord Nederland), Milou H. Martens (Department of Surgery, Laurentius Hospital, Roermond), Miriam L. Hoven-Gondrie (Department of Surgery, Gelderse Vallei, Ede), Marie-Jeanne T. F. D. Vrancken Peeters (Department of Surgery, Netherlands Cancer Institute–Antoni van Leeuwenhoek Hospital and Amsterdam University Medical Centre, Amsterdam), Patricia Jansen (Department of Surgery, Elizabeth Tweesteden Hospital, Tilburg), Peter A. Neijenhuis (Department of Surgery, Alrijne Hospital, Leiderdorp), Rhodé M. Bijlsma (Department of Medical Oncology, University Medical Centre Utrecht, Cancer Centre, Utrecht), Robert-Jan Schipper (Department of Surgery, Catharina Hospital, Eindhoven), Ramon R. J. P. van Eekeren (Department of Surgery, Rijnstate Hospital, Arnhem), Thomas Schok (Department of Surgery, VieCuri Medical Centre, Venray), Tim C. van Sprundel (Department of Surgery, Ommelander Hospital, Groningen), Tim H. C. Damen (Department of Plastic, Reconstructive, and Hand Surgery, Ikazia Hospital, Rotterdam; van Weel Bethesda Hospital, Dirksland; Spijkenisse MC, Spijkenisse), Titia E. Lans (Department of Surgery, Admiraal de Ruyterziekenhuis, Goes and Vlissingen), Vivianne C. G. Tjan-Heijnen (Department of Medical Oncology, GROW, Maastricht University Medical Centre, Maastricht), and Yvonne L. J. Vissers (Department of Surgery, Zuyderland Medical Centre, Heerlen).
Funding
The authors have no funding to declare.
Disclosure
The authors declare no conflict of interest.
Supplementary material
Supplementary material is available at BJS Open online.
Data availability
The authors used data from the NABON breast cancer audit (NBCA), a nationwide clinical audit. Data are available upon reasonable request.
References
- 1. Maes-Carballo M, Gomez-Fandino Y, Reinoso-Hermida A, Estrada-Lopez CR, Martin-Diaz M, Khan KS et al. Quality indicators for breast cancer care: a systematic review. Breast 2021;59:221–231 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Dreyer NA, Garner S. Registries for robust evidence. JAMA 2009;302:790–791 [DOI] [PubMed] [Google Scholar]
- 3. Stey AM, Russell MM, Ko CY, Sacks GD, Dawes AJ, Gibbons MM. Clinical registries and quality measurement in surgery: a systematic review. Surgery 2015;157:381–395 [DOI] [PubMed] [Google Scholar]
- 4. Fung CH, Lim YW, Mattke S, Damberg C, Shekelle PG. Systematic review: the evidence that publishing patient care performance data improves quality of care. Ann Intern Med 2008;148:111–123 [DOI] [PubMed] [Google Scholar]
- 5. Vos EL, Lingsma HF, Jager A, Schreuder K, Spronk P, Vrancken Peeters MTFD et al. Effect of case-mix and random variation on breast cancer care quality indicators and their rankability. Value Health 2020;23:1191–1199 [DOI] [PubMed] [Google Scholar]
- 6. van Bommel AC, Spronk PE, Vrancken Peeters MT, Jager A, Lobbes M, Maduro JH et al. Clinical auditing as an instrument for quality improvement in breast cancer care in The Netherlands: the national NABON breast cancer audit. J Surg Oncol 2017;115:243–249 [DOI] [PubMed] [Google Scholar]
- 7. Stimuleren uitkomstgerichte zorg . https://www.rijksoverheid.nl/onderwerpen/kwaliteit-van-de-zorg/stimuleren-uitkomstgerichte-zorg (accessed 27 July 2023)
- 8.Eindrapport aandoeningswerkgroep Mammacarcinoom: Programma Uitkomstgerichte Zorg – Lijn 1 ‘Meer inzicht in uitkomsten’. www.platformuitkomstgerichtezorg.nl; 2023.
- 9. Beck N, Hoeijmakers F, van der Willik EM, Heineman DJ, Braun J, Tollenaar RAEM et al. National comparison of hospital performances in lung cancer surgery: the role of case mix adjustment. Ann Thorac Surg 2018;106:412–420 [DOI] [PubMed] [Google Scholar]
- 10. Lijftogt N, Vahl AC, Wilschut ED, Elsman BHP, Amodio S, van Zwet EW et al. Adjusted hospital outcomes of abdominal aortic aneurysm surgery reported in the Dutch surgical aneurysm audit. Eur J Vasc Endovasc Surg 2017;53:520–532 [DOI] [PubMed] [Google Scholar]
- 11. Fischer C, Lingsma H, Hardwick R, Cromwell DA, Steyerberg E, Groene O. Risk adjustment models for short-term outcomes after surgical resection for oesophagogastric cancer. Br J Surg 2016;103:105–116 [DOI] [PubMed] [Google Scholar]
- 12. van Dishoeck AM, Lingsma HF, Mackenbach JP, Steyerberg EW. Random variation and rankability of hospitals using outcome indicators. BMJ Qual Saf 2011;20:869–874 [DOI] [PubMed] [Google Scholar]
- 13. Clavien PA, Barkun J, de Oliveira ML, Vauthey JN, Dindo D, Schulick RD et al. The Clavien–Dindo classification of surgical complications: five-year experience. Ann Surg 2009;250:187–196 [DOI] [PubMed] [Google Scholar]
- 14. Panhofer P, Ferenc V, Schutz M, Gleiss A, Dubsky P, Jakesz R et al. Standardization of morbidity assessment in breast cancer surgery using the Clavien Dindo classification. Int J Surg 2014;12:334–339 [DOI] [PubMed] [Google Scholar]
- 15. van Linschoten RCA, Amini M, van Leeuwen N, Eijkenaar F, den Hartog SJ, Nederkoorn PJ et al. Handling missing values in the analysis of between-hospital differences in ordinal and dichotomous outcomes: a simulation study. BMJ Qual Saf 2023;32:742–749 [DOI] [PubMed] [Google Scholar]
- 16. Van Buuren S, Groothuis-Oudshoorn KM. Multivariate imputation by chained equations in R. J Stat Softw 2011;45:1–67 [Google Scholar]
- 17. Ten Wolde B, Kuiper M, de Wilt JHW, Strobbe LJA. Postoperative complications after breast cancer surgery are not related to age. Ann Surg Oncol 2017;24:1861–1867 [DOI] [PubMed] [Google Scholar]
- 18. Al-Hilli Z, Wilkerson A. Breast surgery: management of postoperative complications following operations for breast cancer. Surg Clin North Am 2021;101:845–863 [DOI] [PubMed] [Google Scholar]
- 19. Jonczyk MM, Jean J, Graham R, Chatterjee A. Trending towards safer breast cancer surgeries? Examining acute complication rates from a 13-year NSQIP analysis. Cancers (Basel) 2019;11:253. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. de Boniface J, Szulkin R, Johansson ALV. Major surgical postoperative complications and survival in breast cancer: Swedish population-based register study in 57 152 women. Br J Surg 2022;109:977–983 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21. Al-Hilli Z, Thomsen KM, Habermann EB, Jakub JW, Boughey JC. Reoperation for complications after lumpectomy and mastectomy for breast cancer from the 2012 national surgical quality improvement program (ACS-NSQIP). Ann Surg Oncol 2015;22:S459–S469 [DOI] [PubMed] [Google Scholar]
- 22. Potter S, Trickey A, Rattay T, O'Connell RL, Dave R, Baker E et al. Therapeutic mammaplasty is a safe and effective alternative to mastectomy with or without immediate breast reconstruction. Br J Surg 2020;107:832–844 [DOI] [PubMed] [Google Scholar]
- 23. Berlin NL, Tandon VJ, Qi J, Kim HM, Hamill JB, Momoh AO et al. Hospital variations in clinical complications and patient-reported outcomes at 2 years after immediate breast reconstruction. Ann Surg 2019;269:959–965 [DOI] [PubMed] [Google Scholar]
- 24. Fischer C, Lingsma HF, van Leersum N, Tollenaar RAEM, Wouters MW, Steyerberg EW. Comparing colon cancer outcomes: the impact of low hospital case volume and case-mix adjustment. Eur J Surg Oncol 2015;41:1045–1053 [DOI] [PubMed] [Google Scholar]
- 25. Algera MD, Baldewpersad Tewarie NMS, Driel WJV, van Ham MAPC, Slangen BFM, Kruitwagen RFPM et al. Case-mix adjustment to compare hospital performances regarding complications after cytoreductive surgery for ovarian cancer: a nationwide population-based study. Int J Gynecol Cancer 2023;33:534–542 [DOI] [PubMed] [Google Scholar]
- 26. Lemij AA, van der Plas-Krijgsman WG, Bastiaannet E, Merkus JWS, van Dalen T, Vulink AJE et al. Predicting postoperative complications and their impact on quality of life and functional status in older patients with breast cancer. Br J Surg 2022;109:595–602 [DOI] [PubMed] [Google Scholar]
- 27. Jonczyk MM, Fisher CS, Babbitt R, Paulus JK, Freund KM, Czerniecki B et al. Surgical predictive model for breast cancer patients assessing acute postoperative complications: the breast cancer surgery risk calculator. Ann Surg Oncol 2021;28:5121–5131 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28. Mrad MA, Al Qurashi AA, Shah Mardan QNM, Alqarni MD, Alhenaki GA, Alghamdi MS et al. Predictors of complications after breast reconstruction surgery: a systematic review and meta-analysis. Plast Reconstr Surg Glob Open 2022;10:e4693. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29. Elfrink AKE, van Zwet EW, Swijnenburg RJ, den Dulk M, van den Boezem PB, Mieog JSD et al. Case-mix adjustment to compare nationwide hospital performances after resection of colorectal liver metastases. Eur J Surg Oncol 2021;47:649–659 [DOI] [PubMed] [Google Scholar]
- 30.Federatie Medisch Specialisten (FMS). Guideline breast reconstruction: 5. Plastic surgeon for oncoplastic surgery. https://richtlijnendatabase.nl/en/richtlijn/breast_reconstruction/plastic_surgeon_for_oncoplastic_surgery.html (accessed 23 February 2024)
- 31. Sterne JA, White IR, Carlin JB, Spratt M, Royston P, Kenward MG et al. Multiple imputation for missing data in epidemiological and clinical research: potential and pitfalls. BMJ 2009;338:b2393. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32. Ong WL, Schouwenburg MG, van Bommel ACM, Stowell C, Allison KH, Benn KE et al. A standard set of value-based patient-centered outcomes for breast cancer: the International Consortium for Health Outcomes Measurement (ICHOM) initiative. JAMA Oncol 2017;3:677–685 [DOI] [PubMed] [Google Scholar]
- 33. Dehal A, Abbas A, Johna S. Comorbidity and outcomes after surgery among women with breast cancer: analysis of nationwide in-patient sample database. Breast Cancer Res Treat 2013;139:469–476 [DOI] [PubMed] [Google Scholar]
- 34. Beck N, van Bommel AC, Eddes EH, van Leersum NJ, Tollenaar RA, Wouters MW et al. The Dutch Institute for Clinical Auditing: achieving Codman's dream on a nationwide basis. Ann Surg 2020;271:627–631 [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
The authors used data from the NABON breast cancer audit (NBCA), a nationwide clinical audit. Data are available upon reasonable request.