Skip to main content
PMC home page
Cancers logoLink to Cancers
. 2021 Oct 11;13(20):5077. doi: 10.3390/cancers13205077

Hospital Variation in Cancer Treatments and Survival OutComes of Advanced Melanoma Patients: Nationwide Quality Assurance in The Netherlands

Jesper van Breeschoten 1,2, Alfonsus J M van den Eertwegh 2, Liesbeth C de Wreede 3, Doranne L Hilarius 4, Erik W van Zwet 3, John B Haanen 5, Christian U Blank 5,6, Maureen J B Aarts 7, Franchette W P J van den Berkmortel 8, Jan Willem B de Groot 9, Geke A P Hospers 10, Ellen Kapiteijn 11, Djura Piersma 12, Rozemarijn S van Rijn 13, Marion A M Stevense-den Boer 14, Astrid A M van der Veldt 15, Gerard Vreugdenhil 16, Marye J Boers-Sonderen 17, Karijn P M Suijkerbuijk 18, Michel W J M Wouters 1,3,19,*
PMCID: PMC8533953  PMID: 34680228

Abstract

Simple Summary

The survival of advanced melanoma patients has improved significantly over the last decade due to the introduction of new systemic therapies. It is unknown whether survival outcomes of advanced melanoma patients differ between melanoma centers in the Netherlands. This research aimed to assess center variation in treatments and 2-year survival probabilities of advanced melanoma patients diagnosed between 2013 and 2017 in the Netherlands. Significant center variation in 2-year survival probabilities of patients diagnosed in 2014–2015 was observed after correcting for case-mix and treatment with new systemic therapies. The different use of new systemic therapies partially explained the observed variation. From 2016 onwards, no significant difference in 2-year survival was observed between centers. This study shows the added value of quality monitoring with a national registry that enables the study of variation between centers.

Abstract

Background: To assure a high quality of care for patients treated in Dutch melanoma centers, hospital variation in treatment patterns and outcomes is evaluated in the Dutch Melanoma Treatment Registry. The aim of this study was to assess center variation in treatments and 2-year survival probabilities of patients diagnosed between 2013 and 2017 in the Netherlands. Methods: We selected patients diagnosed between 2013 and 2017 with unresectable IIIC or stage IV melanoma, registered in the Dutch Melanoma Treatment Registry. Centers’ performance on 2-year survival was evaluated using Empirical Bayes estimates calculated in a random effects model. Treatment patterns of the centers with the lowest and highest estimates for 2-year survival were compared. Results: For patients diagnosed between 2014 and 2015, significant center variation in 2-year survival probabilities was observed even after correcting for case-mix and treatment with new systemic therapies. The different use of new systemic therapies partially explained the observed variation. From 2016 onwards, no significant difference in 2-year survival was observed between centers. Conclusion: Our data suggest that between 2014 and 2015, after correcting for patient case-mix, significant variation in 2-year survival probabilities between Dutch melanoma centers existed. The use of new systemic therapies could partially explain this variation. In 2013 and between 2016 and 2017, no significant variation between centers existed.

Keywords: advanced melanoma, survival, center variation

1. Introduction

The systemic treatment of metastatic melanoma has rapidly evolved over the last decade. Before 2011, unresectable stage IIIC and IV melanoma patients had few treatment options. However, since the introduction of the anti-CTLA-4 inhibitor ipilimumab (2011) [1], many new systemic treatment options have been approved (Figure S1). The approval of anti-PD-1 monoclonal antibodies (2015) [2,3], BRAF-inhibitors (2013, 2013, 2019), and MEK-inhibitors (2014, 2015, 2019) [4,5,6,7] and combination therapy of ipilimumab plus nivolumab [8] have resulted in improved overall survival (OS) of advanced melanoma patients over the last decade [9,10,11]. In the Netherlands, after approval by the EMA, an independent national committee has to approve the individual entities for reimbursement. This approval for reimbursement came in 2015 for nivolumab and pembrolizumab and in 2016 for dabrafenib in combination with trametinib and ipilimumab plus nivolumab.

The introduction of these new systemic treatments posed several challenges for healthcare specialists as well as policymakers. First, patient groups had to be identified that benefit most from the new systemic therapies. Secondly, these new therapies are expensive, and there were doubts about their cost-effectiveness. Finally, experience in recognizing and treating severe adverse events that occur in administering checkpoint inhibitors was initially limited.

To address these challenges, since 2012, every patient with unresectable stage IIIc or IV melanoma in the Netherlands has been treated in a dedicated melanoma center and registered in the Dutch Melanoma Treatment Registry (DMTR). The setup of this national registry was one of two conditions the minister of health dictated as a condition for approval of the implementation of the new therapies. The second condition was the centralization of the care of patients with advanced melanoma. Based on their expertise in treating advanced melanoma and their geographic location (Figure S2), fourteen melanoma centers were selected by the Dutch Society of Medical Oncologists (NVMO). Data from this nationwide registry are, amongst other purposes, used to develop quality indicators to assess hospital performance.

In surgery, the benchmarking of performance on quality indicators is common [12]. In other domains such as medical oncology, the benchmarking of performance between centers and individual oncologists is not yet widely implemented. The primary aim of this study was to assess center variation in survival up to two years after diagnosis with advanced melanoma in 2013–2017 in the Netherlands. The secondary aim was to investigate whether differences in treatment patterns may be related to survival outcomes.

2. Materials and Methods

2.1. Study Design and Population

This study used data from the Dutch Melanoma Treatment Registry (DMTR). Data are prospectively collected from diagnosis until death. Follow-up is checked every three months by a trained data manager for changes in disease status, patients or disease characteristics, and treatment. A detailed description of the DMTR setup has been published by Jochems et al. [13]. For this study, all patients aged ≥ 18 years and older, diagnosed between 1 January 2013 and 31 December 2017 with unresectable stage IIIC-IV melanoma, were included. Patients with uveal and mucosal melanoma were excluded from this analysis.

2.2. Melanoma Centers

All fourteen melanoma centers in the Netherlands were included in this study. Of these fourteen centers, seven centers are academic centers, and seven centers are general hospitals. Melanoma centers are spread across the country to provide patients with the best geographical coverage (Figure S2). Patients are assigned to the center where a medical oncologist first saw them.

2.3. Predictors of Overall Survival

Prognostic patient and tumor characteristics used for the adjustment of differences in case-mix between the 14 melanoma centers were based on previous studies [14,15]. All regression models included: age at diagnosis, gender (male, female), baseline Eastern Cooperative Oncology Group Performance Status (ECOG PS) (0–1, ≥2), baseline lactate dehydrogenase levels (LDH; normal, 250–500 U/L, >500 U/L), organs with distant metastasis (<3 organ sites, ≥3 organ sites involved), brain metastasis (none, asymptomatic, and symptomatic), liver metastasis (yes, no), and BRAFV600 mutational status (wild-type, mutant).

2.4. Statistical Analysis

Baseline patient and disease characteristics were analyzed using descriptive statistics. OS was defined as the time from diagnosis with unresectable IIIc or stage IV disease until death from any cause. Patients alive or lost to follow-up were right-censored at the time of last registered contact. Patients with a follow-up longer than 2 years were artificially censored at 2 years. We chose 2-year survival as our primary outcome as 2-year follow-up was completed for almost all patients diagnosed between 2013 and 2017.

Cox proportional hazard models were used to assess the association of prognostic factors with survival probability during the first 2 years after diagnosis. Patient, tumor, and center characteristics were included in the first Cox multivariable regression model with a shared frailty (random effect) for center ID for up to 2-year survival probability to adjust for differences in patient case-mix between different centers (model I). In all models, we assumed that the frailty followed a gamma distribution. Shared frailty models attempt to separate true differences between centers from random fluctuation. These models yield “Empirical Bayes estimates” of the residual center effects, which were expressed as “center hazard ratios (HRs)” with respect to the national average [16]. To assess the influence of the use of new systemic therapies on differential outcomes in the centers, we extended the first model with first-line treatment with new systemic therapies (anti-PD-1 monotherapy, BRAF/MEK inhibitors, and/or ipilimumab plus nivolumab) as a patient characteristic (model II) [17]. In the final model (model III), we added a time-dependent covariate of new systemic therapies (anti-PD-1 monotherapy, BRAF/MEK inhibitors, and/or ipilimumab plus nivolumab) to model I. The three models used in this study are summarized below:

  • I.

    Cox regression model for OS with patient and tumor characteristics as covariates and center ID modeled by a random effect;

  • II.

    Cox regression model for OS with patient and tumor characteristics as covariates and center ID modeled by a random effect and baseline first-line systemic therapy as a patient characteristic;

  • III.

    Cox regression model for OS with patient and tumor characteristics as covariates and center ID modeled by a random effect and new systemic therapies (anti-PD-1 monotherapy, BRAF/MEK inhibitors, and ipilimumab plus nivolumab) as a time-dependent covariate.

To determine which factors were associated with up to 2-year survival in all diagnosis years, we used model III. Treatment patterns of the three centers with the highest and lowest center HRs for patients diagnosed between 2013–2015 and 2016–2017 were compared using Sankey diagrams [18]. We chose these years as anti-PD-1, and the combination of BRAF/MEK inhibitors was available from 2015.

The degree of hospital variation between centers was expressed using median hazard ratios (mHRs). This concept has been described according to the methods by Austin et al. and can be interpreted as the median increase in hazard of mortality when comparing a patient at a hospital with higher mortality to a patient at a hospital with lower mortality [19]. The mHR was calculated and described by Austin et al., separately for each year of diagnosis for model I to III. Center variation was considered significant when the lower limit of the 95% confidence interval of the mHR was > 1.00.

Statistical software used was R [20] (version 4.0.1 ; packages car [21], tidyverse [22], survival [23], survminer [24], frailtyEM [25] and ggalluvial [26]).

3. Results

Between 2013 and 2017, a total of 3820 patients were diagnosed with unresectable stage IIIC/IV melanoma (Figure S3). Center size ranged from 110 in the smallest center to 955 patients diagnosed between 2013 and 2017 in the largest center. Baseline patient and tumor characteristics are shown in Table 1. Overall, patients had a median age of 64, ECOG PS 0–1 (84.8%), stage IV-M1c disease (76.7%), elevated LDH (36.8%), brain metastases (28.4%), liver metastases (29.1%), BRAFV600 mutation (45.6%), and metastases in ≥3 organ sites (46.4%). Patient and tumor characteristics of patients diagnosed between 2013 and 2017 for each center separately are shown in Table S1, and the number of patients is shown in Table S2. Median follow-up of all patients was 13.3 months (95%CI 12.5–14.2).

Table 1.

Patient and tumor characteristics of patients diagnosed between 2013 and 2017. If ≥5% of data are missing for a variable, this is indicated by the category “unknown”.

Variable Level Total (N) = 3820
Age (median (range)) 64 (18–97)
Gender (%) Male 2254 (59.0)
Female 1565 (41.0)
ECOG PS (%) 0–1 2832 (84.8)
≥2 507 (15.2)
Unknown 481 (12.6)
Stage (%) IIIc unresectable 190 (5.0)
IV-M1a 297 (7.8)
IV-M1b 397 (10.5)
IV-M1c 2903 (76.7)
LDH (%) Normal 2215 (63.1)
1–2 × ULN 816 (23.2)
>2 × ULN 479 (13.6)
Unknown/not determined 310 (8.1)
Brain metastases (%) No 2672 (71.5)
Yes, asymptomatic 334 (8.9)
Yes, symptomatic 730 (19.5)
Liver metastases (%) No 2667 (70.9)
Yes 1094 (29.1)
Organ sites (%) <3 2047 (53.6)
≥3 1773 (46.4)
BRAFV600 mutation (%) Wild-type 1741 (45.6)
Mutant 2079 (54.4)
Open in a new tab

3.1. Factors Associated with 2-Year Survival

To determine which factors were associated with up to 2-year survival, we used a multivariable shared frailty Cox model (model III). The results show that each year of age (HR = 1.02, 95%CI: 1.01–1.02), ECOG PS ≥2 (HR = 1.91, 95%CI: 1.71–2.14), elevated LDH (250–500 U/L HR = 1.33, 95%CI: 1.20–1.47 and >500 U/L HR = 2.33, 95%CI: 2.06–2.64), brain metastases (asymptomatic HR = 1.44, 95%CI: 1.26–1.65 and symptomatic HR = 1.82, 95%CI: 1.65–2.01), liver metastases (HR = 1.31, 95%CI: 1.18–1.44), number of organ sites ≥3 (HR = 1.50, 95%CI: 1.37–1.65), and BRAFV600 wild-type (HR = 1.16, 95%CI: 1.07–1.26) were significantly associated with higher risk of death during the first 2 years since diagnosis (Table S3).

3.2. Hospital Variation

Between 2013 and 2017, 2-year survival probabilities increased from 29% to 44% for all centers together. Based on the mHR of model I to III, significant variation between centers in 2-year survival probabilities existed in 2014–2015, but there was no significant variation for patients diagnosed in 2013 and between 2016 and 2017 (Figure 1). Model-based center-specific HRs of the different centers corrected for case-mix covariates are shown in Figure 2. In the year with the largest variation (2014), between-center HRs ranged between 0.72 and 1.36 relative to the national average. Between 2013 and 2015, mHR was 1.13. Figure 3 shows the model III-based survival curves for a male 80-year-old reference patient with 1 × elevated LDH, an ECOG PS of ≥2 with 0–2 organ sites, no liver or brain metastases, and BRAFV600 wild-type tumor diagnosed in 2014 vs. the same patient diagnosed in 2016.

Figure 1.

Figure 1

Open in a new tab

Hospital variation in 2-year survival probabilities expressed as mHRs in three Cox frailty models with gender, age at diagnosis, ECOG PS, LDH levels, number of organ sites, liver metastases, brain metastases, and BRAF mutational status as covariates and year of diagnosis as a factor. Intervals represent 95% confidence intervals of the mHRs. A mHR of 1.00 means that no center variation is present. The larger the mHRs, the larger the variation in 2-year probabilities between centers. The figure shows that significant variation in 2-year survival probabilities was significant between 2013 and 2015 and not significant in 2016 and 2017.

Figure 2.

Figure 2

Open in a new tab

HR for death until 2 years of each center between 2013 and 2017 in model III. Higher ratios mean that a center has a lower observed/expected ratio for 2-year survival corrected for gender, age at diagnosis, ECOG PS, LDH levels, number of organ sites, liver metastases, brain metastases, and BRAFV600 mutational status as covariates in a specific year of diagnosis. HRs = hazard ratios.

Figure 3.

Figure 3

Open in a new tab

Model III-based predicted survival curves of patients diagnosed in 2014 (left) and 2016 (right) with the following characteristics: 80-year-old male, with 1 × elevated LDH, ECOG PS of ≥2 with 0–2 organ sites, no liver or brain metastases, BRAF wild-type tumor, who would be treated in the centers with the highest, lowest, and average HR in the dataset. HR = hazard ratio.

3.3. Treatment Patterns

Treatment patterns were different in the centers over the years due to the introduction of anti-PD-1, BRAF/MEK combination therapy, and combination therapy of ipilimumab plus nivolumab. Treatment patterns of the three centers with the lowest HRs (centers with the best outcome) and the highest HRs (centers with the worst outcome) for death until 2 years are shown in Figure 4 and Figure 5. Overall, the centers with the lowest HRs treated a higher percentage of patients with anti-PD-1 antibodies (31% vs. 20%) and BRAF/MEK inhibitors (19% vs. 5%) in the first three lines of treatment compared to centers with higher HRs between 2013 and 2015. Compared to the treatment patterns for patients diagnosed in 2013–2015, this is different for patients diagnosed in 2016–2017. Overall, a majority of patients diagnosed between 2016 and 2017 received anti-PD-1 antibodies (48% vs. 60%) and/or BRAF/MEK combination therapy (40% vs. 29%) and/or ipilimumab + nivolumab (26% vs. 7%) either in the first, second, or third line for centers with the highest HRs (lowest 2-year survival) vs. centers with the lowest HRs (highest 2-year survival), respectively.

Figure 4.

Figure 4

Open in a new tab

Stacked bar plots showing the treatments in the first, second, and third line of treatment. The three hospitals with the highest HRs (N = 413) and lowest 2-year survival are compared to the three hospitals with the lowest HRs (N = 674) according to the model. The top row shows patients diagnosed between 2013 and 2015, and the bottom row shows patients diagnosed between 2016 and 2017. Treatments administered after 3 December 2020 and labels of percentages < 1% are not shown in this barplot.

Figure 5.

Figure 5

Open in a new tab

Bar plots showing the percentage of patients with a specific systemic therapy somewhere in their treatment after diagnosis with advanced melanoma. Yellow bars represent patients diagnosed between 2013–2015, and the green bar plots show patients diagnosed between 2016 and 2017. Columns depicted by “highest” are the three centers with the highest HRs for 2-year survival, and “lowest” are the centers with the lowest HRs for 2-year survival.

3.4. Impact of Anti-PD-1 and BRAF/MEK Inhibitors on Survival Probabilities

Depending on the center, 3–29% of patients diagnosed between 2013 and 2015 were treated with first-line anti-PD-1, BRAF/MEK combination therapy, and/or ipilimumab plus nivolumab anywhere in the first three lines of systemic therapy. Treatment with these systemic therapies in the first line was significantly associated with a lower hazard of death until 2 years after diagnosis (model II with all years combined: HR = 0.53, 95%CI: 0.49–0.58, p-value < 0.001) over all years of diagnosis. The inclusion of treatment with new systemic therapies as a time-dependent variable was also significantly associated with a lower hazard of death until 2 years after diagnosis (model III with all years combined: HR = 0.66, 95%CI: 0.61–0.72, p-value < 0.001).

4. Discussion

Based on a nationwide prospective registry, this report describes hospital variation in 2-year survival probabilities and treatment patterns of advanced melanoma patients between 2013 and 2017 in the Netherlands. Between 2013 and 2017, nationwide 2-year survival probabilities of advanced melanoma increased from 29% to 44%. The observed variation between centers was statistically significant after correcting for case-mix and treatment with new systemic therapies in 2014–2015 and was non-significant in 2013 and 2016–2017. Two-year survival probabilities were significantly associated with the first-line treatment of anti-PD-1, BRAF/MEK inhibitors, and/or ipilimumab plus nivolumab (HR = 0.62, 95%CI: 0.54–0.73, p-value < 0.001), with higher 2-year survival probabilities in centers that treated more patients in the first line with these therapies. Between 2013 and 2015, mHR was 1.13.

Until now, no study has described that hospital variation in survival outcomes for advanced melanoma patients may be related to differences in treatment patterns between centers. This study shows the added value of quality monitoring with a national registry that enables the study of variation between centers. Despite the constantly changing therapeutic landscape between 2013 and 2017, Dutch melanoma centers have ensured that, currently, no significant treatment and 2-year survival variation between centers exists. This may be attributed to several reasons. Firstly, in 2012, Dutch melanoma care was centralized to fourteen centers. This resulted in the establishment of more high-volume melanoma centers. A previous study has shown that for patients with advanced melanoma, treatment in high-volume hospitals (n > 10) was associated with improved OS [27]. This study does not observe such an effect, possibly by the minimum volume of twenty newly diagnosed patients each year. Secondly, the fourteen melanoma centers meet quarterly in scientific meetings, sharing new developments and best practices. This may have enabled the quick transmission of knowledge and practice across the different centers.

The current study shows that center variation in 2-year survival probabilities was only significant in patients diagnosed between 2014 and 2015. Approval and reimbursement of these new systemic therapies happened mainly in 2014–2015. However, some centers had access to these treatments before an official decision on reimbursement by participation in trials, compassionate use, or extended access programs, which might explain the variation in 2-year survival probabilities. The bar plots presented in this study show this effect, as the best performing centers have higher percentages of these new treatments in the first, second, and third line of anti-PD-1 antibodies (31% vs. 20%) and BRAF/MEK inhibitors (19% vs. 5%) between 2013 and 2015 (Figure 5). Over the years, this inequality in access has disappeared, resulting in complete access to these systemic treatment options in every melanoma center in the Netherlands. However, the percentage of patients treated with combination therapy of ipilimumab plus nivolumab remains different between the centers with the lowest and highest HRs (26% vs. 7%), but this does not lead to significant differences in 2-year survival. Future research should focus on current differences between centers in using combination therapy of ipilimumab plus nivolumab.

A previous real-world study of treatment patterns in the USA using observational data shows that about 15% of all patients received first-line treatment with ipilimumab in 2016 [28]. In comparison, in our study, we observe almost no patients diagnosed between 2016 and 2017 treated with ipilimumab monotherapy as a first-line treatment across all centers in the Netherlands. This can be explained by the study of Robert et al., who showed that treatment with pembrolizumab resulted in a better OS as compared to the treatment with ipilimumab [2], or by Larkin et al. [29], who showed that combination treatment with nivolumab and ipilimumab resulted in significantly better OS compared to ipilimumab monotherapy. Rates of patients receiving anti-PD-1, BRAF/MEK-inhibitors, and combination therapy of ipilimumab plus nivolumab are comparable across both studies. However, compared to randomized clinical trials, 2-year survival probabilities in this cohort study are lower [7,30], possibly due to the inclusion of older patients, more patients with stage IV-M1c disease, and a higher ECOG PS of ≥2 [31].

The limitation of this study is that Cox models with a frailty component shrink the estimates compared to crude estimates. These models allow for the separation of observed center effects into explained variation by case-mix and center characteristics, unexplained differences between center outcomes, and random fluctuations [32]. Using these center-specific estimates helps in the interpretation of outcomes of small centers and prevents over-estimation of the effects. However, as a downside of using these frailties, the effects of some centers may be underestimated, and differences between the worst and best-performing centers may even be larger than presented in this study [17].

Based on the presented results, significant hospital variation in 2-year OS of advanced melanoma patients existed in 2014–2015 in the Netherlands. This significant hospital variation can partially be explained by case-mix and differences in percentages of patients treated with anti-PD-1, BRAF/MEK combination therapy, and combination therapy of ipilimumab and nivolumab. From 2016 onwards, no significant hospital variation corrected for case-mix was found, possibly due to similar treatment patterns across all centers, as a result of the nationwide collaboration of medical oncologists in the DMTR. Currently, the two-year survival probabilities of advanced melanoma patients in the Netherlands are comparable and do not differ between centers. However, in countries with no quality assurance in the form of centralization and a national registry with feedback, variations in outcomes between hospitals treating advanced melanoma patients might still be present.

5. Conclusions

In conclusion, we show significant differences in 2-year survival between Dutch melanoma centers between 2014 and 2015. The different pace of implementation of new cancer treatment of metastatic melanoma affected survival outcomes, but this difference disappeared in later years. To improve overall survival, new registered cancer treatments must be implemented by collaboration between medical professionals, regulatory authorities, and the organization within hospitals.

Acknowledgments

The authors thank all physicians and data managers who registered the patient data in the Dutch Melanoma Treatment Registry.

Supplementary Materials

The following are available online at https://www.mdpi.com/article/10.3390/cancers13205077/s1, Figure S1: Flow of new systemic therapies, Figure S2: Geographical location of melanoma centers, Figure S3: Flow chart of the included patients, Table S1: Patient and tumor characteristics of patients in each individual center (2013–2017), Table S2: Number of newly diagnosed advanced melanoma patients per center, Table S3: Multivariable Cox regression for OS.

Click here for additional data file. (336.5KB, zip)

Author Contributions

Conceptualization, J.v.B., A.J.M.v.d.E., L.C.d.W., D.L.H., E.W.v.Z., J.B.H., C.U.B., M.J.B.A., F.W.P.J.v.d.B., J.W.B.d.G., G.A.P.H., E.K., D.P., R.S.v.R., M.A.M.S.-d.B., A.A.M.v.d.V., G.V., M.J.B.-S., K.P.M.S. and M.W.J.M.W.; Formal analysis, J.v.B., A.J.M.v.d.E., L.C.d.W., D.L.H., E.W.v.Z. and M.W.J.M.W.; Methodology, J.v.B., A.J.M.v.d.E., L.C.d.W., E.W.v.Z. and M.W.J.M.W.; Investigation, A.J.M.v.d.E., J.B.H., C.U.B., M.J.B.A., F.W.P.J.v.d.B., J.W.B.d.G., G.A.P.H., E.K., D.P., R.S.v.R., M.A.M.S.-d.B., A.A.M.v.d.V., G.V., M.J.B.-S., K.P.M.S. and M.W.J.M.W.; Writing—original draft, J.v.B., A.J.M.v.d.E., L.C.d.W., D.L.H. and M.W.J.M.W.; Writing—review and editing, J.v.B., A.J.M.v.d.E., L.C.d.W., D.L.H., E.W.v.Z., J.B.H., C.U.B., M.J.B.A., F.W.P.J.v.d.B., J.W.B.d.G., G.A.P.H., E.K., D.P., R.S.v.R., M.A.M.S.-d.B., A.A.M.v.d.V., G.V., M.J.B.-S., K.P.M.S. and M.W.J.M.W. All authors have read and agreed to the published version of the manuscript.

Funding

For the Dutch Melanoma Treatment Registry (DMTR), the Dutch Institute for Clinical Auditing foundation received a start-up grant from governmental organization The Netherlands Organization for Health Research and Development (ZonMW, project number 836002002). The DMTR is structurally funded by Bristol Myers Squibb, Merck Sharpe & Dohme, Novartis and Roche Pharma. Roche Pharma stopped funding in 2019 and Pierre Fabre started funding of the DMTR in 2019. For this work, no funding was granted.

Institutional Review Board Statement

In compliance with Dutch regulations, the DMTR was approved by a medical ethical committee (METC Leiden University Medical Center, 2013) and is not considered subject to the Medical Research Involving Human Subjects Act.

Informed Consent Statement

In the participation agreement between DICA and the participants in clinical audits, to which the DICA Regulations also apply, further conditions are set for the use of data for scientific research. These conditions include that only pseudonymised or anonymised data that cannot (directly) be traced back to individual patients may be made available. The parties also agree that, in accordance with the applicable laws and regulations, the participants will set up a system of no-objection that offers the patient the possibility to explicitly object to the use of his/her data for scientific research in advance. In this way, the patient’s privacy will be protected as much as possible.

Data Availability Statement

The datasets generated during and/or analyzed during the current study are not publicly available due to privacy regulations in the Netherlands but are available from the corresponding author on reasonable request.

Conflicts of Interest

A.v.d.E. has advisory relationships with Amgen, Bristol Myers Squibb, Roche, Novartis, MSD, Pierre Fabre, Sanofi, Pfizer, Ipsen, and Merck, has received research study grants not related to this paper from Sanofi, Roche, Bristol Myers Squibb, Idera, and TEVA, has received travel expenses from MSD Oncology, Roche, Pfizer, and Sanofi, and has received speaker honoraria from BMS and Novartis. MBS has consultancy/advisory relationships with Pierre Fabre, MSD, and Novartis. J.d.G. has consultancy/advisory relationships with Bristol Myers Squibb, Pierre Fabre, Servier, MSD, and Novartis. G.H. has consultancy/advisory relationships with Amgen, Bristol Myers Squibb, Roche, MSD, Pfizer, Novartis, and Pierre Fabre and has received research grants not related to this paper from Bristol Myers Squibb, and Seerave, which were paid to the institution. E.K. has consultancy/advisory relationships with Bristol Myers Squibb, Novartis, Merck, and Pierre Fabre and received research grants not related to this paper from Bristol Myers Squibb. KS has advisory relationships with Bristol Myers Squibb, Novartis, MSD, Pierre Fabre, and Abbvie and received honoraria from Novartis, MSD, and Roche. A.v.d.V. has consultancy relationships with Bristol Myers Squibb, MSD, Roche, Novartis, Pierre Fabre, Pfizer, Sanofi, Ipsen, Eisai, and Merck. C.U.B. has received commercial research grants from Novartis, BristolMyers Squibb, and NanoString; is a paid advisory board member for Bristol Myers Squibb, MSD, Roche, Novartis, GlaxoSmithKline, AstraZeneca, Pfizer, Lilly, GenMab, and Pierre Fabre; and holds ownership interest in Uniti Cars, Neon Therapeutics, and Forty Seven. J.H. has advisory relationships with Aimm, Achilles Therapeutics, Amgen, AstraZeneca, Bayer, Bristol Myers Squibb BioNTech, GSK, Immunocore, Ipsen, MSD, Merck Serono, Molecular Partners, Novartis, Neogene Therapeutics, Pfizer, Roche/Genentech, Sanofi, Seattle Genetics, Third Rock Ventures, and Vaximm and has received research grants not related to this paper from Amgen, Bristol Myers Squibb, MSD, BioNTech, Neogene Therapeutics, and Novartis. MA has advisory board/consultancy honoraria from Amgen, Bristol Myers Squibb, Novartis, MSD-Merck, Merck-Pfizer, Pierre Fabre, Sanofi, Astellas, and Bayer and received research grants not related to the current work and paid to the institute from Merck-Pfizer. RS has advisory board/consultancy honoraria from Pfizer and an expert meeting fee from Roche. All grants were paid to the institutions. The funders had no role in the writing of this article or decision to submit it for publication. All remaining authors have declared no conflict of interest.

Footnotes

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

References

  • 1.Hodi F.S., Sosman J.A., Haanen J.B., Gonzalez R., Robert C., Schadendorf D., Hassel J.C., Akerley W., Van Den Eertwegh A.J.M. Improved Survival with Ipilimumab in Patients with Metastatic Melanoma. N. Engl. J. Med. 2010;8:711–723. doi: 10.1056/NEJMoa1003466. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Robert C., Schachter J., Long G.V., Arance A., Grob J.J., Mortier L., Daud A., Carlino M.S., McNeil C., Lotem M., et al. Pembrolizumab versus ipilimumab in advanced melanoma. N. Engl. J. Med. 2015;26:2521–2532. doi: 10.1056/NEJMoa1503093. [DOI] [PubMed] [Google Scholar]
  • 3.Robert C., Long G.V., Brady B., Dutriaux C., Maio M., Mortier L., Hassel J.C., Rutkowski P., McNeil C., Kalinka-Warzocha E., et al. Nivolumab in previously untreated melanoma without BRAF mutation. N. Engl. J. Med. 2015;4:320–330. doi: 10.1056/NEJMoa1412082. [DOI] [PubMed] [Google Scholar]
  • 4.Ascierto P.A., McArthur G.A., Dréno B., Atkinson V., Liszkay G., Di Giacomo A.M., Mandalà M., Demidov L., Stroyakovskiy D., Thomas L., et al. Cobimetinib combined with vemurafenib in advanced BRAFV600-mutant melanoma (coBRIM): Updated efficacy results from a randomised, double-blind, phase 3 trial. Lancet Oncol. 2016;9:1248–1260. doi: 10.1016/S1470-2045(16)30122-X. [DOI] [PubMed] [Google Scholar]
  • 5.Chapman P.B., Hauschild A., Robert C., Haanen J.B., Ascierto P., Larkin J., Dummer R., Garbe C., Testori A., Maio M., et al. Improved survival with vemurafenib in melanoma with BRAF V600E mutation. N. Engl. J. Med. 2011;26:2507–2516. doi: 10.1056/NEJMoa1103782. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Hauschild A., Grob J.J., Demidov L.V., Jouary T., Gutzmer R., Millward M., Rutkowski P., Blank C.U., Miller W.H., Kaempgen E., et al. Dabrafenib in BRAF-mutated metastatic melanoma: A multicentre, open-label, phase 3 randomised controlled trial. Lancet. 2012;9839:358–365. doi: 10.1016/S0140-6736(12)60868-X. [DOI] [PubMed] [Google Scholar]
  • 7.Robert C., Karaszewska B., Schachter J., Rutkowski P., Mackiewicz A., Stroiakovski D., Lichinitser M., Dummer R., Grange F., Mortier L., et al. Improved overall survival in melanoma with combined dabrafenib and trametinib. N. Engl. J. Med. 2015;1:30–39. doi: 10.1056/NEJMoa1412690. [DOI] [PubMed] [Google Scholar]
  • 8.Larkin J., Chiarion-Sileni V., Gonzalez R., Grob J.-J., Rutkowski P., Lao C.D., Cowey C.L., Schadendorf D., Wagstaff J., Dummer R., et al. Five-Year Survival with Combined Nivolumab and Ipilimumab in Advanced Melanoma. N. Engl. J. Med. 2019;16:1535–1546. doi: 10.1056/NEJMoa1910836. [DOI] [PubMed] [Google Scholar]
  • 9.Donia M., Ellebaek E., Øllegaard T.H., Duval L., Aaby J.B., Hoejberg L., Køhler U.H., Schmidt H., Bastholt L., Svane I.M. The real-world impact of modern treatments on the survival of patients with metastatic melanoma. Eur. J. Cancer. 2019;108:25–32. doi: 10.1016/j.ejca.2018.12.002. [DOI] [PubMed] [Google Scholar]
  • 10.European Medicines Agency Keytruda. [(accessed on 14 July 2020)]. Available online: https://www.ema.europa.eu/en/medicines/human/EPAR/keytruda#authorisation-details-section/
  • 11.European Medicines Agency Opdivo. [(accessed on 14 July 2020)]. Available online: https://www.ema.europa.eu/en/search/search?search_api_views_fulltext=opdivo/
  • 12.Beck N., Van Bommel A.C., Eddes E.H., Van Leersum N.J., Tollenaar R.A., Wouters M.W. The Dutch Institute for Clinical Auditing: Achieving Codman’s Dream on a Nationwide Basis. Ann. Surg. 2020;4:627–631. doi: 10.1097/SLA.0000000000003665. [DOI] [PubMed] [Google Scholar]
  • 13.Jochems A., Schouwenburg M.G., Leeneman B., Franken M.G., van den Eertwegh A.J.M., Haanen J.B.A.G., Gelderblom H., Uyl-de Groot C.A., Aarts M.J.B., van den Berkmortel F.W.P.J., et al. Dutch Melanoma Treatment Registry: Quality assurance in the care of patients with metastatic melanoma in the Netherlands. Eur. J. Cancer. 2017;72:156–165. doi: 10.1016/j.ejca.2016.11.021. [DOI] [PubMed] [Google Scholar]
  • 14.Manola J., Atkins M., Ibrahim J., Kirkwood J. Prognostic factors in metastatic melanoma: A pooled analysis of Eastern Cooperative Oncology Group trials. J. Clin. Oncol. 2000;22:3782–3793. doi: 10.1200/JCO.2000.18.22.3782. [DOI] [PubMed] [Google Scholar]
  • 15.Long G.V., Grob J.J., Nathan P., Ribas A., Robert C., Schadendorf D., Lane S.R., Mak C., Legenne P., Flaherty K.T., et al. Factors predictive of response, disease progression, and overall survival after dabrafenib and trametinib combination treatment: A pooled analysis of individual patient data from randomised trials. Lancet Oncol. 2016;12:1743–1754. doi: 10.1016/S1470-2045(16)30578-2. [DOI] [PubMed] [Google Scholar]
  • 16.Schetelig J., de Wreede L.C., Andersen N.S., Moreno C., van Gelder M., Vitek A., Karas M., Michallet M., Machaczka M., Gramatzki M., et al. Centre characteristics and procedure-related factors have an impact on outcomes of allogeneic transplantation for patients with CLL: A retrospective analysis from the European Society for Blood and Marrow Transplantation (EBMT) Br. J. Haematol. 2017;4:521–533. doi: 10.1111/bjh.14791. [DOI] [PubMed] [Google Scholar]
  • 17.Therneau T.M., Grambsch P.M. Modeling Survival Data: Extending the Cox Model. Springer; New York, NY, USA: 2000. pp. 39–77. [Google Scholar]
  • 18.Schmidt M. The Sankey diagram in energy and material flow management: Part I: History. J. Ind. Ecol. 2008;1:82–94. doi: 10.1111/j.1530-9290.2008.00004.x. [DOI] [Google Scholar]
  • 19.Austin P.C., Wagner P., Merlo J. The median hazard ratio: A useful measure of variance and general contextual effects in multilevel survival analysis. Stat. Med. 2017;6:928–938. doi: 10.1002/sim.7188. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.R Core Team . R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing; Vienna, Austria: 2017. [Google Scholar]
  • 21.Fox J., Weisberg S. An R Companion to Applied Regression. Sage; Thousand Oaks, CA, USA: 2019. [Google Scholar]
  • 22.Wickham H., Averick M., Bryan J., Chang W., McGowan L.D., François R., Grolemund G., Hayes A., Henry L., Hester J., et al. Welcome to the tidyverse. J. Open Source Softw. 2019;43:1686. doi: 10.21105/joss.01686. [DOI] [Google Scholar]
  • 23.Thernau T.M. A Package for Survival Analysis in R. 2020. [(accessed on 11 October 2021)]. Available online: https://CRAN.R-project.org/package=survival.
  • 24.Kassambara A., Kosinski M., Biecek P., Fabian S. Drawing Survival Curves Using “ggplot2”. 2020. [(accessed on 11 October 2021)]. Available online: https://CRAN.R-project.org/package=survminer.
  • 25.Balan T.A., Putter H. frailtyEM: An R Package for Estimating Semiparametric Shared Frailty Models. J. Stat. Softw. 2019;7:1–29. doi: 10.18637/jss.v090.i07. [DOI] [Google Scholar]
  • 26.Brunson J.C. Ggalluvial: Layered Grammar for Alluvial Plots. J. Open Source Softw. 2020;49:2017. doi: 10.21105/joss.02017. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Huo J., Lairson D.R., Du X.L., Chan W., Jiang J., Buchholz T.A., Guadagnolo B.A. Hospital Case Volume Is Associated with Improved Survival for Patients with Metastatic Melanoma. Am. J. Clin. Oncol Cancer Clin. Trials. 2016;5:491–496. doi: 10.1097/COC.0000000000000074. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Tarhini A., Atzinger C., Gupte-Singh K., Johnson C., Macahilig C., Rao S. Treatment patterns and outcomes for patients with unresectable stage III and metastatic melanoma in the USA. J. Comp. Eff. Res. 2019;7:461–473. doi: 10.2217/cer-2019-0003. [DOI] [PubMed] [Google Scholar]
  • 29.Larkin J., Chiarion-Sileni V., Gonzalez R., Grob J.J., Cowey C.L., Lao C.D., Schadendorf D., Dummer R., Smylie M., Rutkowski P., et al. Combined Nivolumab and Ipilimumab or Monotherapy in Untreated Melanoma. N. Engl. J. Med. 2015;1:23–34. doi: 10.1056/NEJMoa1504030. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Ascierto P.A., Long G.V., Robert C., Brady B., Dutriaux C., Di Giacomo A.M., Mortier L., Hassel J.C., Rutkowski P., McNeil C., et al. Survival Outcomes in Patients with Previously Untreated BRAF Wild-Type Advanced Melanoma Treated with Nivolumab Therapy: Three-Year Follow-Up of a Randomized Phase 3 Trial. JAMA Oncol. 2019;2:187–194. doi: 10.1001/jamaoncol.2018.4514. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.van Zeijl M.C.T., Ismail R.K., de Wreede L.C., van den Eertwegh A.J.M., de Boer A., van Dartel M., Hilarius D.L., Aarts M.J.B., van den Berkmortel F.W.P.J., Boers-Sonderen M.J., et al. Real-world outcomes of advanced melanoma patients not represented in phase III trials. Int J. Cancer. 2020;12:3461–3470. doi: 10.1002/ijc.33162. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Glidden D.V., Vittinghoff E. Modelling clustered survival data from multicentre clinical trials. Stat. Med. 2004;3:369–388. doi: 10.1002/sim.1599. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Click here for additional data file. (336.5KB, zip)

Data Availability Statement

The datasets generated during and/or analyzed during the current study are not publicly available due to privacy regulations in the Netherlands but are available from the corresponding author on reasonable request.

Articles from Cancers are provided here courtesy of Multidisciplinary Digital Publishing Institute (MDPI)

RESOURCES