Validation, revision and extension of the Mantle Cell Lymphoma International Prognostic Index in a population-based setting

Abstract

Background

The aim of this study was to validate the Mantle Cell Lymphoma International Prognostic Index in a population-based cohort and to study the relevance of its revisions.

Design and Methods

We analyzed data from 178 unselected patients with stage III or IV mantle cell lymphoma, registered between 1994 and 2006 in the Eindhoven Cancer Registry. Follow-up was completed up to January 1^st, 2008. Multiple imputations for missing covariates were used. Validity was assessed by comparing observed survival in our cohort with predicted survival according to the original Mantle cell lymphoma International Prognostic Index. A revised model was constructed with Cox regression analysis. Discrimination was assessed by a concordance statistic (‘c’).

Results

The original Mantle cell lymphoma International Prognostic Index could stratify our cohort into three distinct risk groups based on Eastern Cooperative Group performance status, white blood cell count, lactate dehydrogenase level, and age, with the discrimination being nearly as good as in the original cohort (c 0.65 versus 0.63). A modified model including performance status in five categories (0/1/2/3/4) instead of two (0–1/2–4), the presence of B-symptoms (yes/no) and sex (male/female) in addition to the original variables resulted in a better prognostic index (c 0.75).

Conclusions

The Mantle cell lymphoma International Prognostic Index is a valid tool for risk stratification, comparison of prognosis, and treatment decisions in an unselected Dutch population-based setting. Although the index can be significantly improved, external validation on an independent data set is warranted before broad application of the modified instrument could be recommended.

Introduction

Mantle cell lymphoma (MCL) is a relatively rare lymphoma entity accounting for approximately 3% to 6% of all cases of non-Hodgkin’s lymphoma. It has a poor prognosis with a reported median overall survival of only 30 to 43 months. Treatment results have been unsatisfactory, although a substantial variation in outcome has been noted among individual cases with some patients achieving long-lasting remissions.¹ A validated prognostic index would greatly help in developing new treatment strategies based on risk and prognosis, and for evaluating and choosing between different available treatment options.

Recently, a new clinical prognostic index was proposed for MCL: the Mantle Cell Lymphoma International Prognostic Index (MIPI).¹ This index is based on data derived from three large randomized clinical trials and proposes three risk groups according to the probability of survival. The score was defined on 455 patients. Several candidate prognostic factors were included, but same of these were excluded in multiple regression, because of a high number of missing values.

In the original MIPI, a four-variable model included the risk factors Eastern Cooperative Group (ECOG) performance status, white blood cell (WBC) count, lactate dehydrogenase (LDH) level, and age.¹ However, the MIPI has not been validated yet. Validation is particularly important in a population-based setting in order to prove the usefulness of the index in a general health care environment including more patients with advanced age and/or severe co-morbidity. Restrictive eligibility criteria, such as advanced age, serious comorbidity, poor performance status and impairment of organ function, might have biased the results of the trial-based series.² We previously found that comorbid conditions were present in 48% of unselected patients with aggressive non-Hodgkin’s lymphoma under the age of 60, and in as many as 79% of those older than 60.³ Comorbidity, if serious enough, is an independent prognostic factor.^4–6

We recently showed that the performance of the Follicular Lymphoma International Prognostic Index (FLIPI) could be significantly improved by a more refined coding of age and by including the presence of cardiovascular disease.⁷ We, therefore, considered that the performance of the MIPI might also be improved by adding others risk factors, including comorbidity.

The aim of this study was to validate the original MIPI in a population-based cohort and to study possibilities for improving the index.

Design and Methods

Study population and data collection

The Eindhoven Cancer Registry records data on all patients newly diagnosed with cancer in the southern part of the Netherlands, an area with 2.4 million inhabitants, ten general hospitals and two large radiotherapy institutes.⁸ Treatment decisions are generally made in multi-disciplinary meetings, within the framework of the comprehensive cancer center. Trained registration clerks actively collect data on diagnosis, topography, histology, stage and information about initial treatment (delivered within 6 months after diagnosis) from hospital medical records. The medical record is generally regarded as the most complete source of information on a patient’s past and current health status.⁹

Since 1993 the Eindhoven Cancer Registry also registers the presence of serious comorbidity with prognostic impact at the time of cancer diagnosis, using a slightly modified version of the widely used Charlson comorbidity index.^4,10 Comorbidity was defined as any other disease that was present at the time of cancer diagnosis. Comorbidities were registered as dichotomous variables (yes/no), according to the medical history of the patient, the use of relevant drugs and diagnostic work-up. Cardiovascular disease and chronic obstructive pulmonary disease are diseases with significant influence on survival.^3,11 These were analyzed separately for their impact on prognosis. Cardiovascular disease included myocardial infarction, heart failure, angina pectoris, coronary artery bypass graft, peripheral arterial disease and cerebrovascular diseases.

Data from our regional cancer registry were handled according to the specifications of the officially recognized code of conduct on the use of data in health research.

All patients with stage III and IV MCL newly diagnosed between 1994 and 2006 were included (N=181). Selection was based on the World Health Organization (WHO) classification, documented from the medical records and registered in the cancer registry as ICD-O-3 morphology code 9673 and ICD-O-2 morphology code 9672, with the exclusion of tumors with localization in the stomach, bowel, lung, salivary glands, eye and skin. Patients with lymphoma diagnosed at autopsy were not selected. Additional data [Performance status according to WHO criteria, LDH level, hemoglobin level, albumin level, beta-2-microglobulin, Ki-67, chemotherapeutic regimen, platelets and WBC counts (lymphocytes, granulocytes and monocytes)] were collected from a new study of the medical records.

A prognostic index was calculated according to the original MIPI:¹ MIPI_{original/refitted} score = 0.03535*age (years) + 0.6978 (if ECOG >1) + 1.367*log10 LDH (ULN) + 0.9393*log10 WBC (10⁶).

This score classifies patients with a total score below 5.7 as low risk, patients with a score of 5.7 to 6.2 as intermediate risk and patients with a score equal to or higher than 6.2 as high risk.¹

Follow-up was completed up to January 1^st, 2008, with vital status obtained from the municipal personal records. Survival time was defined as the time from diagnosis until death or the end of the study.

Statistical analysis

Missing values may occur selectively across patients. Exclusion of patients with missing values might, therefore, bias the results. We, therefore, imputed missing covariates using correlations between variables. We used a multiple imputation procedure in which each missing value was imputed five times. Imputed values were drawn from the predictive distribution in an imputation model that included all risk factors (age, LDH, total leukocyte and lymphocyte, granulocyte, and platelet counts, performance status, number of comorbidities, cardiovascular disease, chronic obstructive pulmonary disease, sex, spleen involvement, B-symptoms, stage, albumin and hemoglobin) and the survival outcome. Imputation of missing predictor values using the outcome is preferred over imputation without outcome and is not self-fulfilling prophecy.¹² The variation among the five imputations reflects the uncertainty with which the missing values can be predicted. Multiple imputations resulted in five completed datasets, which were analyzed with standard statistical methods. The results were combined to produce overall estimates and standard errors that reflected missing data uncertainty.¹³ All analyses were performed for both complete cases as well as for single and multiple imputations. All results are reported with multiple imputed data, except for the Kaplan-Meier analyses, which were based on a single imputed data set. We checked the results of the randomly chosen single imputations, and these were comparable with those of the multiple imputation.

Validation of the MIPI started with a comparison of the hazard ratios of the risk factors (refitted MIPI). We checked whether the coefficients in the Cox regression equation needed to be updated, based on likelihood ratio statistics,¹⁴ and whether other cut-off points should be used for the categorical variables (revised MIPI). Next, we considered the extension of the model to include chronic obstructive pulmonary disease (yes/no), cardiovascular disease (yes/no), the number of comorbidities (yes/no or no/one/more than one), and a combination of these variables. We calculated the explained variation by the covariates as R² = 1 – exp (−LR/n), where LR is the likelihood ratio. Furthermore we evaluated whether sex, the presence of B-symptoms, stage, chemotherapeutic regimen, transplantation, hemoglobin level, beta-2-microglobulin level, albumin level and lymphocyte, granulocyte, monocyte and platelet counts, and a combination of these variables could further improve the MIPI (modified MIPI). We used a stepwise approach, to include the variable which improved the model most. Finally we tested the value of Ki-67 level, to validate the biological index of MIPI (MIPIb).

We used the c-statistic to study discrimination, which reflected the ability of the modified MIPI to assign higher predicted risks to subjects who died during the follow-up than to subjects who survived during the follow-up period. We used a bootstrap re-sampling procedure to correct for statistical optimism in the c-statistic for the refitted and modified models. Modeling was repeated in 200 bootstrap samples, with model testing in the original sample.¹⁵ Statistical analyses were performed using SAS software (version 9.1, SAS Institute Inc., Cary, North Carolina, USA, 1999), and R software (v 2.5.1, R Foundation for Statistical Computing, Vienna, Austria), with multiple imputation using the aregImpute function.

Results

Of the 181 patients with stage III or IV MCL, three were excluded, because sufficient information could not be obtained for them. Our population-based study included more patients with advanced age and more patients with a lower performance status, as compared to the original MIPI study (Table 1). As seen in the original study, the percentages of missing values were high for lymphocyte, granulocyte, and monocyte counts, and albumin and serum β₂-microglobulin levels. Furthermore it was difficult to gather data on functional status from the medical records in a quarter of the patients. Ki-67 was tested only sporadically and for 78% of the patients information about this test was missing. For those tumors for which Ki-67 testing was done, the test result in the medical record varied from an exact percentage to a description of the results (for example positive or high). We tried to divide these results into three categories: low or less than 10%, positive or between 10% and 29%, and high or 30% or above.

Table 1.

Patients’ baseline characteristics.

In our study population 126 patients died, with the 1-and 5-year survival rates being 80% and 34%, respectively, resulting in a lower median overall survival than in the original study (26 versus 57 months), with a median follow-up of the surviving patients of 47 versus 32 months, respectively.

For 60 patients the MIPI score could not be calculated because of missing values. The 118 MCL patients with complete data were categorized as low (31%), intermediate (19%) and high risk (51%), according to the MIPI score. With multiple imputations these proportions changed to 28%, 23%, and 49%, respectively. The 1- and 5-year overall survival rates in the low-risk group were 88% and 54%, those in the intermediate-risk group were 83% and 41%, while the high-risk group had the worst overall survival rates: 68% and 20%, respectively. The MIPI score discriminated our cohort nearly as well as the original cohort (c 0.65 versus 0.63).

While the discrimination of the whole model was comparable, the hazard ratio of performance status was somewhat higher, and the hazard ratios of LDH level and WBC count were a little lower than in the original study (Table 2). To correct for the underestimation of performance status in the original MIPI we considered using the ECOG score in five categories in the revised model.

Table 2.

Hazard ratios and 95% confidence intervals in the original, refitted, and modified MIPI models.

Furthermore, we investigated whether extending the model could improve the prognostic index. The R² of the refitted MIPI model was 23%, with a likelihood ratio of 47. A modified model which also included the presence of B symptoms and sex resulted in a substantially and significantly (P<0.001) better R² (49%) and likelihood ratio (121). Hemoglobin level, albumin level, number of comorbidities and the presence of cardiovascular disease were significant prognostic factors in univariate analyses (data not shown). However, at multivariate analysis the MIPI model could not be further improved by extension to include chronic obstructive pulmonary diseases, cardiovascular disease, number of comorbidities, stage, treatment, transplantation, hemoglobin level, albumin level, beta-2-microglobulin level, or lymphocyte, granulocyte, monocyte and platelet counts (data not shown).

The modified model can be calculated by: MIPI_modified score = 0.0453*age (years) + 0.5706*ECOG (subgroup) + 0.3854*log10 LDH (ULN) + 0.2035*log10 WBC (10⁶) + 0.7994 (if B symptoms present) + 0.7820 (if male patient). Three subgroups were defined with the cut-off points 4.65 and 5.90, leading to some loss in prognostic performance (R² 43% and likelihood ratio 100). Potential cut-points were assessed as in the original study in order to find the best discrimination between groups.¹ The low-risk group contained 64 (36%) patients, the intermediate-risk group 75 (42%) and the high-risk group 39 (22%) patients. The median survival times were 90, 29, and 6 months in the low, intermediate and high risk groups, as defined by the modified MIPI, compared to more than 90, 51, and 29 months, respectively, for groups defined by the original MIPI. This resulted in 1-and 5-year overall survival rates of 95% and 66% for the low-risk group, 81% and 24% for the intermediate-risk group and 38% and 0% for the high-risk group. The modified model provided a better discrimination of groups with different survival rates than that provided by the original MIPI (Figure 1).

Figure 1.

Kaplan-Meier curve of the refitted and modified MIPI. 1 = low-risk group, 2 = intermediate-risk group, 3 = high-risk group. Numbers of patients, per subgroup and time period, are presented at the bottom of the figure. Figure based on a single imputation of missing covariates.

As regards the biological index (MIPIb), we tried to collect data on the proliferation marker Ki-67, but this marker was not tested or poorly recorded in the medical records. For those cases with an available test on Ki-67 the positivity did not contribute to the MIPI when divided into four categories. No improvement in the MIPI was noted even with multiple imputations of the missing Ki-67 values.

Discussion

Prognostic models should be valid for daily clinical practice, allowing risk stratification and comparison of prognosis, thus providing a rationale for treatment decisions. Validation of a prognostic index in population-based settings is important because it shows whether the index is functional in daily practice. In our population-based setting the MIPI was valid, but could be significantly improved by a more refined coding of performance status and by including the presence of B symptoms and sex as risk factors.

The most likely reason for the higher proportion of patients with advanced age in our population-based cohort is that the original study was based on data from clinical trials, with restrictive selection criteria. For instance the European MCL trial¹⁶ had an age limit of up to 65 years. Furthermore all three trials^16–18 that formed the basis for the original study excluded patients with serious concomitant diseases, poor performance status, or significant impairment of organ function.¹ We found the prognostic value of age to be independent of the presence of co-morbidity and performance status and might, therefore, reflect unknown co-morbid or pathophysiological conditions more frequently encountered in elderly patients with subsequent less tolerance of treatment.

In several studies comorbidity was found to be an independent prognostic factor for survival in patients with non-Hodgkin’s lymphoma.^3,6 Although the presence of comorbidity in general, and cardiovascular disease in particular, were significant prognostic factors in univariate analyses, these factors did not improve the prognostic performance of the MIPI model. This is probably explained by the fact that performance status was included in the model, and this factor partly reflects the presence of comorbidity.¹⁹ The higher proportion of patients with a poorer performance status in our study could also be the reason for the relatively low impact of performance status in the original model containing only very few patients with a poor performance status.

It remains to be debated whether the poor prognosis associated with advanced age and poorer performance status should be a reason for a different, more aggressive treatment approach, because previous studies have shown that such patients experience more side effects of treatment.²⁰ This aspect should preferably be investigated prospectively. Furthermore, studying cause-specific survival may also help to unravel this issue, since part of the worse prognosis might also be due to mortality from the comorbidity itself.

The above mentioned factors could also be the explanation for the lower median survival in our study compared to that in the original study.¹ Of note, the survival rates in other population-based studies were similar to that in our study.^21,22

The incidence of MCL is known to be higher in males.^1,21,23 Although sex was not found to have a prognostic effect in either the original study or in other studies,^1,3,21,24 it did improve the MIPI significantly in our study. In diffuse large B-cell lymphoma the prognostic effect of sex was found after the introduction of rituximab treatment.²⁵ In our study only 4% of the patients were treated with rituximab, so we do not think that this treatment could explain the emergent prognostic effect of sex. Since we cannot provide a good explanation for this effect, it is important to validate the modified MIPI, with sex as a covariate, in other populations.

Another interesting observation in our study is that the presence of B-symptoms had an important prognostic effect and could improve the MIPI. This effect had also been reported in univariate analyses in some earlier studies,^1,26 but disappeared in multivariable analyses, in contrast to other studies which showed that it was an important independent risk factor also in multivariable analyses.^21,27

The prognostic effect of the biological marker Ki-67 could not be tested reliably in our study because of a very high percentage of missing values and no specific coding in the medical records. The percentage of missing values in the original study (48%) was also high. Since recent studies have shown that Ki-67 is an important prognostic factor in patients with MCL,^{24,26,28–30} it is important that the prognostic significance of Ki-67 in the MIPI model should be investigated.

The current analyses did not include patients with limited stage I or II MCL, because the MIPI was not designed for these patients. Hoster et al.¹ stated that the prognostic relevance of stage was not consistently seen in the literature. Moreover, the proportion of patients with stage I or II disease is rather low in MCL and patients with such disease require a different therapeutic approach. Thus Hoster et al. limited their investigation to patients with advanced stage MCL managed with standardized treatment options (CHOP, 56%; R-CHOP, 31%; and MCP, 11%).^16–18 The original data were also limited to patients who should have tolerated moderately intensive chemotherapy. In our study the proportion of unselected patients receiving moderately intensive treatment was obviously lower than in trial-based studies, such as those on which the original publication was based. More patients were treated with relatively mild regimens (43% with CHOP or CHOP-like regimens, 15% with R-CHOP, 4% with induction treatment for autologous stem cell transplantation, 19% with palliative chemotherapy and 18% with no chemotherapy). Treatment decisions are probably correlated with prognostic factors in our retrospective study. When we included treatment in our multivariate analyses, this variable gave no additional prognostic value over the other factors of the MIPI and we can, therefore, conclude that the other prognostic factors are more important for this population of patients.

Several candidate prognostic factors were included in the original study,¹ but some of these were excluded from the multiple regression analysis because of a high number of missing values. It is now widely recognized that complete case analyses with missing values in the data set can lead to bias of the results and are statistically inefficient.¹³ Nowadays, methods for handling missing data are becoming more standard and software is more readily available. Multiple imputation is considered a sound statistical methodology for handling complex missing data problems,^13,31 thereby contributing to statistically more reliable retrospective analyses, including ours.

In conclusion, the MIPI is a valid instrument for aiding risk stratification, comparison of prognosis, and treatment decisions in the setting of an unselected Dutch population of patients with MCL. Although the MIPI can be significantly improved, external validation on an independent data set is warranted before broad application of the modified index can be recommended.