Clustering of trend data using joinpoint regression models

Abstract

In this paper, we propose methods to cluster groups of two-dimensional data whose mean functions are piecewise linear into several clusters with common characteristics such as the same slopes. To fit segmented line regression models with common features for each possible cluster, we use a restricted least squares method. In implementing the restricted least squares method, we estimate the maximum number of segments in each cluster by using both the permutation test method and the Bayes Information Criterion (BIC) method, and then propose to use the BIC to determine the number of clusters. For a more effective implementation of the clustering algorithm, we propose a measure of the minimum distance worth detecting, and illustrate its use in two examples. We summarize simulation results to study properties of the proposed methods and also prove the consistency of the cluster grouping estimated with a given number of clusters. The presentation and examples in this paper focus on the segmented line regression model with the ordered values of the independent variable, which has been the model of interest in cancer trend analysis, but the proposed method can be applied to a general model with design points either ordered or un-ordered.

1 Introduction

Statistical similarity of objects has been studied in many different contexts by using various statistical procedures. A situation that received a considerable attention in the past decade is where each object is composed with a series of two-dimensional data points that can be described by a time-series model or a regression model, and clustering such objects has been of interest in many applications. An important issue in clustering time-series data has been to identify a good measure of (dis)similarity between the two or more time-series data sets, and a wide range of similarity measures were proposed in literature [1, 2, 3]. Using the similarity measure of choice, then, a classical clustering algorithm can be applied to a group of objects. Another approach proposed in literature is to use a model based clustering, more specifically to include a cluster membership variable in the model and estimate the cluster membership distribution using an EM algorithm or a Bayesian posterior distribution. For example, Qin and Self [4] proposed a model-based clustering method to cluster genes with a similar regression relationship to the covariate(s).

This paper is motivated from the question on how to identify and combine the neighboring age groups for which cancer incidence or mortality rates follow the same trend. For cancer trend analysis, Kim et al. [5] proposed a joinpoint regression model to describe trend changes, and it has been successfully applied in various trend analyses. The joinpoint regression model assumes that its regression mean function is piecewise linear and the segments are continuously connected at unknown change-points. For a non-linear model such as a polynomial model, the slope of the trend changes continuously and it makes the interpretation of when trend changes occur more difficult, compared to the joinpoint regression model. With an aim to cluster age groups with similar cancer incidence/mortaility trends together and to summarize cluster characteristics using joinpoint regression models, our focus in this paper would be on how to cluster joinpoint regression models, but the general idea of this paper can be applied to various other models with the independent variable ordered or un-ordered.

To describe cancer rate trend changes, Kim et al. [5] used a grid search to obtain a segmented line fit for a model with a given number of change-points and proposed a permutation test procedure to select the number of change-points. They implemented the procedure in Joinpoint software available at the National Cancer Institute website (http://surveillance.cancer.gov/joinpoint), and enhanced features such as a continuous fitting, sequential stopping rules, Bayes Information Criterion, etc. have been added since its first release in 1998. Kim et al. [6] extended this method to compare two groups of data that follow joinpoint regression models and discussed how to apply two group comparability tests to more than two group situations.

Our specific aim in this paper is to cluster groups of two-dimensional data points, whose mean functions are expressed as joinpoint regression mean functions, into several clusters sharing common characteristics such as the same slopes. We first consider the model with a fixed number of clusters and fit the model by using a restricted least squares method and determine the best grouping. Using similar arguments of Kim and Kim [7], we prove that the estimated grouping converges to the true grouping. In doing so, we present detailed arguments for the case where the objects are in order such as five-year age groups in cancer rate analysis and one wishes to cluster neighboring age groups together, and extend the result to a general situation with un-ordered objects. We then discuss how to select the number of clusters and how to apply the proposed method in practice.

This paper is organized as follows. In Section 2, we present the mathematical model and formulate the restricted least squares estimates. Section 3 includes the asymptotic results. In Sections 4 and 5, we present methods to select the number of clusters, summarize the simulation results to examine the performance of the proposed procedure, and propose a practical method to produce a more parsimonious grouping. Section 6 includes examples, and concluding remarks are given in Section 7.

2 Model and Fitting

Suppose that G objects come from M populations and that each object is presented as a multidimensional random vector that follows a segmented line regression model. The objects in each population are assumed to share some common characteristics, and our interest in this paper is on the cases where the objects in each population have the identical or parallel mean functions. That is, we observe n_g pairs of data points {(x_g,1, y_g,1), … , (x_g,ng, y_g,ng)} for g = 1, 2, … , G and assume that μ_g,i = E(y_g,i∣x_g,i) = β_g,0 + β_g,1x_g,i + δ_g,1(x_g,i − τ_g,1)⁺ + … + δ_g,κg (x_g,i − τ_g,κg)⁺ for i = 1, … , n_g, where a⁺ = max(0, a) and κ_g is the unknown number of change-points for the object g. We also assume that y_g,i = μ_g,i + ∊_g,i, where the ∊_g,i’s are independent with variance ${\sigma }_g^2$ and have a symmetric distribution. Since the G objects are from M populations, there exists an M-partition of {1, 2, … , G}, π = {π₁, π₂, … , π_M}, such that

1. π_m ⊂ {1, 2, … , G} for m = 1, … , M,

2. π_i ⋂ π_j = ∅ for i ≠ j, and

3. ${\cup }_{m=1}^M{\pi }_m=\{1,2,\dots ,G\}$.

In order to fit the model at a given M, consider the set of all possible partitions of {1, 2, … , G} and call it Ω = {π₁, π₂, … , π_L}, where each element of Ω partitions the G objects into M clusters and L is the number of all possible partitions. Then, we estimate the model parameters including π by minimizing the object function,

$$\sum _{m=1}^M\sum _{j\in {\pi }_m}{w}_{m,j}{\Vert {\mathit{y}}_{m,j}-{\mu }_{m,j}\Vert }^2,$$

where π = {π₁, … , π_M}, the w_m,j denote the weights such that Var(w_m,jy_m,j) = σ²I for all m and j, y_m,j and μ_m,j denote the observation and its mean vectors for the j-th object in the m-th cluster of the partition π, respectively, and ∥ · ∥ is the Euclidean norm. How to assign w_m,j will be discussed later. If our goal is to cluster G objects into M clusters where each cluster contains the objects whose mean functions are equal, then at each m = 1, … , M,

$$\left({\kappa }_g,{\tau }_{g,1},\dots ,{\tau }_{g,{\kappa }_g},{\beta }_{g,0},{\beta }_{g,1},{\delta }_{g,1},\dots ,{\delta }_{g,{\kappa }_g}\right)\equiv \left({\kappa }^{\left(m\right)},{\tau }_1^{\left(m\right)},\dots ,{\tau }_{{\kappa }^{\left(m\right)}}^{\left(m\right)},{\beta }_0^{\left(m\right)},{\beta }_1^{\left(m\right)},{\delta }_1^{\left(m\right)},\dots ,{\delta }_{{\kappa }^{\left(m\right)}}^{\left(m\right)}\right)$$

for all g ∈ π_m. If we are interested in clustering together the objects whose mean functions are parallel, then at each m = 1, … , M,

$$\left({\kappa }_g,{\tau }_{g,1},\dots ,{\tau }_{g,{\kappa }_g},{\beta }_{g,1},{\beta }_{g,1},{\delta }_{g,1},\dots ,{\delta }_{g,{\kappa }_g}\right)\equiv \left({\kappa }^{\left(m\right)},{\tau }_1^{\left(m\right)},\dots ,{\tau }_{{\kappa }^{\left(m\right)}}^{\left(m\right)},{\beta }_1^{\left(m\right)},{\delta }_1^{\left(m\right)},\dots ,{\delta }_{{\kappa }^{\left(m\right)}}^{\left(m\right)}\right)$$

for all g ∈ π_m. Figure 1 illustrates a situation with two clusters where two objects in each cluster share the same trend, that is, the parallel mean functions with the same change-point and slopes, which is usually of our interest in cancer trend analysis. The left panel includes the four objects where the objects denoted by ∘ and * are from one population with ${\kappa }^{\left(1\right)}=1,{\tau }_1^{\left(1\right)}=6,{\beta }_1^{\left(1\right)}=0.15$, and ${\delta }_1^{\left(1\right)}=-0.18$, and the two objects denoted by ∎ and • come from a different population with ${\kappa }^{\left(2\right)}=1,{\tau }_1^{\left(2\right)}=8,{\beta }_1^{\left(2\right)}=0.15$, and ${\delta }_1^{\left(2\right)}=-0.13$. The right panel shows the data generated with the same mean functions, but with a smaller value of σ_g, which shows a difference between the two clusters more clearly. Data sets in the left panel will be used later in our simulation study to assess the performance of the proposed method, and it will be observed that the proposed method performs very well for a data set with noise of this size.

Figure 1.

Example Data Sets

For the parametrization describe above, we now fit the model using a restricted least squares method. With ${\widehat{\mu }}_{m,j}\left({\pi }_l\right)$ denoting the mean vector fitted by the weighted least squares (WLS) method with the weight, w_m,j, at a partition π_l = {π_l,1, … , π_l,M}, we correspondingly estimate the cluster grouping as $\widehat{\pi }={({\widehat{\pi }}_1,\dots ,{\widehat{\pi }}_M)}^{\prime }$ such that

$$Q\left(\widehat{\pi }\right)=\underset{l=1,\dots ,L}{\min }\sum _{m=1}^M\sum _{j\in {\pi }_{l,m}}{w}_{m,j}{\Vert {\mathit{y}}_{m,j}-{\widehat{\mu }}_{m,j}\left({\pi }_l\right)\Vert }^2=\sum _{m=1}^M\sum _{j\in {\widehat{\pi }}_m}{w}_{m,j}{\Vert {\mathit{y}}_{m,j}-{\widehat{\mu }}_{m,j}\Vert }^2.$$

A situation that we are particulary interested in and will pursue first is the case where G objects are in order and clusters are to be formed over the neighboring objects. For example, in cancer trend studies, data are collected in different age groups: 0 − 4, 5 − 9, … , 85+, and the trends in nearby age groups tend to be more similar and can be grouped into one cluster. For such ordered partition, π = {{1, … , ρ₁}, {ρ₁ + 1, … , ρ₂}, … , {ρ_M−1 + 1, … , G}}, where ρ = (ρ₁, … , ρ_M−1)’ denotes the breaking points where the first M − 1 clusters end. Our aim is then to estimate the breaking point parameters, ρ = (ρ₁, … , ρ_M−1)’, as $\widehat{\rho }$ such that

$$Q\left(\widehat{\rho }\right)=Q({\widehat{\rho }}_1,{\widehat{\rho }}_2,\dots ,{\widehat{\rho }}_{M-1})=\underset{\mathit{d}\in \{\mathit{d}=(d_1,\dots ,d_{M-1}):d_1<d_2<\cdots <d_{M-1}\}}{\min }Q\left(\mathit{d}\right),$$

where the object function is

$$Q\left(\mathit{d}\right)=Q(d_1,d_2,\dots ,d_{M-1})=\sum _{m=1}^M\sum _{j=d_{m-1}+1}^{d_m}{w}_{m,j}{\Vert {\mathit{y}}_{m,j}-{\widehat{\mu }}_{m,j}\left(\mathit{d}\right)\Vert }^2$$

with d₀ = 0 and d_M = G. To estimate ρ, we note that Q(d) decreases as the number of change-points κ^(m) increases, so we find the restricted least squares estimates of κ^(m) as kmax^(m), where the kmax^(m) are predetermined constants such that κ^(m) ≤ kmax^(m). In the next section, we show that $\widehat{\rho }$ estimated under the condition that κ^(m) ≤ kmax^(m) or equivalently ${\widehat{\kappa }}^{\left(m\right)}={\mathit{kmax}}^{\left(m\right)}$ is consistent. We also discuss how to choose kmax^(m) in Section 4.

Remark 1

In practice, the weight w_g is often unknown, and we may use w_g such that $\frac{1}{w_g}={\Vert {\mathit{y}}_g-{\widehat{\mu }}_g\left({\mathit{kmax}}_g\right)\Vert }^2∕(n-q)$, where ${\widehat{\mu }}_g\left({\mathit{kmax}}_g\right)$ is the mean vector fitted with κ_g = kmax_g for an appropriately chosen kmax_g and q is the number of unknown free parameters. For the j-th object in the m-th cluster at a given partition π, we may also use $w_{m,j}={\lim }_{r\to \infty }{w}_{m,j}^{\left(r\right)}\left({\mathit{kmax}}^{\left(m\right)}\right)$, if the limit exists, such that for r = 1, 2, … ,

$$\frac{1}{w_{m,j}^{\left(r\right)}\left({\mathit{kmax}}^{\left(m\right)}\right)}={\Vert {\mathit{y}}_{m,j}-{\tilde{\mu }}_{m,j}^{(r-1)}\left({\mathit{kmax}}^{\left(m\right)}\right)\Vert }^2∕(n-q),$$

where ${\tilde{\mu }}_{m,j}^{\left(0\right)}\left({\mathit{kmax}}^{\left(m\right)}\right)$ denotes the mean vector estimated by the ordinary least squares method assuming κ^(m) = kmax^(m) and the ${\tilde{\mu }}_{m,j}^{\left(r\right)}\left({\mathit{kmax}}^{\left(m\right)}\right)$ are iteratively updated using $w_{m,j}^{\left(r\right)}\left({\mathit{kmax}}^{\left(m\right)}\right)$.

Remark 2

If $\mathrm{Var}({\mathit{y}}_1,\dots ,{\mathit{y}}_G)=\left(\begin{array}{ccc}\hfill {\sigma }_1^2{V}_1\hfill & \hfill \cdots \hfill & \hfill 0\hfill \\ \hfill ⋮\hfill & \hfill \ddots \hfill & \hfill ⋮\hfill \\ \hfill 0\hfill & \hfill \cdots \hfill & \hfill {\sigma }_G^2{V}_G\hfill \end{array}\right)$ where V_g ≠ I_ng×ng for g = 1, … , G, the weighted least squares fit can be obtained by using a weight matrix W which can be estimated following a similar iterative method described in Remark 1.

3 Asymptotics

Assume that there are M clusters with M given, and for m = 1, … , M, let θ^(m) and τ^(m) be the regression coefficients and the change-points for the m-th cluster, respectively. Recall that the partition, π = {π₁, … , π_M}, indicates which cluster the object g belongs to. We first present the results for the case where G-objects are in order, mainly for notational simplicity, and then will discuss its generalization to un-ordered situations. In the ordered situation where π_m = {ρ_m−1 + 1, ρ_m−1 + 2, … , ρ_m} with ρ₀ = 1 and ρ_M = G, we let ${\theta }^{\left(m\right)}={({\beta }_0^{\left(m\right)},{\beta }_1^{\left(m\right)},{\delta }_1^{\left(m\right)},\dots ,{\delta }_{{\kappa }^{\left(m\right)}}^{\left(m\right)})}^{\prime }$ for the model with the identical mean function in a cluster, ${\theta }^{\left(m\right)}={({\beta }_{{\rho }_{m-1}+1,0},\dots ,{\beta }_{{\rho }_m,0},{\beta }_1^{\left(m\right)},{\delta }_1^{\left(m\right)},\dots {\delta }_{{\kappa }^{\left(m\right)}}^{\left(m\right)})}^{\prime }$ for the objects in the m-th cluster, and ${\tau }^{\left(m\right)}={({\tau }_1^{\left(m\right)},{\tau }_2^{\left(m\right)},\dots ,{\tau }_{{\kappa }^{\left(m\right)}}^{\left(m\right)})}^{\prime }$ for the model with parallel mean functions.

Let

$$\theta ={({\theta }^{{\left(1\right)}^{\prime }},{\theta }^{{\left(2\right)}^{\prime }},\dots ,{\theta }^{{\left(M\right)}^{\prime }})}^{\prime },\tau ={({\tau }^{{\left(1\right)}^{\prime }},{\tau }^{{\left(2\right)}^{\prime }},\dots ,{\tau }^{{\left(M\right)}^{\prime }})}^{\prime },\text{and}\gamma ={({\theta }^{\prime },{\tau }^{\prime })}^{\prime }.$$

We also let κ = (κ⁽¹⁾, κ⁽²⁾, … , κ^(M))’ and express the parameter set as a parameter triple (ρ, κ, γ). Note that γ depends on ρ = (ρ₁, … , ρ_M−1) and κ.

To study asymptotic properties of the least squares estimators of ρ and γ, we assume the following conditions for the independent variable x. Note that τ⁽⁰⁾ denote the true change-points, and we also assume that the x’s are scaled such that min_g,i x_g,i = 0, max_g,i x_g,i = 1, without loss of generality, and the same number of data points for any object g, n_g = n.

Assumption 1

For each object g, the independent variable x_g has a positive and continuous density function in any small neighborhoods of 0, 1 and true change-points of all objects. Also x_g is independent of the error, ∊_g.

For the case of nonrandom design points, the following assumption can replace Assumption 1. Note that the data spacing should be of order O(1/n) to satisfy this assumption.

Assumption 1’

Let {h_n} be a sequence of constants for which O(1/n) ≤ h_n. For each g, the number of data points in any small neighborhoods with volume h_n of 0, 1 and true change-points of all objects is of order at least nh_n.

When the G-objects are in order, denote the mean function for the parallel model as

$${\mu }_g(\rho ,\kappa ,\gamma ,x)=\sum _{m=1}^M\left\{{\beta }_{g,0}+{\beta }_1^{\left(m\right)}x+{\delta }_1^{\left(m\right)}{\left(x-{\tau }_1^{\left(m\right)}\right)}^{+}\cdots +{\delta }_{{\kappa }^{\left(m\right)}}^{\left(m\right)}{\left(x-{\tau }_{{\kappa }^{\left(m\right)}}^{\left(m\right)}\right)}^{+}\right\}I\left({\rho }_{m-1}+1\le g\le {\rho }_m\right),$$

where ρ₀ = 0, ρ_M = G, τ_g,0 = 0, τ_g,κ^(m)+1 = 1, and I(A) = 1 if A is true and zero otherwise. The μ_g(ρ, κ, γ, x) for the equal mean function model are defined similarly.

Let

$$Q_n(\rho ,\kappa ,\gamma ,\mathit{w})=\frac{1}{\mathit{nG}}\sum _{g=1}^G{w}_g\sum _{i=1}^n{(y_{g,i}-{\mu }_g(\rho ,\kappa ,\gamma ,x_{g,i}\left)\right)}^2,$$

and consider

$$G_n(\rho ,\kappa ,\gamma ,\mathit{w})=Q_n(\rho ,\kappa ,\gamma ,\mathit{w})-Q_n({\rho }^{\left(0\right)},{\kappa }^{\left(0\right)},{\gamma }^{\left(0\right)},\mathit{w}),$$

where w = (w₁, … , w_G)’ and (ρ⁽⁰⁾, κ⁽⁰⁾, γ⁽⁰⁾) denote the true parameter vector. Let $\widehat{\rho }={\widehat{\rho }}_{\mathit{kmax}}$ minimize infγ Q_n(ρ, kmax, γ, w), where kmax = (kmax⁽¹⁾, kmax⁽²⁾, … , kmax^(M))’ are predetermined constants such that kmax^(m) ≥ κ^(m),(0) for each m = 1, … , M.

To establish the consistency of $\widehat{\rho }$, we first prove the following lemma.

Lemma

Under Assumption 1 (or Assumption 1’), for any δ > 0,

$$\underset{n\to \infty }{\lim }\underset{\rho \ne {\rho }^{\left(0\right)}}{\inf }\underset{\gamma }{\inf }{\mathit{EG}}_n(\rho ,\mathit{kmax},\gamma ,\mathit{w})>0.$$

Theorem 1

Under Assumption 1 (or Assumption 1’),

$$\underset{n\to \infty }{\lim }P(\widehat{\rho }={\rho }^{\left(0\right)})=1.$$

See Appendix for the proofs of Lemma and Theorem 1.

Remark 1

Lemma and Theorem 1 hold with estimated weights if E(w_g∊_g,i) = 0 and 1/w_g converges to ${\sigma }_g^2$ in probability. These two conditions are satisfied with w_g in Remark 1 of Section 1 such that $\frac{1}{w_g}={\Vert {\mathit{y}}_g-{\widehat{\mu }}_g\left({\mathit{kmax}}_g\right)\Vert }^2∕(n-q)$.

Remark 2

In the case of un-ordered objects, Theorem 1 can be re-written for $\widehat{\pi }$ such that ${\lim }_{n\to \infty }P(\widehat{\pi }={\pi }^{\left(0\right)})=1$.

Remark 3

Assumption 1’ holds in the equally spaced time point case, which is our interest with cancer rate analysis. Letting n get bigger in these asymptotic results means that observations are measured more frequently, and the large sample results will hold for reasonably frequently measured data points, which could be yearly measurements sometimes or monthly measurements in some other cases.

4 Selection of M and Simulations

In Sections 2 and 3, we used predetermined values of kmax^(m) and assumed that M is given, which is usually not known in practice. In this section, we propose a method to select M as well as discuss how to choose kmax^(m), and conduct simulations to compare accuracies of various selection procedures. First, we propose a data-driven method to find kmax^(m) following the suggestion made in Kim et al. [6]. In the ordered object case with (ρ₁, … , ρ_M−1) = (d₁, … , d_M−1), we use the maximum number of change-points for the m-th cluster estimated as

$${\mathit{kmax}}^{\left(m\right)}=\max \left({\widehat{\kappa }}_{d_{m-1}+1},{\widehat{\kappa }}_{d_{m-1}+2},\dots ,{\widehat{\kappa }}_{d_m},{\widehat{\kappa }}_m^{\left(0\right)}\right),$$

where ${\widehat{\kappa }}_g$ is the number of change-points selected for object g and ${\widehat{\kappa }}_m^{\left(0\right)}$ is the common number of change-points estimated for all of the d_m − d_m−1 objects in the m-th cluster.

To determine the number of change-points and the number of clusters, we considered both the hypothesis testing method based on permutation tests and the Bayes Information Criterion (BIC) selection method. The permutation procedure to select the number of change-points, κ, is described in Kim et al. [5] and its idea is summarized here. To estimate the number of change-points, κ, we start with testing the null hypothesis of H₀ : κ = k₀ against the alternative hypothesis of H₁ : κ = k₁(> k₀). When H₀ is rejected and k₁ > k₀ + 1, we test for κ = k₀ + 1 versus κ = k₁, while we proceed to the test of κ = k₀ versus κ = k₁ − 1 when H₀ is not rejected and k₁ − 1 > k₀. We repeat testing until we test the null hypothesis of H₀ : κ = k against the alternative of H₁ : κ = k + 1 for some k (k₀ ≤ k < k₁). Due to a difficulty to find an analytic distribution of the test statistic, Kim et al. [5] proposed to use the permutation distribution of the test statistic to estimate its p-value at each step, and the significance level at each step is adjusted to control the over-fitting probability to a conventional level of α. This procedure is directly applied to the clustering context to estimate the number of clusters M using the hypothesis testing method. For the BIC, we used the following functions and chose k and m to minimize BIC(k) and $\widetilde{\mathrm{BIC}}\left(m\right)$, respectively:

$$\begin{array}{cc}\hfill \mathrm{BIC}\left(k\right)& =\log \left(\frac{{\mathrm{RSS}}_k}{n}\right)+\frac{2k}{n}\log n,\hfill \\ \hfill \widetilde{\mathrm{BIC}}\left(m\right)& =\log \left(\frac{{\widetilde{\mathrm{RSS}}}_m}{N}\right)+\frac{p}{N}\log N,\hfill \end{array}$$

where RSS_k denotes the residual sum of squares for an object of size n fitted for a joinpoint model with k change-points, ${\widetilde{\mathrm{RSS}}}_m$ denotes the total sum of squared errors for G objects separated into m clusters, $N={\sum }_{g=1}^G{n}_g$, and p is defined as

$$p=(m-1)+\sum _{j=1}^n\{2k_j+1+(d_j-d_{j-1}\left)\right\}=(m-1)+2\sum _{j=1}^m{k}_j+m+G,$$

with $({\widehat{\rho }}_1,\dots ,{\widehat{\rho }}_{M-1})=(d_1,\dots ,d_{M-1})$, d₀ = 0 and d_m = G. We denote the values of k and m that minimize BIC(k) and $\widetilde{\mathrm{BIC}}\left(m\right)$ as $\widehat{\kappa }$ and $\widehat{M}$, respectively.

It has been found in Kim et al. [8] that the permutation test tends to be conservative compared to the BIC method in finding $\widehat{\kappa }$, and similar results are expected for the selection of kmax^(m) and the number of clusters, M. Although selecting a most parsimonious model is usually our goal in cancer trend analysis, the use of permutation tests both for finding the maximum number of change-points kmax^(m) and the number of clusters M (JPerm-CPerm method) is too time consuming and its performance was not very satisfactory in our preliminary simulation study. Combinations of model selection methods such as JPerm-CBIC, JBIC-CPerm, and JBIC-CBIC to find kmax^(m) and $\widehat{M}$ have been also considered in our preliminary simulation study, and we observed that the combinations of JPerm-CBIC and JBIC-CBIC perform a lot better with higher probabilities of correctly selecting M and correctly identifying the clusters than the JPerm-CPerm and the JBIC-CPerm combinations. This indicates that the permutation test to select the number of clusters may not have high power, although the method to select the number of change-points has a less significant effect on the overall performance, which could be explained by the proposed method to use kmax^(m). The JPerm-CBIC and JBIC-CBIC combinations produced comparable probabilities of correct selections in most cases, and we used both the JPerm-CBIC and JBIC-CBIC combinations for the detailed study summarized in the tables below.

We considered the log-linear model with normally distributed errors:

$$y_{g,i}=\log r_{g,i}={\mu }_{g,i}+{∊}_{g,i},$$

where the r’s denote the original rates and the ∊’s are independent and normally distributed with mean zero and variance ${\sigma }_g^2$ for g = 1, … , G, and i = 1, … , 30. In the simulation study, we estimated the probability of correctly selecting the number of clusters, M, and also the probability of correctly identifying clusters, {π₁, … , π_M} = {{1, … , ρ₁}, {ρ₁ + 1, … , ρ₂}, … , {ρ_M−1 + 1, … , G}}. When the permutation method was used to choose kmax^(m), we used the number of simulations, N_sim = 619, and the number of permutations, N_perm = 199, chosen following the suggestion of Boos and Zhang [9]. For the simulations with the JBIC-CBIC method, we used N_sim = 619.

Tables 1, 2, 3, and 4 summarize the simulation results for one, two, and three cluster situations, with various choices of σ_g and various cluster mean functions. Simulations are conducted to estimate P_M and P_C, the probabilities of correctly selecting the number of clusters, M, and of correctly identifying the clusters, respectively. In the tables, we considered the joinpoint regression mean functions motivated from actual cancer trends, Hodgkin’s lymphoma for Table 1, brain cancer for Tables 2 and 3, and prostate cancer for Table 4. In Table 1, we see that three choices of σ_g from 0.1 to 0.5 do not influence P_M and P_C, for the case of G = 6 and M = 1. However, σ_g does influence P_M and P_C when M = 2 as shown in Cases II-1, II-2, II-3 and II-4 of Table 2. Table 2 summarizes the simulation results for G = 6 and M = 2, and boldface numbers indicate simulation parameter values different from the reference parameter values in Case II-1. Table 3 summarizes simulation results when the first cluster mean function is a joinpoint regression model with one change-point, which is the only difference from Case II-2 in Table 2. The boldface numbers in Table 3 indicate simulation parameter values of the first cluster different from those of the second cluster in the reference case of Case II-2. The mean functions of the two clusters in Cases II-2-1, II-2-2, II-2-3, and II-2-4 allow changes both in the slope parameters and the location of the change-point, while the remaining cases in Table 3 considers the situations where two cluster mean functions are different only in the change-point or in the slope parameter. In Cases II-2-5 and II-2-6, two cluster mean functions share the same change-point, but the slopes after the change-point are different. Cases II-2-7 and II-2-8 consider the situation where two cluster mean functions share the same slope parameters, but the locations of the change-point are different. The simulation results indicate that the proposed method identifies clusters whose mean functions are different only in the change-point or only in the slope parameters as well, and its performance is reasonably good if the effect size is relatively large.

Table 1.

Case I with G = 6 and M = 1

Case	σ g	μg(x)	JPerm-CBIC		JBIC-CBIC
			PM	PC	PM	PC
		μ1(x) = 0.3 − 0.002x
		μ2(x) = 0.6 − 0.002x
I-1	σg = 0.10	μ3(x) = 1.0 − 0.002x	0.9693	0.9693	0.9515	0.9515
	(g = 1, … , 6)	μ4(x) = 1.3 − 0.002x
		μ5(x) = 1.6 − 0.002x
		μ6(x) = 2.0 − 0.002x
	σ1 = 0.30
	σ2 = 0.30
I-2	σ3 = 0.20	same as μg in Case I-1	0.9661	0.9661	0.9435	0.9435
	σ4 = 0.20
	σ5 = 0.10
	σ6 = 0.10
I-3	σg = 0.50	same as μg in Case I-1	0.9693	0.9693	0.9515	0.9515
	(g = 1, … , 6)

Table 2.

Case II with G = 6 and M = 2

Case	ρ 1	σ g	μg(x)	JPerm-CBIC		JBIC-CBIC		Δ
				PM	PC	PM	PC
			μ1(x) = 1.5 + 0.004x
			μ2(x) = 2.0 + 0.004x
II-1	3	σg = 0.10	μ3(x) = 2.5 + 0.004x	0.9871	0.9871	0.9725	0.9725	593.36
		(g = 1, … , 6)	μ4(x) = 1.5 + 0.15x − 0.13(x − 8)+
			μ5(x) = 2.0 + 0.15x − 0.13(x − 8)+
			μ6(x) = 2.5 + 0.15x − 0.13(x − 8)+
		σg = 0.50
II-2	3	(g = 1, … , 6)	same as μg in Case II-1	0.8691	0.8207	0.8756	0.8239	23.73
		σg = 0.30
		(g = 1, 2, 3)
II-3	3	σg = 0.80	same as μg in Case II-1	0.6801	0.5961	0.6931	0.6058	16.26
		(g = 4, 5, 6)
		σg = 1.0
II-4	3	(g = 1, … , 6)	same as μg in Case II-1	0.2068	0.1470	0.2520	0.1712	5.93
			μ1(x) = 1.5 + 0.004x
			μ2(x) = 2.0 + 0.004x
II-5	3	σg = 0.50	μ3(x) = 2.5 + 0.004x	0.9758	0.9548	0.9467	0.9289	38.80
		(g = 1, … , 6)	μ4(x) = 1.5 + 0.15x − 0.13(x − 10)+
			μ5(x) = 2.0 + 0.15x − 0.13(x − 10)+
			μ6(x) = 2.5 + 0.15x − 0.13(x − 10)+
			μ1(x) = 1.5 + 0.004x
			μ2(x) = 2.0 + 0.004x
II-6	3	σg = 0.50	μ3(x) = 2.5 + 0.004x	0.9855	0.9822	0.9693	0.9677	58.43
		(g = 1, … , 6)	μ4(x) = 1.5 + 0.15x − 0.13(x − 12)+
			μ5(x) = 2.0 + 0.15x − 0.13(x − 12)+
			μ6(x) = 2.5 + 0.15x − 0.13(x − 12)+
			μ1(x) = 1.5 + 0.004x
			μ2(x) = 2.0 + 0.004x
II-7	2	σi = 0.50	μ3(x) = 1.5 + 0.15x − 0.13(x − 8)+	0.8643	0.8029	0.8514	0.7835	21.10
		(i = 1, … , 6)	μ4(x) = 2.0 + 0.15x − 0.13(x − 8)+
			μ5(x) = 2.5 + 0.15x − 0.13(x − 8)+
			μ6(x) = 3.0 + 0.15x − 0.13(x − 8)+
			μ1(x) = 1.5 + 0.004x
			μ2(x) = 2.0 + 0.004x
II-8	3	σg = 0.10	μ3(x) = 2.5 + 0.004x	0.9871	0.9871	0.9725	0.9725	4211.56
		(g = 1, … , 6)	μ4(x) = 1.5 + 0.004x − 0.13(x − 8)+
			μ5(x) = 2.0 + 0.004x − 0.13(x − 8)+
			μ6(x) = 2.5 + 0.004x − 0.13(x − 8)+
			μ1(x) = 1.5 + 0.004x
			μ2(x) = 2.0 + 0.004x
II-9	3	σg = 0.50	μ3(x) = 2.5 + 0.004x	0.9871	0.9871	0.9693	0.9693	168.46
		(g = 1, … , 6)	μ4(x) = 1.5 + 0.004x − 0.13(x − 8)+
			μ5(x) = 2.0 + 0.004x − 0.13(x − 8)+
			μ6(x) = 2.5 + 0.004x − 0.13(x − 8)+
			μ1(x) = 1.5 + 0.004x
			μ2(x) = 2.0 + 0.004x
II-10	3	σg = 0.50	μ3(x) = 2.5 + 0.004x	0.9855	0.9855	0.9677	0.9677	141.96
		(g = 1, … , 6)	μ4(x) = 1.5 + 0.004x − 0.13(x − 10)+
			μ5(x) = 2.0 + 0.004x − 0.13(x − 10)+
			μ6(x) = 2.5 + 0.004x − 0.13(x − 10)+

Table 3.

Case II with G = 6 and M = 2

Case	ρ 1	σ g	μg(x)	JPerm-CBIC		JBIC-CBIC		Δ
				PM	PC	PM	PC
			μ1(x) = 1.5 + 0.004x
			μ2(x) = 2.0 + 0.004x
II-2	3	σg = 0.50	μ3(x) = 2.5 + 0.004x	0.8691	0.8207	0.8756	0.8239	23.73
		(g = 1, … , 6)	μ4(x) = 1.5 + 0.15x − 0.13(x − 8)+
			μ5(x) = 2.0 + 0.15x − 0.13(x − 8)+
			μ6(x) = 2.5 + 0.15x − 0.13(x − 8)+
			μ1(x) = 1.5 + 0.15x − 0.17(x − 6)+
			μ2(x) = 2.0 + 0.15x − 0.17(x − 6)+
II-2-1	3	σg = 0.50	μ3(x) = 2.5 + 0.15x − 0.17(x − 6)+	0.8514	0.7884	0.8465	0.8078	28.25
		(g = 1, … , 6)	μ4(x) = 1.5 + 0.15x − 0.13(x − 8)+
			μ5(x) = 2.0 + 0.15x − 0.13(x − 8)+
			μ6(x) = 2.5 + 0.15x − 0.13(x − 8)+
			μ1(x) = 1.5 + 0.15x − 0.17(x − 10)+
			μ2(x) = 2.0 + 0.15x − 0.17(x − 10)+
II-2-2	3	σg = 0.50	μ3(x) = 2.5 + 0.15x − 0.17(x − 10)+	0.1422	0.0969	0.1163	0.0921	8.38
		(g = 1, … , 6)	μ4(x) = 1.5 + 0.15x − 0.13(x − 8)+
			μ5(x) = 2.0 + 0.15x − 0.13(x − 8)+
			μ6(x) = 2.5 + 0.15x − 0.13(x − 8)+
			μ1(x) = 1.5 + 0.15x − 0.18(x − 6)+
			μ2(x) = 2.0 + 0.15x − 0.18(x − 6)+
II-2-3	3	σg = 0.50	μ3(x) = 2.5 + 0.15x − 0.18(x − 6)+	0.9354	0.9095	0.9402	0.9257	40.53
		(g = 1, … , 6)	μ4(x) = 1.5 + 0.15x − 0.13(x − 8)+
			μ5(x) = 2.0 + 0.15x − 0.13(x − 8)+
			μ6(x) = 2.5 + 0.15x − 0.13(x − 8)+
			μ1(x) = 1.5 + 0.15x − 0.18(x − 10)+
			μ2(x) = 2.0 + 0.15x − 0.18(x − 10)+
II-2-4	3	σg = 0.50	μ3(x) = 2.5 + 0.15x − 0.18(x − 10)+	0.3764	0.3021	0.3796	0.3263	14.09
		(g = 1, … , 6)	μ4(x) = 1.5 + 0.15x − 0.13(x − 8)+
			μ5(x) = 2.0 + 0.15x − 0.13(x − 8)+
			μ6(x) = 2.5 + 0.15x − 0.13(x − 8)+
			μ1(x) = 1.5 + 0.15x − 0.17(x − 8)+
			μ2(x) = 2.0 + 0.15x − 0.17(x − 8)+
II-2-5	3	σg = 0.50	μ3(x) = 2.5 + 0.15x − 0.17(x − 8)+	0.4766	0.4087	0.4782	0.4233	15.95
		(g = 1, … , 6)	μ4(x) = 1.5 + 0.15x − 0.13(x − 8)+
			μ5(x) = 2.0 + 0.15x − 0.13(x − 8)+
			μ6(x) = 2.5 + 0.15x − 0.13(x − 8)+
			μ1(x) = 1.5 + 0.15x − 0.18(x − 8)+
			μ2(x) = 2.0 + 0.15x − 0.18(x − 8)+
II-2-6	3	σg = 0.50	μ3(x) = 2.5 + 0.15x − 0.18(x − 8)+	0.7431	0.6931	0.7625	0.7173	24.92
		(g = 1, … , 6)	μ4(x) = 1.5 + 0.15x − 0.13(x − 8)+
			μ5(x) = 2.0 + 0.15x − 0.13(x − 8)+
			μ6(x) = 2.5 + 0.15x − 0.13(x − 8)+
			μ1(x) = 1.5 + 0.15x − 0.13(x − 6)+
			μ2(x) = 2.0 + 0.15x − 0.13(x − 6)+
II-2-7	3	σg = 0.50	μ3(x) = 2.5 + 0.15x − 0.13(x − 6)+	0.0565	0.0372	0.0614	0.0339	1.96
		(g = 1, … , 6)	μ4(x) = 1.5 + 0.15x − 0.13(x − 8)+
			μ5(x) = 2.0 + 0.15x − 0.13(x − 8)+
			μ6(x) = 2.5 + 0.15x − 0.13(x − 8)+
			μ1(x) = 1.5 + 0.15x − 0.13(x − 15)+
			μ2(x) = 2.0 + 0.15x − 0.13(x − 15)+
II-2-8	3	σg = 0.50	μ3(x) = 2.5 + 0.15x − 0.13(x − 15)+	0.8401	0.7835	0.8481	0.7964	28.94
		(g = 1, … , 6)	μ4(x) = 1.5 + 0.15x − 0.13(x − 8)+
			μ5(x) = 2.0 + 0.15x − 0.13(x − 8)+
			μ6(x) = 2.5 + 0.15x − 0.13(x − 8)+

Table 4.

Case III with G = 10 and M = 3

Case	ρ	σ g	μg(x)	JPerm-CBIC		JBIC-CBIC
				PM	PC	PM	PC
			μ1(x) = 1 + 0.03x + 0.12(x − 10)+ − 0.10(x − 20)+
			μ2(x) = 1.5 + 0.03x + 0.12(x − 10)+ − 0.10(x − 20)+
			μ3(x) = 2 + 0.03x + 0.12(x − 10)+ − 0.10(x − 20)+
			μ4(x) = 3 + 0.03x + 0.10(x − 10)+     +0.05(x − 14)+ − 0.14(x − 19)+
III-1	ρ1 = 3	σg = 0.05	μ5(x) = 3.5 + 0.03x + 0.10(x − 10)+     +0.05(x − 14)+ − 0.14(x − 19)+	0.8546	0.8401	0.7932	0.7819
	ρ2 = 6	(g = 1, … , 10)
			μ6(x) = 4 + 0.03x + 0.10(x − 10)+     +0.05(x − 14)+ − 0.14(x − 19)+
			μ7(x) = 4.5 + 0.02x + 0.10(x − 10)+      −0.02(x − 16)+ − 0.05(x − 20)+
			μ8(x) = 5 + 0.02x + 0.10(x − 10)+     −0.02(x − 16)+ − 0.05(x − 20)+
			μ9(x) = 5.5 + 0.02x + 0.10(x − 10)+     −0.02(x − 16)+ − 0.05(x − 20)+
			μ10(x) = 6 + 0.02x + 0.10(x − 10)+     −0.02(x − 16)+ − 0.05(x − 20)+

In Tables 2 and 3, we observe that P_M and P_C increase as the difference between the clusters gets big, and we note that P_M and P_C can be presented as monotonic functions of an effect size,

$$\Delta =\frac{{({\eta }^{\left(1\right)}-{\eta }^{\left(2\right)})}^{\prime }({\eta }^{\left(1\right)}-{\eta }^{\left(2\right)})}{\frac{1}{{\rho }_1^2}\sum _{g=1}^{{\rho }_1}{\sigma }_g^2+\frac{1}{{(G-{\rho }_1)}^2}\sum _{g={\rho }_1+1}^G{\sigma }_g^2},$$

where ${\eta }^{\left(m\right)}={\mu }^{\left(m\right)}-{\stackrel{‒}{\mu }}^{\left(m\right)}$ with ${\mu }^{\left(m\right)}={\left({\mu }_m\right(x_1),\dots ,{\mu }_m(x_n\left)\right)}^{\prime }$, the m-th cluster mean vector, and ${\stackrel{‒}{\mu }}^{\left(m\right)}=\frac{{\sum }_{i=1}^n{\mu }_m\left(x_i\right)}{n}{(1,\dots ,1)}^{\prime }$. We also note in Table 3 that P_M and P_C are relatively lower than those in Table 2 although they still increase as Δ increases. This seems to be because the true mean function for the first cluster in Table 3 has one change-point, which may not have been correctly identified by the permutation or BIC procedure, while the probability of correctly specifying no-change-point model for the first cluster in Table 2 could be relatively high. Table 4 reports the simulation results for a three cluster situation.

5 Minimum Difference Worth Detecting (MDWD)

When we applied the clustering method proposed in this article to real data, we observed that the method is quite powerful in detecting even small differences among the clusters. To make the clustering more meaningful for practical use, we introduce a measure to provide a threshold for distinguishing two clusters. We call this measure the minimum difference worth detecting (MDWD). The MDWD given below is developed mainly to cluster the models with same slopes together, and a similar measure can be used when one’s goal is to cluster the model with identical mean functions together. Specifically, the clustering is accomplished by the following procedure. For each pair of consecutive clusters, C₁ and C₂, we first remove the estimated change-points found in each cluster from the original data points if they coincide with original data x-values, and denote the remaining data points $\{{\tilde{x}}_1,\dots ,{\tilde{x}}_p\}$. That is,

$$\{{\tilde{x}}_1,\dots ,{\tilde{x}}_p\}=\{x_1,\dots ,x_n\}\backslash \{{\widehat{\tau }}_{1,1},\dots ,{\widehat{\tau }}_{1,k_1},{\widehat{\tau }}_{2,1},\dots ,{\widehat{\tau }}_{2,k_2}\},$$

where {x₁, … , x_n} are the data points and ${\widehat{\tau }}_{l,j}$ is the jth change-point estimated for the cluster C_l, l = 1, 2. We then find the estimated annual percent change (APC) for the segment where ${\tilde{x}}_j$ lies for each object, and denote them ${\widehat{\zeta }}_j^{\left(1\right)}$ and ${\widehat{\zeta }}_j^{\left(2\right)}$ respectively. Note that the estimated APCs are ${\widehat{\zeta }}_j^{\left(l\right)}=100(e^{{\widehat{\beta }}_{\left(l\right)}}-1)$, where ${\widehat{\beta }}_{\left(l\right)}$ is the estimated slope of the segment where ${\tilde{x}}_j$ lies for the l-th cluster. We now define the distance between the two clusters as

$$d(C_1,C_2)=\frac{\sum _{j=1}^p\mid {\widehat{\zeta }}_j^{\left(1\right)}-{\widehat{\zeta }}_j^{\left(2\right)\mid }\mid }{p}.$$

If the distance d(C₁, C₂) is smaller than the pre-determined MDWD, the two clusters C₁ and C₂ are deemed not meeting the MDWD criteria. More specifically, we adopt the following procedure that mixes clustering method and the MDWD criteria. For a given set of G objects,

• Step 1. Set a maximum number of clusters, M_max. For each m ∈ {1, … , M_max}, apply the proposed clustering method to determine the best grouping of m clusters. Then, use the BIC method to estimate the number of clusters, which is denoted by $\widehat{M}$.

• Step 2. Set the MDWD, say at φ, for example. For $m=\widehat{M}$, examine if the estimated adjacent clusters meet the MDWD criteria. That is, examine if d(C_j, C_j+1) ≥ φ for all j = 1, … , m − 1.

• Step 3. If m = 1 or all the neighboring pairs meet the MDWD criteria, then stop.

• Step 4. If there are at least one neighboring pairs of the estimated clusters not meeting the MDWD criteria, then reduce m by one to $\widehat{M}-1$. Repeat Steps 2-4.

The procedure requires all distances of neighboring pairs of suggested clusters bigger than the MDWD so that the suggested one forms more practical clusters. Clustering is conducted minimizing the restricted sum of squared errors by fitting an identical/parallel mean function model and using a permutation procedure or the BIC selection method, and thus the individual mean functions of the objects in one cluster do not differ using a criteria of statistical significance. The size of these differences may be small, moderate, or large depending on the statistical power to detect differences. However, in cases where there is substantial statistical power to detect even small differences, the MDWD criterion may override the statistical criteria when the differences are smaller than an ad hoc cutoff value set based on practical consideration. Note that the implementation of the MDWD seeks for an optimal grouping of the objects at each m, and thus there is a possibility that the objects combined into one cluster for the situation with m clusters may get separated into different clusters when Steps 2-3 are repeated for the situation with m−1 clusters after Step 4. By repeating Steps 2-3 after finding a difference that does not meet the MDWD criterion, we are assured that the resulting clusters represent an optimal grouping, albeit under a smaller m.

6 Example

To demonstrate the proposed methods, we analyze cancer incidence using 9 registries from the National Cancer Institute’s Surveillance Epidemiology and End Results (SEER) program representing 10% of the US population, and U.S. mortality data collected from the states by the National Center for Health Statistics. We will consider the US prostate cancer mortality rates and thyroid cancer incidence rates both from 1975-2009 for illustration of our methods.

Prostate cancer is the second most common cancer cause of death in the U.S. among men with an estimated 28,170 deaths in 2012 [10]. Figure 2 shows the usual age-adjusted trend for all combined. Mortality was rising slowly in the 1970’s until the late 1980’s when it started rising rapidly, leveled off briefly, and then has fallen since 1994. The decline in mortality is associated with advances in treatment, and possibly PSA screening for prostate cancer, which started in the early 1990’s (See Etzioni et al. [11]). The rise and fall in prostate cancer mortality around the time of the rapid introduction of PSA testing in the general population are consistent with a hypothesis that a fixed percent of the rising and falling pool of recently diagnosed patients who die of other causes may be mislabeled as dying of prostate cancer [12].

Figure 2.

Prostate Cancer Mortality

To determine if the trends are consistent across all ages we applied the clustering algorithm proposed. Since prostate cancer is a disease of older men and there are zero counts in some of the younger age groups, we collapsed 19 age groups 0 − 4, 5 − 9, … , 80 − 84, and 85+ into 11 age groups by combining ages 0 − 39 into one group. We then used JPerm-CBIC method, the permutation procedure to fit the joinpoint models and the BIC to select the number of clusters. Using the clustering procedure, the 11 age groups are grouped into 6 clusters: ages 0−54, 55−64, 65 − 74, 75 − 79, 80 − 84, and 85+. Figure 3 shows the fitted Joinpoint trends as well as the observed mortality rates in each age group. With so many prostate cancer deaths occurring each year there is sufficient statistical power to detect even small differences in trends. While the trends differ somewhat in each of the 6 clusters, it was felt that the differences between some of the clusters were small and would be difficult to interpret, so we applied a MDWD criteria.

Figure 3.

Prostate Cancer Mortality with 6 Clusters

We considered a few different MDWD’s. For MDWD = 0.5%, the age groups are clustered into the same six clusters as in MDWD = 0% , while with a MDWD of 1.0% and 1.5% there were two clusters (ages 0 − 84 and 85+), and with a MDWD of 2% all of the age groups were combined. Based on our knowledge of cancer trends and what is considered a “large difference,” we decided that a threshold of 1% was most reasonable, and the trends based on two clusters are shown in Figure 4. The trends in the two clusters are not parallel, as the age 0 − 84 group has a smaller APC until 1987. Since 1987, both clusters show a sudden increase, but the trend for 0 − 84 age group started turning downward since 1991 and decreased even further since 1994, while the downward trend in age 85+ group did not appear until 1993.

Figure 4.

Prostate Cancer Mortality with 2 Clusters at MDWD=1%

The second example is thyroid cancer incidence rates for white females. Thyroid cancer is the 5th most common cancer among women in 2012 [10] with an estimated 43,201 cases in the US. It is the fastest rising cancer in the U.S. although it is unclear whether thyroid cancer is really on the rise or whether doctors were just getting better at detecting smaller tumors that may have gone unnoticed in previous decades (See Davies and Welch [13]). Figure 5 shows the trend for the overall age-adjusted rates, which shows a rise of over 5% a year from 1993 to 1999 and over 7% a year from 1999 onward. We use the clustering methodology to determine if the rise differs across age groups. Since SEER 9 represents 10% of the U.S. population, there are just over 4,000 cases per year in the these registries (with considerably less statistical power to detect differences than for the 28,000 prostate cancer deaths). In the original 19 age groups, some of the younger age groups have zero counts, thus we lumped them into one age group. In particular, age groups 0 − 4, 5 − 9, and 10 − 14 are combined into one group 0 − 14, with 16 age groups remaining. We found that there are only 4 distinguishable clusters: 0 − 24, 25 − 29, 30 − 79, and 80+. The trends for each cluster are shown in Figure 6. In the three clusters 0 − 24, 25 − 29, and 80+, only one joinpoint is selected. For age group 0 − 24, the change-point is located at 2007, where a sudden dramatic increase in cancer incidence is observed. For age group 25 − 29, the incidence trend is stable or slightly decreasing until 1987 when a slow upward trend is shown. For age group 80+, the incidence rate is stable until 1997 when the incidence rate increases at 5% per year. The age group 30 − 79 shows an increasing trend from 1980. Because this is the age group where the bulk of the cases are, there is more statistical power to detect changes in trend. There are 5 change-points found in this age group. From 1980 to 1993 there was moderate increase of 2.76% per year, followed a large increase of over 5% per year from 1993 to 2000, a large increase of over 11% per year from 2000 to 2003, and an slightly smaller increase of almost 7% per year from 2003 onward. Running a MDWD for 0.5%, 1.0%, 1,5% and 2.0% still yielded 4 clusters. Although overall the trends across these four age clusters look somewhat similar, the locations of the change-points and the annual percent changes between the clusters are very different. Analysis of this type can help isolate potential causes of the rise in thyroid cancer.

Figure 5.

Thyroid Cancer Incidence for White Females

Figure 6.

Thyroid Cancer Incidence with 4 Clusters

In these two examples, the MDWD performed as expected. In the case of prostate cancer, small differences between clusters (which were statistically significant) were collapsed using the MDWD criteria. For female thyroid cancer, the smaller sample size means that only reasonably large differences can be detected, and thus the MDWD criteria did not yield any further collapsing of the age groups. For each area of application, what is an appropriate MDWD is a subjective determination, and it is gained through experience in analyzing the data. For cancer we feel that a reasonable MDWD is approximately 1%, although in specific analysis situations this may differ.

7 Discussion

In this paper, we proposed a method to cluster groups of observations that follow joinpoint regression models into several clusters that share similar characteristics. We implemented various methods to select the common number of change-points for each set of data and also to select the number of clusters, and suggested to use the MDWD for more practical clustering. For these model selection problems, we considered combinations of the permutation test procedure and BIC, and recommend the methods of JPerm-CBIC or JBIC-CBIC as a faster way to cluster, which use the permutation procedure or the BIC to select the number of change-points, respectively, and the BIC for the number of clusters. Although the presentation and examples of this paper focused on cancer trend analysis where the design points are the ordered time-points, the method works for situations with un-ordered design points, x, and the asymptotic results in Section 3 also consider such situations. In some situations, one may wish to cluster objects only according to the locations of change-points or based on the slope parameters regardless of the locations of change-points, and then the proposed method can be applied with a parametrization, different from those considered in Section 2 but incorporating the question of one’s interests. In cancer trend analysis, however, it would be rarely of our interests to consider the clustering problem only based on the change-points or on the slope parameter. Also, note that the idea of the proposed method can be applied to other models such as a general linear/non-linear regression model as long as the mean vector for the j-th object in the m-th cluster is estimated and thus the residual sum of squares can be calculated, but the asymptotic results presented in this paper may not work in general.

Another method proposed in Kim et al. [6] to compare multiple objects is to use multiple pairwise tests, where a pairwise comparability test is used for each pair of objects to test if the two joinpoint mean functions are parallel or identical and then the significance levels are adjusted to incorporate multiple comparisons conducted. In our preliminary simulation study, we conducted simulations to compare the exhaustive clustering proposed in this paper with the multiple pairwise comparison with a sequentially rejective Bonferroni procedure proposed by Holm [14] to improve its power. We observed that the exhaustive clustering of this paper has a higher probability of correctly classifying multiple objects into clusters than the multiple pairwise tests, and we expect a better performance of the exhaustive clustering in general since it incorporates all available data for clustering.

However, the exhaustive clustering proposed in this paper is time consuming, and its application may not be practical when G is large and/or one wants to cluster un-ordered objects. An ad-hoc approach is to avoid estimating the number of change-points, κ, and/or the locations of the change-points, τ, and this can be done either by using a polynomial regression model or fitting a segmented linear model with the change-points assumed to be at some pre-selected values. As indicated in Kim et al. [6], however, the over-specification or mis-specification of the model may lower the accuracy of the clustering procedure. Similarly, under-performance is expected when clustering is conducted by using a general distance/similarity measure ignoring the underlying model structure, as proposed in the context of mining time-series data. Another approach to replace the exhaustive clustering is to treat the cluster membership information as a missing information and use an EM algorithm with additional assumptions on the distribution as in Qin and Self [4], who proposed a method to cluster linear models and linear mixed models. In our future research, we plan to investigate if these alternative procedures could improve computational efficiency of the procedure proposed in this paper, without losing too much of accuracy and/or limiting the applicability of the proposed method to a class of specific distributions. Another future research problem is on large sample properties of $\widehat{M}$ and effects of the data-driven choice of kmax^(m) on the consistency of $\widehat{\pi }$ presented in Section 3. The use of a fixed kmax^(m) can be implemented with the possible maximum number of change-points at given n, although the use of such kmax^(m) may take impractically long time, and we also anticipate the BIC or a modified version of the BIC to provide a consistent estimator of M, and thus the consistency of $\widehat{\pi }$. In any case, these asymptotic results imply that when we have enough number of data points around the true change-points, the estimated grouping and estimated regression parameters would be close to the true grouping and true parameter values.

The clustering algorithm will be included in a future version of Joinpoint and interested readers are recommended to visit the Joinpoint website for updated information. (http://surveillance.cancer.gov/joinpoint).

Table 5.

Clustering results with different values of MDWD for U.S. Prostate Cancer Mortality from 1975 - 2009.

MDWD (φ)	0	0.5%	1%	1.5%	2%
${\widehat{M}}^{\phi }$	6	6	2	2	1
Age groups	0 − 54, 55 − 64, 65 − 74, 75 − 79, 80 − 84, 85+	0 − 54, 55 − 64, 65 − 74, 75 − 79, 80 − 84, 85+	0 − 84, 85+	0 − 84, 85+	0 − 85+

Table 6.

Clustering results with different values of MDWD for U.S. female thyroid cancer incidence from 1975 - 2009.

MDWD (φ)	0	0.5%	1%	1.5%	2%
${\widehat{M}}^{\phi }$	4	4	4	4	4
Age groups	0 − 24, 25 − 29, 30 − 79, 80+	0 − 24, 25 − 29, 30 − 79, 80+	0 − 24, 25 − 29, 30 − 79, 80+	0 − 24, 25 − 29, 30 − 79, 80+	0 − 24, 25 − 29, 30 − 79, 80+

APPENDIX.

Lemma: Sketch of Proof

Note that

$$\begin{array}{cc}\hfill G_n(\rho ,\kappa ,\gamma ,\mathit{w})=& \frac{1}{\mathit{nG}}\sum _{g=1}^G{w}_g\sum _{i=1}^n{\left({\mu }_g(\rho ,\kappa ,\gamma ,x_{g,i})-{\mu }_g({\rho }^{\left(0\right)},{\kappa }^{\left(0\right)},{\gamma }^{\left(0\right)},x_{g,i})\right)}^2\hfill \\ \hfill & -\frac{2}{\mathit{nG}}\sum _{g=1}^G{w}_g\sum _{i=1}^n{∊}_{g,i}\left({\mu }_g(\rho ,\kappa ,\gamma ,x_{g,i})-{\mu }_g({\rho }^{\left(0\right)},{\kappa }^{\left(0\right)},{\gamma }^{\left(0\right)},x_{g,i})\right).\hfill \end{array}$$

We sketch a proof for a simple situation with M = 2, G = 4, ρ⁽⁰⁾ = 2 and κ⁽⁰⁾ = (1, 1)′ and the case of nonrandom design data points. Let ρ = ρ⁽⁰⁾ + 1 = 3. We first note that E(∊_g,i) = 0 since ∊_i and −∊_i have the same distribution. Then, it is true that for any γ, EG_n(ρ, kmax, γ, w) is greater than or equal to

$$\frac{1}{\mathit{nG}}\sum _{g=2,3}{w}_g\sum _{i=1}^n{\left({\mu }_g(3,\mathit{kmax},\gamma ,x_{g,i})-{\mu }_g(2,{\kappa }^{\left(0\right)},{\gamma }^{\left(0\right)},x_{g,i})\right)}^2.$$ (1)

Let

$$\mu ({\beta }_0,{\beta }_1,\delta ,\tau ,\kappa ,x)={\beta }_0+{\beta }_{1}x+{\delta }_1{(x-{\tau }_1)}^{+}+\cdots +{\delta }_{\kappa }{(x-{\tau }_{\kappa })}^{+},$$

where δ = (δ₁, … , δ_κ)’ and τ = (τ₁, … , τ_κ)’. Then

$$\begin{array}{cc}\hfill {\mu }_2(2,{\kappa }^{\left(0\right)},{\gamma }^{\left(0\right)},x)=& \mu ({\beta }_{2,0}^{\left(0\right)},{\beta }_1^{\left(1\right),\left(0\right)},{\delta }_1^{\left(1\right),\left(0\right)},{\tau }_1^{\left(1\right),\left(0\right)},1,x),\hfill \\ \hfill {\mu }_3(2,{\kappa }^{\left(0\right)},{\gamma }^{\left(0\right)},x)=& \mu ({\beta }_{3,0}^{\left(0\right)},{\beta }_1^{\left(2\right),\left(0\right)},{\delta }_1^{\left(2\right),\left(0\right)},{\tau }_1^{\left(2\right),\left(0\right)},1,x).\hfill \end{array}$$

It is natural to assume that for any constant c there exist positive constants c₁ and c₂ such that

$$\mid {\mu }_2(2,{\kappa }^{\left(0\right)},{\gamma }^{\left(0\right)},x)-{\mu }_3(2,{\kappa }^{\left(0\right)},{\gamma }^{\left(0\right)},x)+c\mid >c_1$$

for x in a neighborhood with volume c₂ of 0, 1 or ${\tau }_1^{\left(j\right),\left(0\right)}$ for j = 1, 2.

Since for g = 2, 3,

$${\mu }_g(3,\mathit{kmax},\gamma ,x)=\mu ({\beta }_{g,0},{\beta }_1^{\left(1\right)},{\delta }^{\left(1\right)},{\tau }^{\left(1\right)},{\mathit{kmax}}^{\left(1\right)},x),$$

it is true that μ₂(3, kmax, γ, x) = μ₃(3, kmax, γ, x) + c₀ for c₀ = β_2,0 − β_3,0. Therefore, either it is true that

$$\mid {\mu }_2(3,\mathit{kmax},\gamma ,x)-{\mu }_2(2,{\kappa }^{\left(0\right)},{\gamma }^{\left(0\right)},x)\mid >\frac{c_1}{2}$$

for x in a neighborhood with volume c₂/2 of 0, 1 or ${\tau }_1^{\left(j\right),\left(0\right)}$ for j = 1, 2, or the following is correct:

$$\mid {\mu }_3(3,\mathit{kmax},\gamma ,x)-{\mu }_3(2,{\kappa }^{\left(0\right)},{\gamma }^{\left(0\right)},x)\mid >\frac{c_1}{2}$$

for x in a neighborhood with volume c₂/2 of 0, 1 or ${\tau }_1^{\left(j\right),\left(0\right)}$ for j = 1, 2.

By the Assumption 1′, it can be shown that for large n, (1) is greater than or equal to a positive constant which does not depend on γ.

Proof of Theorem 1

Since

$$G_n(\rho ,\mathit{kmax},\gamma ,\mathit{w})={\mathit{EG}}_n(\rho ,\mathit{kmax},\gamma ,\mathit{w})+\left(G_n\right(\rho ,\mathit{kmax},\gamma ,\mathit{w})-{\mathit{EG}}_n(\rho ,\mathit{kmax},\gamma ,\mathit{w}\left)\right),$$

we have

$$\begin{array}{cc}\hfill \underset{\mid \rho -{\rho }^{\left(0\right)}\mid >\delta }{\inf }\underset{\gamma }{\inf }{G}_n(\rho ,\mathit{kmax},\gamma ,\mathit{w})\ge & \underset{\mid \rho -{\rho }^{\left(0\right)}\mid >\delta }{\inf }\underset{\gamma }{\inf }{\mathit{EG}}_n(\rho ,\mathit{kmax},\gamma ,\mathit{w})\hfill \\ \hfill & -\underset{\mid \rho -{\rho }^{\left(0\right)}\mid >\delta }{\sup }\underset{\gamma }{\sup }\mid G_n(\rho ,\mathit{kmax},\gamma ,\mathit{w})-{\mathit{EG}}_n(\rho ,\mathit{kmax},\gamma ,\mathit{w})\mid .\hfill \end{array}$$

By Lemma, inf_{∣ρ− ρ⁽⁰⁾∣>δ}inf_γ EG_n(ρ, kmax, γ, w) > 0 for large n. And

$$\underset{\mid \rho -{\rho }^{\left(0\right)}\mid >\delta }{\sup }\underset{\gamma }{\sup }\mid G_n(\rho ,\mathit{kmax},\gamma ,\mathit{w})-{\mathit{EG}}_n(\rho ,\mathit{kmax},\gamma ,\mathit{w})\mid =o_p\left(1\right),$$

by Lemma 2.1 of Kim and Kim [7]. Thus, with probability approaching one,

$$\begin{array}{cc}\hfill \underset{\mid \rho -{\rho }^{\left(0\right)}\mid >\delta }{\inf }\underset{\gamma }{\inf }{G}_n(\rho ,\mathit{kmax},\gamma ,\mathit{w})>0=& G_n({\rho }^{\left(0\right)},{\kappa }^{\left(0\right)},{\gamma }^{\left(0\right)},\mathit{w})\hfill \\ \hfill \ge & \underset{\gamma }{\inf }{G}_n({\rho }^{\left(0\right)},{\kappa }^{\left(0\right)},\gamma ,\mathit{w})\hfill \\ \hfill \ge & \underset{\gamma }{\inf }{G}_n({\rho }^{\left(0\right)},\mathit{kmax},\gamma ,\mathit{w})\hfill \end{array}$$

as n → ∞. This completes the proof.