A robust latent CUSUM chart for monitoring customer attrition

Abstract

In competitive business, such as insurance and telecommunications, customers can easily replace one provider for another, which leads to customer attrition. Keeping customer attrition rate low is crucial for companies, since retaining a customer is more profitable than recruiting a new one. As a main statistical process control (SPC) method, the CUSUM scheme is able to detect small and persistent shifts in customer attrition. However, customer attrition summaries are typically available on an uneven time scale (e.g. 4-week and 5-week ‘business month’), which may not satisfy the assumptions of traditional CUSUM designs. This paper mainly develops a latent CUSUM chart based on an exponential model for monitoring ‘monthly’ customer attrition, under varying time scales. Both maximum likelihood and least squares methods are studied, where the former mostly performs better and the latter is advantageous for quite small shifts. We apply a Markov chain algorithm to obtain the average run length (ARL), make calibrations for different combinations of parameters, and present reference tables of cutoffs. Three more complicated models are considered to test the robustness of deviations from the initial model. Furthermore, a real example of monitoring monthly customer attrition from a Chinese insurance company is used to illustrate the scheme.

1. Introduction

Customer relationship management (CRM) has obtained increasing attention in modern business management. It increases a business's potential for profit by improving internal communication, making the selling process more effective and increasing customer satisfaction and loyalty. Recently, customer attrition has become an important issue for many organisations, especially in subscription based businesses, where customers have a formal, contractual relationship which must be ended, like telecommunications market and insurance industry [2,10]. Pettersson [16] showed that the cost of recruiting new customers is higher than the cost of retaining them, thus it is crucial for companies in these trades to monitor their customer population in order to keep churn rates low.

The customer attrition rate is a direct indicator of customer satisfaction and is important in forecasting the overall financial health of a company, as Szymanski and Churchill said [20]. It is also one of the key metrics for customer-based businesses, such as banks, insurance firms, telecommunications and internet service providers. Chen [3] pointed out that, if companies cannot take measures to retain customers before their status deteriorates, the customers may never come back, resulting in wasted investment and loss of future earnings. Therefore reducing customer attrition rate is of great interest for most companies in competitive markets. Generally, involuntary customer attrition such as that caused by death and relocation is inevitable, and, as a result, a company cannot completely eliminate attrition. But it is critical to detect unusual customer attrition beyond an acceptable rate.

Constructing a prediction mechanism to monitor customer attrition [5,8,12,13] and analyze causal factors [14] has become a popular topic in the past few years. Oghojafor et al. [14], using stepwise logistic regression, examined the effect of socio-economic factors on customer attrition by testing the factors that caused the subscribers to leave one service provider for another. The development of data mining techniques has enhanced the ability to predict customer attrition. Coussement and Poel [5] compared three classification techniques to distinguish churners from non-churners, and concluded that the Random Forest is effective in improving the predictive performance. He et al. [8] discussed the commercial bank customer attrition prediction based on an SVM model improved by a random sampling method, and showed that it effectively enhanced the prediction accuracy. In addition, Qian et al. [17] proposed using a functional mixture model to profile customer behavior in order to identify and capture attrition patterns. López-Díaz et al. [11] introduced a new stochastic ordering to compare some classifiers used in commercial banking to analyze customer attrition.

In general, researchers translate information on customers' behaviors into quantified variables, so that prediction models can be established. However, most analyses are based on static data and only make predictions on an individual level. It would be better to adopt a dynamic approach so that the trend in customer attrition can be monitored over time. Stoumbos et al. [19] stated that Statistical Process Control (SPC) techniques have been applied to dynamically monitor various industrial processes and later areas outside of manufacturing, for example, business failures prediction [21,22], health-care monitoring and public-health surveillance [26], 3D printing [27] and so on. Tran et al. [23] develops a variable sampling interval EWMA distribution-free control chart for monitoring service quality. Zhang et al. [28] proposes a distribution-free two-sided monitoring scheme to detect abnormal changes in time-between-review and sentiment scores for post-sales online review.

Although the Shewhart chart proposed by Shewhart [18] is still widely used today, the CUSUM (cumulative sum) chart, developed by Page [15], is better at detecting small but persistent shifts by accumulating information across a sequence of samples. Grigg et al. [6] extended the CUSUM chart and applied it to monitor the daily patient mortality rate in a medical setting. Chen and Zhou [4] proposed the cumulative sum (CUSUM) schemes to efficiently monitor the performance of typical queueing systems. The CUSUM control procedure is a graphical method that incorporates past information and performs well under small shifts. It is appropriate for the detection of changes in customer attrition since in practice, shifts in the customer attrition rate are usually small and persistent. There are several studies on customer attrition prediction based on control charts, see Jiang et al. [9]. Chen [3] proposed a novel mechanism based on the gamma CUSUM chart for online customer attrition prediction and showed good accuracy.

However in practice, customer summaries are generally measured on an uneven time scale, e.g. the four week and five week business month, which hampers the use of conventional CUSUM charts. In this paper, we propose a straightforward way to monitor customer attrition based on a latent design. We start with an oversimplified latent model which predicts a customer's risk of leaving in a business month and then build a latent CUSUM control chart that is able to monitor the monthly customer attrition under different time scales, no matter whether this is four weeks or five weeks. Different estimation methods' performances are evaluated and more complicated latent models are considered to check the robustness of the proposed CUSUM chart.

The paper proceeds as follows. In Section 2, we design a latent model to formulate customer attrition rate in a business month and build a corresponding latent Shewhart control chart. Both maximum likelihood estimation (MLE) and least squares estimation (LSE) methods are used to estimate customer attrition rate and their performances are compared. In Section 3, we design a latent CUSUM chart and apply a Markov chain algorithm to obtain the ARL (average run length) and evaluate the performance of the proposed chart. We also make calibrations for different combinations of the parameters and present a reference table of the cutoffs. In Section 4, we discuss robustness by applying the chart to three more complicated models. In Section 5, the latent charts are applied to real business data from a large Chinese insurance company and in the last section, we conclude the paper.

2. Latent model and chart

In this section, an oversimplified exponential model for customer attrition is built. The model assumes that a customer's risk to leave is a constant within a month, which implies that whether the customer leaves or not can be viewed as a Bernoulli trial. It is straightforward to monitor the number of lost customers within a business month. However, according to the business monthly calendar, a single month may be variously described as four weeks or five weeks. Conventional CUSUM control charts may perform poorly under an uneven time scale. Motivated by this, we propose a latent design to measure the probability of losing a customer in a fixed time period and then relate it to that in a ‘business month’. The need for such adjustment is common in the business world when monthly employee turnover, monthly profits and similar items are considered.

Two different methods are used to estimate customer attrition rate in this section. We evaluate their performance and provide our advice for selecting an estimation technique in different situations.

2.1. Latent model design

For each customer, we consider his or her time to departure from the beginning of each month as a latent continuous random variable, ranging from 0 to T weeks. Starting from an oversimplified model, denote $\{T=0,1,\dots \}$ as the time to departure and assume that it follows an exponential distribution. Meanwhile, the corresponding attrition rate is denoted as λ. It can be deduced that for a 4-week month, a customer's probability of attrition is

$$P_4={\int }_0^4\lambda {\mathrm{e}}^{-\lambda t}\mathrm{d}t=1-{\mathrm{e}}^{-4\lambda }.$$

Similarly, for a 5-week month,

$$P_5={\int }_0^5\lambda {\mathrm{e}}^{-\lambda t}\mathrm{d}t=-{\mathrm{e}}^{-\lambda t}{|}_0^5=1-{\mathrm{e}}^{-5\lambda }.$$

In general, the risk of attrition for a T-week month is

$$P_T=1-{\mathrm{e}}^{-T\lambda },$$ (1)

for T>0. Also the relationship between $P_4$ and $P_T$ can be described as

$$P_T=1-(1-P_4{)}^{\frac{T}{4}}.$$ (2)

It is shown that the attrition risk of a customer for any time period can be estimated when λ and T are known. T is determinative for each month, while λ should be estimated using historical data.

2.2. Parameter estimation

As stated before, each customer's behavior is assumed to independently follow a Bernoulli distribution. Denote $n_i$ as the number of individuals at risk (total number of customers) at the beginning of month i, and $x_i$ as the corresponding number who attrit. It is assumed that $x_i$ follows a Binomial distribution, i.e, $x_i\sim Bin(n_i,P_{T_i})$, where $p_{T_i}=1-{\mathrm{e}}^{-\lambda T_i}$ is the probability of customer attrition for month i, and $T_i$ is the length (in weeks) of month i. Usually, λ is unknown in practice and must be estimated from historical data. Maximum likelihood estimation (MLE), is the one of the most straightforward methods to estimate λ, while the least squares estimator (LSE) provides an alternative estimation method. MLE and LSE are compared later.

Firstly, it can be deduced that the latent likelihood function is

$$\begin{array}{rl}L(\lambda |x_1,\dots ,x_n)& =\prod _{i=1}^n(\genfrac{}{}{0ex}{}{n_i}{x_i})P_{T_i}^{x_i}(1-P_{T_i}{)}^{n_i-x_i}\\ & =\prod _{i=1}^n(\genfrac{}{}{0ex}{}{n_i}{x_i})(1-{\mathrm{e}}^{-\lambda T_i}{)}^{x_i}{\mathrm{e}}^{-\lambda T_i(n_i-x_i)},\end{array}$$

and the log likelihood function is

$$\begin{array}{rl}l(\lambda |x_1,\dots ,x_n)& =\sum _{i=1}^n\log (\genfrac{}{}{0ex}{}{n_i}{x_i})+\sum _{i=1}^n{x}_i\log (P_{T_i})+\sum _{i=1}^n(n_i-x_i)\log (1-P_{T_i})\\ & =\sum _{i=1}^n\log (\genfrac{}{}{0ex}{}{n_i}{x_i})+\sum _{i=1}^n{x}_i\log (1-{\mathrm{e}}^{-\lambda T_i})-\sum _{i=1}^n(n_i-x_i)\lambda T_i.\end{array}$$

Thus the MLE is

$${\hat{\lambda }}_{MLE}=\arg \underset{\lambda }{max}l(\lambda |x_1,\dots ,x_n).$$

Based on the Equation (1) as well as the invariance property of the MLE, we have ${\hat{P}}_{T,MLE}=1-{\mathrm{e}}^{{\hat{\lambda }}_{MLE}T}$.

The Shewhart control chart, whose points depend solely on the information of the corresponding subgroup sample, is sensitive to large process shifts. The chart can visually present transition probabilities through time and it would be more informative (in the sense that it is easier to extract information) if the center line is both meaningful and constant. We propose to plot V-statistics based on the sample attrition rate of each month, denoted as $f_i=\frac{x_i}{n_i}$ for month i. For a 4-week control chart, the V-statistic is ${\hat{V}}_i={\hat{f}}_i\frac{P_4}{P_{T_i}}$ with a constant mean $P_4$ and variance $\frac{(1-P_{Ti})P_4^2}{n_i{P}_{T_i}}$.

Therefore, a standard Shewhart chart can be formulated as:

$$\begin{array}{rl}& UCL=P_4+3\sqrt{Var({\hat{V}}_i)},\\ & CL=P_4,\\ & LCL=P_4-3\sqrt{Var({\hat{V}}_i)}.\end{array}$$ (3)

Based on the Shewhart chart, an iterative weighted LSE method is designed to estimate $P_4$, which can be deduced as a function of the attrition rate λ. In practice, the method converges quickly and can be carried out as follows:

Step 1: Choose an initial value for λ;

Step 2: Compute $(\widehat{P_4}{)}_i={\hat{x}}_{i,T}\frac{P_4}{P_T}$ and its variance;

Step 3: Take a weighted average of $(\widehat{P_4}{)}_i$ with weights inversely proportional to $Var((\widehat{P_4}{)}_i)$;

Step 4: Calculate λ from the weighted average of $(\widehat{P_4}{)}_i$;

Step 5: Repeat steps 1 to 4 until convergence and denote the result as ${\text{LSE}}_i$.

We carry out a simulation and compare the two estimation methods, that is LSE and MLE, using the ratio of their mean square errors (MSE):

$$r=\frac{{\mathrm{\Sigma }}_{i=1}^{n.sim}({\text{LSE}}_i-P_4{)}^2}{{\mathrm{\Sigma }}_{i=1}^{n.sim}({\text{MLE}}_i-P_4{)}^2},$$

where n.sim is the number of replicates. LSE performs better than MLE when $\gamma <1$; otherwise, MLE performs better. The simulation is designed to mimic customer attrition of an insurance company for 36 months (m = 36). Among the 36 months, 2/3 of them are simulated as 4-week months, which conforms to reality. We set n.sim = 1000 and replicate the calculation process 10000 times to obtain the mean value and the standard error of r. Table 1 illustrates the comparisons of MLE and LSE under different values of $P_4$ and n, where n represents the number of customers that the company monitors. It is shown that MLE performs better than LSE in most of the cases. However, LSE performs better when the number of customers is large and the attrition rate is low. Also, r is significantly different from 1 at the 0.05 level in most cases. Simulation results are also available for m = 60 with the same selection of $P_4$ and n. These results show a similar pattern to the 36-month results.

Table 1.

Comparison of MLE and LSE (36-month simulation results).

	n
$P_4$	1000	2000	5000	10000	20000	50000	80000	100000
0.001	${1.011}^{\ast }$	${1.017}^{\ast }$	${1.033}^{\ast }$	${0.971}^{\ast }$	${0.881}^{\ast }$	${0.757}^{\ast }$	${0.724}^{\ast }$	${0.712}^{\ast }$
	(5.40e-5)	(8.12e-5)	(1.15e-4)	(1.32e-4)	(1.40e-4)	(1.24e-4)	(9.61e-5)	(9.00e-5)
0.005	${1.006}^{\ast }$	${0.991}^{\ast }$	${0.935}^{\ast }$	${0.883}^{\ast }$	${0.836}^{\ast }$	${0.785}^{\ast }$	${0.780}^{\ast }$	${0.782}^{\ast }$
	(1.98e-5)	(2.90e-5)	(4.68e-5)	(4.86e-5)	(5.71e-5)	(9.18e-5)	(1.24e-4)	(1.46e-4)
0.01	${0.981}^{\ast }$	${0.979}^{\ast }$	${0.978}^{\ast }$	${0.971}^{\ast }$	${0.960}^{\ast }$	${0.940}^{\ast }$	${0.921}^{\ast }$	${0.912}^{\ast }$
	(1.23e-5)	(1.68e-5)	(2.98e-5)	(4.24e-5)	(5.01e-5)	(6.34e-5)	(7.21e-5)	(7.54e-5)
0.05	${1.004}^{\ast }$	${0.999}^{\ast }$	${0.996}^{\ast }$	${0.998}^{\ast }$	${1.000}^{\ast }$	${1.003}^{\ast }$	${1.004}^{\ast }$	${1.004}^{\ast }$
	(6.87e-6)	(1.00e-5)	(1.14e-5)	(1.23e-5)	(1.31e-5)	(1.61e-5)	(1.87e-5)	(2.09e-5)
0.1	${1.000}^{\ast }$	${1.000}^{\ast }$	${1.000}^{\ast }$	1.000	1.000	${1.000}^{\ast }$	${1.000}^{\ast }$	${1.000}^{\ast }$
	(4.71e-6)	(4.31e-6)	(4.17e-6)	(4.25e-6)	(4.26e-6)	(4.60e-6)	(4.70e-6)	(4.92e-6)
0.25	${1.001}^{\ast }$	${1.001}^{\ast }$	${1.001}^{\ast }$	${1.000}^{\ast }$	1.000	${0.999}^{\ast }$	${0.997}^{\ast }$	${0.997}^{\ast }$
	(1.25e-5)	(1.32e-5)	(1.52e-5)	(1.78e-5)	(2.21e-5)	(3.21e-5)	(3.91e-5)	(4.29e-5)
0.5	${1.007}^{\ast }$	${1.007}^{\ast }$	${1.007}^{\ast }$	${1.008}^{\ast }$	${1.008}^{\ast }$	${1.011}^{\ast }$	${1.012}^{\ast }$	${1.012}^{\ast }$
	(3.03e-5)	(3.01e-5)	(3.05e-5)	(3.07e-5)	(3.14e-5)	(3.35e-5)	(3.43e-5)	(3.50e-5)
0.75	${1.011}^{\ast }$	${1.011}^{\ast }$	${1.011}^{\ast }$	${1.011}^{\ast }$	${1.011}^{\ast }$	${1.010}^{\ast }$	${1.009}^{\ast }$	${1.009}^{\ast }$
	(6.68e-5)	(6.69e-5)	(6.75e-5)	(6.69e-5)	(6.84e-5)	(7.01e-5)	(7.37e-5)	(7.45e-5)
0.9	${1.039}^{\ast }$	${1.039}^{\ast }$	${1.039}^{\ast }$	${1.039}^{\ast }$	${1.038}^{\ast }$	${1.038}^{\ast }$	${1.038}^{\ast }$	${1.038}^{\ast }$
	(1.25e-4)	(1.26e-4)	(1.25e-4)	(1.27e-4)	(1.25e-4)	(1.27e-4)	(1.28e-4)	(1.28e-4)
0.95	${1.082}^{\ast }$	${1.077}^{\ast }$	${1.063}^{\ast }$	${1.054}^{\ast }$	${1.050}^{\ast }$	${1.046}^{\ast }$	${1.044}^{\ast }$	${1.043}^{\ast }$
	(1.78e-4)	(1.77e-4)	(1.76e-4)	(1.75e-4)	(1.77e-4)	(1.76e-4)	(1.76e-4)	(1.81e-4)
0.99	${1.199}^{\ast }$	${1.194}^{\ast }$	${1.192}^{\ast }$	${1.187}^{\ast }$	${1.177}^{\ast }$	${1.163}^{\ast }$	${1.144}^{\ast }$	${1.133}^{\ast }$
	(3.20e-4)	(3.25e-4)	(3.19e-4)	(3.23e-4)	(3.17e-4)	(3.18e-4)	(3.17e-4)	(3.09e-4)
0.999	${1.662}^{\ast }$	${1.695}^{\ast }$	${1.725}^{\ast }$	${1.688}^{\ast }$	${1.567}^{\ast }$	${1.132}^{\ast }$	${1.258}^{\ast }$	${1.239}^{\ast }$
	(6.61e-4)	(6.82e-4)	(7.08e-4)	(6.74e-4)	(6.18e-4)	(5.49e-4)	(5.14e-4)	(5.18e-4)

${}^a$ r=MSE(LSE)/MSE(MLE); if r<1, LSE is better, otherwise MLE is better.

${}^b$ standard errors are listed in brackets.

${}^c$* indicates that r is significantly different from 1 at a significance level of 0.05.

3. CUSUM latent chart

As for business, it is critical to detect small and persistent shifts in main financial variables. Generally, a Shewhart chart cannot detect small shifts well, while the CUSUM chart is expected to perform efficiently for such shifts.

The CUSUM chart is designed on the basis of Wald's Sequential Probability Ratio Test (SPRT) algorithm [25]. Instead of building a chart based on the current sample alone, Page [15] developed the CUSUM chart from a sequence of simple likelihood ratio tests to include information throughout the observation period in terms of the accumulated log-likelihood ratio. The SPRT basis of the CUSUM chart allows procedures for non-normal distributions.

3.1. CUSUM design

For a 4-week month, we set the hypothesis as:

$$\begin{array}{rl}{H}_0& :P_{4,0}=P_4,\\ H_A& :P_{4,A}=d_0{P}_{4,0},\end{array}$$

for a specified constant $d_0>1$. Similarly for a 5-month week, by formula (2),

$$\begin{array}{rl}{H}_0& :P_{5,0}=1-(1-P_{4,0}{)}^{5/4},\\ H_A& :P_{5,A}=1-(1-d_0{P}_{4,0}{)}^{5/4}.\end{array}$$

Without loss of generality, we denote the hypotheses for month i which has $T_i$ weeks as:

$$\begin{array}{rl}{H}_0& :P_{T_i,0}=1-(1-P_{4,0}{)}^{T_i/4},\\ H_A& :P_{T_i,A}=1-(1-d_0{P}_{T_i,0}{)}^{T_i/4}.\end{array}$$

The likelihood function based on the simple random sample $\{x_1,x_2,\dots ,x_{\nu }\}$ is:

$$L(P_1,P_2,\dots ,P_{\nu };x_1,x_2,\dots ,x_{\nu })=\prod _{i=1}^{\nu }f(x_i;P_i),$$

Therefore, the log-likelihood ratio is:

$$\begin{array}{rl}\log ({\mathrm{\Lambda }}_{\nu })& =\log \frac{L(P_{T_1,A},P_{T_2,A},\dots ,P_{T_{\nu },A};x_1,x_2,\dots ,x_{\nu })}{L(P_{T_1,0},P_{T_2,0},\dots ,P_{T_{\nu },0};x_1,x_2,\dots ,x_{\nu })}\\ & =\sum _{i=1}^{\nu }\log \frac{f(x_i;P_{T_i,A})}{f(x_i;P_{T_i,0})}.\end{array}$$

Based on the assumption that $x_i\sim Bin(n_i,P_{T_i})$, the log-likelihood ratio can be deduced to be:

$$\begin{array}{rl}\log ({\mathrm{\Lambda }}_{\nu })& =\sum _{i=1}^{\nu }\log \frac{(\genfrac{}{}{0ex}{}{n_i}{x_i})p_{T_i,A}^{x_i}(1-p_{T_i,A}{)}^{n_i-x_i}}{(\genfrac{}{}{0ex}{}{n_i}{x_i})p_{T_i,0}^{x_i}(1-p_{T_i,0}{)}^{n_i-x_i}}\\ & =\sum _{i=1}^{\nu }{x}_i\log \frac{P_{T_i,A}}{P_{T_i,0}}+(n_i-x_i)\log \frac{1-P_{T_i,A}}{1-P_{T_i,0}}\\ & \triangleq \sum _{i=1}^{\nu }{\mathrm{\Psi }}_i.\end{array}$$

Considering that $d_0>1$, in cases when $L(P_1,P_2,\dots ,P_{\nu };x_1,x_2,\dots ,x_{\nu })$ is a non-decreasing function, the SPRT rejects $H_0$ if $\log ({\mathrm{\Lambda }}_{\nu })\ge c$. Referring to Page [15], we monitor the sequence to ensure quick detection of a shift in the mean, especially from a long sequence of in-control observations.

$$\{\sum _{i=1}^{\nu }{\mathrm{\Psi }}_i,\sum _{i=2}^{\nu }{\mathrm{\Psi }}_i,\dots ,\sum _{i=k}^{\nu }{\mathrm{\Psi }}_i\}$$

The CUSUM monitoring statistic is defined as $C_{\nu }^{+}=\underset{1\le k\le \nu }{max}\sum _{i=k}^{\nu }{\mathrm{\Psi }}_i$. Derived by recursion, the statistic can be formulated as:

$$C_{\nu }^{+}=max\{C_{\nu -1}^{+}+{\mathrm{\Psi }}_{\nu },0\},$$

where $C_0^{+}=0$. The CUSUM chart is assumed to signal when $C_{\nu }^{+}>h$, where h>0 is a control limit that is chosen to reach a pre-specified ${\text{ARL}}_0$ value.

In this paper, we concentrate on monitoring upward shifts of customer attrition rate, thus the upward one-sided CUSUM chart is designed above. A downward one-sided CUSUM chart, which may fit circumstances such as the sales revenue of a business, can be built following the same procedure.

3.2. Average run length

An important step in designing a CUSUM chart is selection of the boundary, also called the control limit or cutoff, h. h is chosen based on the tolerable in-control ARL ( ${\text{ARL}}_0$), which measures the in-control performance of a CUSUM chart. Although not perfect, the ARL, the average length from the initial time point of consideration to the occurrence of an OC signal, is standard in quantifying the efficiency of a CUSUM chart. There are three common methods of determining the ARL for a CUSUM chart: the Monte Carlo method, the Markov chain algorithm, and the integral equations method.

In this paper, we apply the Markov chain algorithm, proposed by Brook and Evans [1], to obtain the ARL of the latent CUSUM chart. The Markov chain method approximates the CUSUM chart procedure with a discrete Markov Chain. For the continuous case, we discretize the range of $C_{\nu }^{+}$ into m states, plus state 0 and an absorbing state m + 1:

• State 0: $C_0=0$;

• State i: $(i-1)\mathrm{\Delta }<C_{\nu }\le i\mathrm{\Delta },\text{ for}i=1,2,\dots ,m,\text{ and}\mathrm{\Delta }=h/m$;

• State m + 1 : $C_{m+1}>h$.

Denote $\mathbf{P}$ as an $(m+2)\times (m+2)$ transition probability matrix with elements as follows:

$$p_{ij}=P[C_{\nu }\text{in state}j|C_{\nu -1}\text{in state}i],i,j=0,1,\dots ,m+1.$$

The matrix $\mathbf{P}$ can be expressed as

$$\mathbf{P}=\left[\begin{array}{cc}\mathbf{R}& \mathbf{p}\\ \mathbf{0}& 1\end{array}\right]=\left[\begin{array}{cc}\mathbf{R}& (\mathbf{I}-\mathbf{R})\mathbf{1}\\ \mathbf{0}& 1\end{array}\right],$$

where $\mathbf{1}$ is a vector of length m + 1 with all entries 1. Therefore, $\lambda$ can be found by solving the equation

$$(\mathbf{I}-\mathbf{R})\lambda =\mathbf{1}.$$

And the $i^{th}$ element of $\lambda (i=0,1,\dots ,m)$ gives the ARL for the CUSUM chart with state i as the initial state.

In this paper, the customer attrition, X, is assumed to follow a mixture distribution as follows:

$$X\sim w_{1}Bin(n,P_4)+w_{2}Bin(n,P_5),$$ (4)

where $w_1,w_2\ge 0$ and $w_1+w_2=1$. X degenerates to a binomial distribution when $w_1=1$ or $w_2=1$. According to the business month calendar, there are generally eight 4-week months and four 5-weeks in one natural year. Therefore, we set $w_1=2/3$ and $w_2=1/3$ in this paper.

The approach suggested by Hawkins[7] is used to approximate the transition probabilities, that is

$$\begin{array}{rl}{p}_{i,j}& =P[C_{\nu }\text{in state}j|C_{\nu -1}\text{in state}i]\\ & =P[(j-1)\mathrm{\Delta }<C_{\nu }\le j\mathrm{\Delta }|(i-1)\mathrm{\Delta }<C_{\nu -1}\le i\mathrm{\Delta }],\end{array}$$

where $i=0,\dots ,j=1,\dots ,m.$ Hawkins[7] explored multiple distributions and suggested use of the uniform distribution as a prior measure for $C_{\nu }$. Based on Hawkins approximation and Simpson's rule, the transition probability is

$$\begin{array}{rl}{p}_{i,j}& ={\int }_{(i-1)\mathrm{\Delta }}^{i\mathrm{\Delta }}\{F(j\mathrm{\Delta }-c)-F((j-1)\mathrm{\Delta }-c)\}d\mu (c)\\ & =[\{F((j-i)\mathrm{\Delta })+F((j-i+1)\mathrm{\Delta })+4F(j\mathrm{\Delta }-m_i)\}\\ & -\{F((j-1-i)\mathrm{\Delta })+F((j-i)\mathrm{\Delta })+4F((j-1)\mathrm{\Delta }-m_i)\}]/6,\end{array}$$

where F is the CDF (cumulative distribution function) of X and can be found by numerical approximation. And $m_i$ is the mid-point of state i, that is, $m_i=(2i-1)\mathrm{\Delta }/2$.

Following the approximation procedure, the transition probability can be calculated for different states as follows:

• for state 0,

  $$\begin{array}{rl}& p_{0,0}=F(0),\\ & p_{0,m+1}=1-F(h);\end{array}$$
• for $i=1,2,\dots ,m,$

  $$\begin{array}{rl}& p_{i,0}=\{F(-i\mathrm{\Delta })+F(-(i-1)\mathrm{\Delta })+4F(-m_i)\}/6,\\ & p_{i,m+1}=1-\{F(h-i\mathrm{\Delta })+F(h-(i-1)\mathrm{\Delta })+4F(h-m_i)\}/6;\end{array}$$
• for i = m + 1, the absorbing state,

  $$\begin{array}{rl}& p_{m+1,j}=0,j=0,1,\dots ,m,\\ & p_{m+1,m+1}=1.\end{array}$$

The ARL for a downward CUSUM chart ( ${\text{ARL}}^{-}$) can be established in a similar way as the upward chart ARL ( ${\text{ARL}}^{+}$). Bruyn [24] discussed the calculation of the ARL of the two-sided charts. Under the assumptions of symmetry and normality, the two-sided ARL can be obtained by

$$AR{L}_{\{twosided\}}={[\frac{1}{AR{L}^{+}}+\frac{1}{AR{L}^{-}}]}^{-1}.$$ (5)

In order to give a reference table of the cutoff h, different in-control ${\text{ARL}}_0$s, different magnitudes of shift and also, different assumptions of $P_4$ are considered. As shown in Table 4, we set ${\text{ARL}}_0$ equals to 60, 90, 120 and 200 respectively, to cover short-term and long-term situations for a business. Also we are interested in small probabilities of attrition thus we set $P_4$ equal to 0.01 and 0.12. Furthermore, n ranges from 1000 to 20000, representing different business sizes.

Table 4.

Cutoff h given $AR{L}_0$, n and $d_0$ (left: $P_4=0.01$; right: $P_4=0.012$, $w_1=2/3$ ).

	$d_0$ ( $P_4=0.01$)							$d_0$ ( $P_4=0.012$)
n=1000	1.025	1.05	1.1	1.15	1.2	1.25	1.3	1.025	1.05	1.1	1.15	1.2	1.25	1.3
$AR{L}_0$=60	0.477	0.849	1.386	1.744	1.981	2.168	2.295	0.519	0.919	1.471	1.842	2.078	2.237	2.357
$AR{L}_0$=90	0.592	1.038	1.659	2.058	2.333	2.527	2.656	0.643	1.113	1.755	2.156	2.423	2.603	2.720
$AR{L}_0$=120	0.686	1.185	1.862	2.289	2.583	2.795	2.922	0.740	1.267	1.965	2.391	2.672	2.864	2.987
$AR{L}_0$=200	0.874	1.480	2.250	2.726	3.030	3.253	3.403	0.945	1.575	2.366	2.827	3.135	3.344	3.474
	$d_0$ ( $P_4=0.01$)							$d_0$ ( $P_4=0.012$)
n=2000	1.025	1.05	1.1	1.15	1.2	1.25	1.3	1.025	1.05	1.1	1.15	1.2	1.25	1.3
$AR{L}_0$=60	0.646	1.105	1.705	2.060	2.266	2.398	2.486	0.698	1.183	1.793	2.142	2.341	2.462	2.542
$AR{L}_0$=90	0.794	1.337	2.007	2.403	2.631	2.762	2.842	0.855	1.424	2.110	2.489	2.711	2.835	2.884
$AR{L}_0$=120	0.914	1.514	2.247	2.653	2.899	3.064	3.113	0.983	1.608	2.343	2.744	2.974	3.092	3.161
$AR{L}_0$=200	1.151	1.858	2.670	3.114	3.374	3.531	3.609	1.257	1.980	2.789	3.214	3.453	3.587	3.665
	$d_0$ ( $P_4=0.01$)							$d_0$ ( $P_4=0.012$)
n=5000	1.025	1.05	1.1	1.15	1.2	1.25	1.3	1.025	1.05	1.1	1.15	1.2	1.25	1.3
$AR{L}_0$=60	0.934	1.505	2.125	2.407	2.518	2.496	2.508	0.999	1.590	2.193	2.447	2.565	2.518	2.442
$AR{L}_0$=90	1.136	1.790	2.470	2.779	2.891	2.929	2.870	1.215	1.886	2.554	2.832	2.935	2.889	2.767
$AR{L}_0$=120	1.294	2.005	2.718	3.040	3.181	3.174	3.135	1.383	2.108	2.810	3.098	3.194	3.197	3.122
$AR{L}_0$=200	1.606	2.411	3.190	3.535	3.658	3.637	3.665	1.711	2.526	3.284	3.583	3.708	3.732	3.654
	$d_0$ ( $P_4=0.01$)							$d_0$ ( $P_4=0.012$)
n=10000	1.025	1.05	1.1	1.15	1.2	1.25	1.3	1.025	1.05	1.1	1.15	1.2	1.25	1.3
$AR{L}_0$=60	1.206	1.830	2.400	2.549	2.464	2.275	1.934	1.285	1.915	2.428	2.578	2.468	2.160	1.536
$AR{L}_0$=90	1.450	2.150	2.731	2.924	2.900	2.725	2.460	1.539	2.244	2.808	2.930	2.838	2.612	2.069
$AR{L}_0$=120	1.637	2.386	3.010	3.205	3.190	3.135	2.822	1.737	2.484	3.085	3.209	3.207	2.962	2.601
$AR{L}_0$=200	2.001	2.827	3.495	3.690	3.735	3.598	3.354	2.112	2.935	3.571	3.725	3.655	3.517	3.132
	$d_0$ ( $P_4=0.01$)							$d_0$ ( $P_4=0.012$)
n=20000	1.025	1.05	1.1	1.15	1.2	1.25	1.3	1.025	1.05	1.1	1.15	1.2	1.25	1.3
$AR{L}_0$=60	1.512	2.594	2.606	2.449	2.054	1.334	0.343	1.600	2.224	2.552	2.374	1.796	0.779	0.001
$AR{L}_0$=90	1.797	3.421	2.998	2.874	2.507	1.872	0.874	1.894	2.573	2.940	2.818	2.350	1.379	0.147
$AR{L}_0$=120	2.013	4.402	4.847	3.260	2.873	2.236	1.402	2.117	2.831	3.221	3.155	2.677	1.832	0.674
$AR{L}_0$=200	2.419	3.210	4.847	3.822	3.425	2.912	1.999	2.534	3.304	3.712	3.648	3.274	2.511	1.473

3.3. Calibration

As mentioned above, X, the monthly customer attrition, follows a mixture of binomial distributions. $P_{mix}=\frac{x}{n}$, defined as the mixture attrition risk, is used to calibrate the magnitude of shifts corresponding to different $d_0$. Setting

$$\begin{array}{rl}\mu & =E[P_{mix}]=w_1{P}_4+w_2{P}_5,\\ {\sigma }^2& =Var[P_{mix}]=w_1\frac{P_4(1-P_4)}{n}+w_2\frac{P_5(1-P_5)}{n}+w_1{w}_2(P_4-P_5{)}^2.\end{array}$$ (6)

We use the formula

$$\delta =\frac{P_{mix,A}-\mu }{\sigma },$$

to make calibrations for different combinations $(P_4,d_0,n,w_1)$, where $P_{mix,A}=w_1{P}_{4,A}+w_2{P}_{5,A}$. Tables 2 and 3 present the values of $d_0$ for given δ and n when $P_4=0.01,0.012$ respectively, and still $w_1=2/3$.

It is straightforward to see the relationship between $d_0$ and δ from the tables. For example, when n = 10000, $w_1=2/3$ and $P_4=0.01$, the magnitude of the shift $(1\sigma ,2\sigma ,3\sigma )$ corresponds to $d_0=(1.144,1.289,1.433)$.To be specific, when n = 10000, the expected cutoff to monitor a one sigma shift would be $d_0=1.15$. In this way, we may build a CUSUM chart under the assumption that the tolerance attrition rate increases 15 percent when $(n,P_4,w_1)=(10000,0.01,2/3)$. δ values that are not shown in the table can be calculated using linear interpolation when $d_0$ is given. Table 3 shows similar patterns to Table 2, which means $d_0$ decreases with n and increases with d. Furthermore, $d_0$ in Table 3 is smaller than the corresponding $d_0$ in Table 2 since $P_4$ is slightly larger.

Table 3.

$d_0$ given δ and n ( $P_4=0.012$, $w_1=2/3$).

	n
	1000	2000	5000	10000	20000	50000	100000
$\delta =0.25$	1.074	1.056	1.041	1.035	1.031	1.029	1.028
$\delta =0.5$	1.148	1.111	1.082	1.069	1.062	1.057	1.056
$\delta =0.75$	1.222	1.167	1.123	1.104	1.093	1.086	1.084
$\delta =1$	1.296	1.223	1.164	1.139	1.124	1.115	1.112
$\delta =1.25$	1.370	1.279	1.205	1.174	1.156	1.144	1.139
$\delta =1.5$	1.444	1.334	1.246	1.208	1.187	1.172	1.167
$\delta =1.75$	1.518	1.390	1.287	1.243	1.218	1.201	1.195
$\delta =2$	1.593	1.446	1.328	1.278	1.249	1.230	1.223
$\delta =2.5$	1.741	1.558	1.410	1.347	1.311	1.287	1.279
$\delta =3$	1.889	1.669	1.492	1.417	1.373	1.345	1.335

Table 2.

$d_0$ given δ and n ( $P_4=0.01$, $w_1=2/3$).

	n
	1000	2000	5000	10000	20000	50000	100000
$\delta =0.25$	1.080	1.060	1.043	1.036	1.032	1.029	1.028
$\delta =0.5$	1.161	1.120	1.087	1.072	1.064	1.058	1.056
$\delta =0.75$	1.241	1.180	1.130	1.108	1.096	1.087	1.084
$\delta =1$	1.321	1.240	1.173	1.144	1.128	1.116	1.112
$\delta =1.25$	1.402	1.300	1.217	1.181	1.160	1.145	1.140
$\delta =1.5$	1.482	1.360	1.260	1.217	1.191	1.175	1.169
$\delta =1.75$	1.562	1.419	1.303	1.253	1.223	1.204	1.197
$\delta =2$	1.643	1.479	1.346	1.289	1.255	1.233	1.225
$\delta =2.5$	1.803	1.599	1.433	1.361	1.319	1.291	1.281
$\delta =3$	1.964	1.719	1.520	1.433	1.383	1.349	1.337

The cutoff h corresponding to different $(AR{L}_0,n,d_0,P_4)$ is shown in Table 4. For instance, when n = 5000 and $d_0=1.2$, the cutoffs for $AR{L}_0=(60,90,120,200)$ are (2.518,2.891,3.181,3.658) and (2.565,2.935,3.194,3.708) corresponding to $P_4=0.01,0.012$, respectively. Above all, Table 4 gives a reference table of the cutoff h when other parameters are roughly determined so that latent CUSUM modelers can directly select certain h for use.

ARLs for different magnitudes of shifts after calibration can be compared, under the same combinations $\{AR{L}_0,n,P_4\}$. As shown in Table 5 and Table 6, situations of a medium size business that refers to 10000 customers with a moderate $AR{L}_0$ of 120 when $P_4=0.01$ and $P_4=0.012$ are displayed respectively. The column names represents the values of $d_0$ and h used to construct the CUSUM chart, while the row names provide different shift alternatives, described in terms of $(\sigma ,d)$. Results under other circumstances are available from the authors upon request.

Table 5.

ARL given $d_0,h,\delta$ and n (n = 10000, $P_4=0.01$, $w_1=2/3$).

	$(d_0,h)$
( $\delta ,d$)	(1.025,1.64)	(1.05,2.39)	(1.1,3.01)	(1.15,3.21)	(1.2,3.19)
(0, 1)	119.642	119.930	119.991	119.929	121.353
(0.25, 1.036)	22.382	22.450	25.611	30.069	34.751
(0.50, 1.072)	10.823	9.820	9.840	11.100	12.980
(0.75, 1.108)	7.114	6.146	5.554	5.730	6.343
(1, 1.144)	5.327	4.492	3.837	3.695	3.846
(1.25, 1.181)	4.263	3.546	2.935	2.702	2.682
(1.50, 1.217)	3.594	2.967	2.411	2.159	2.080
(1.75, 1.253)	3.125	2.571	2.062	1.814	1.717
(2, 1.289)	2.776	2.287	1.811	1.576	1.479
(2.5,1.361)	2.297	1.913	1.461	1.272	1.201
(3, 1.433)	2.013	1.645	1.229	1.109	1.071

Table 6.

ARL given $d_0,h,\delta$ and n (n = 10000, $P_4=0.012$, $w_1=2/3$).

	$(d_0,h)$
( $\delta ,d$)	(1.025,1.64)	(1.05,2.39)	(1.1,3.01)	(1.15,3.21)	(1.2,3.19)
(0, 1)	120.056	120.05	119.596	119.988	119.717
(0.25, 1.035)	20.882	21.005	24.462	29.146	34.099
(0.50, 1.069)	10.133	9.160	9.300	10.683	12.772
(0.75, 1.104)	6.609	5.666	5.123	5.359	6.050
(1, 1.139)	4.937	4.124	3.497	3.403	3.586
(1.25, 1.174)	3.969	3.271	2.677	2.487	2.487
(1.50, 1.208)	3.401	2.787	2.236	2.021	1.960
(1.75, 1.243)	2.955	2.419	1.908	1.691	1.609
(2, 1.278)	2.624	2.157	1.672	1.466	1.385
(2.5,1.347)	2.158	1.778	1.322	1.178	1.128
(3, 1.417)	1.913	1.503	1.130	1.057	1.036

Table 5 shows that when $P_4=0.01$, for a shift of $0.25\sigma$, $d_0=1.025$ is the optimized assumption in CUSUM design. For a shift from $0.5\sigma$ to $0.75\sigma$, $d_0=1.1$ is the best choice and $d_0=1.15$ is optimal for a shift of $1\sigma$ or $1.5\sigma$, which may exceed the tolerance level of customer attrition rate for a business. Though it gives the optimal result numerically, we do not recommend using it. It should also be mentioned that the ARLs of $d_0=1.025$, $d_0=1.05$ and $d_0=1.1$ for a $0.25\sigma$ shift are very close to each other, so are the ARLs of $d_0=1.05$ and $d_0=1.1$ for a $0.5\sigma$ shift and the ARLs of $d_0=1.1$ and $d_0=1.15$ for a $1\sigma$ shift. Therefore, we recommend using $d_0=1.1$ in practice when n = 10000.

The patterns when $P_4=0.012$ in Table 6 are quite similar. It is shown that for a shift of $0.25\sigma$, $d_0=1.025$ is the optimal value in CUSUM design. For a shift of $0.5\sigma$, $d_0=1.05$ is the best choice and for a shift of $0.75\sigma$, $d_0=1.1$ is optimal. When monitoring a shift around $1\sigma$ to $1.25\sigma$, we should choose $d_0=1.15$, and $d_0=1.2$ is the best when the shift is larger than $1.25\sigma$. It is suggested to set $d_0=1.1$ in practice when n = 10000.

4. Robustness of the chart

In Section 2, the charts are designed on basis of an oversimplified exponential model. However in practice, models used to formulate customer behaviors would be much more complicated. In this section, we study the robustness of the latent CUSUM chart from the simple exponential model. Three more complicated cases are considered when modeling customer attrition and the performance of our proposed latent CUSUM chart is evaluated as a whole.

4.1. Mixture exponential model

We start with a mixture exponential model, that is

$$T\sim (1-ϵ)Exp({\lambda }_1)+ϵExp({\lambda }_2),$$ (7)

where $ϵ\in [0,1)$ is a measurement of the degree of contamination, and $Exp(\cdot )$ denotes an exponential distribution. The parameters $ϵ,{\lambda }_1,{\lambda }_2$ are determined to preserve the whole attrition rate per month, keeping $w_1{P}_4+w_2{P}_5$ as a constant. Based on the assumption of the mixture exponential model, $P_4$ and the according $P_5$ are respectively as follows

$$\begin{array}{rl}{P}_4& =(1-ϵ)(1-{\mathrm{e}}^{-4{\lambda }_1})+ϵ(1-{\mathrm{e}}^{-4{\lambda }_2}),\\ P_5& =(1-ϵ)(1-{\mathrm{e}}^{-5{\lambda }_1})+ϵ(1-{\mathrm{e}}^{-5{\lambda }_2}).\end{array}$$

When $ϵ=0$, the mixture model reduces to the original exponential model.

The hypothesis of the model is set to be

$$H_0:P_{4,0}=P_4,H_A:P_{4,A}=d_0{P}_{4,0}$$

And the ARLs of the Mixture Exponential model under different parameter settings are displayed in Table 7. Here two different situations of $({\lambda }_1,{\lambda }_2)$ are considered, that is ${\lambda }_1/{\lambda }_2=0.5$ and ${\lambda }_1/{\lambda }_2=4$. And the contamination parameter is set as $ϵ=0,0.01,0.05,0.1,0.25$ and 0.5. It is demonstrated that the model is robust since ARLs are quite stable across different settings of ϵ.

Table 7.

ARL of Mixture Exponential model. $h=3.01,P_4=0.01,d_0=1.10,w_1=2/3,n=10000$.

	$ϵ({\lambda }_1/{\lambda }_2=0.5)$						$ϵ({\lambda }_1/{\lambda }_2=4)$
	0	0.01	0.05	0.1	0.25	0.5	0	0.01	0.05	0.1	0.25	0.5
d = 1	120.0	120.0	120.0	120.0	119.9	119.9	120.0	120.1	120.0	120.0	119.9	122.6
$d=1.03$	31.69	31.69	31.68	31.67	31.65	31.65	31.69	31.70	31.69	31.68	31.65	32.10
$d=1.06$	12.82	12.82	12.82	12.81	12.81	12.81	12.82	12.82	12.82	12.81	12.81	12.92
$d=1.09$	7.14	7.14	7.14	7.14	7.14	7.14	7.14	7.14	7.14	7.14	7.14	7.19
$d=1.12$	4.83	4.83	4.83	4.83	4.83	4.83	4.83	4.83	4.83	4.83	4.83	4.85
$d=1.15$	3.65	3.65	3.65	3.65	3.65	3.65	3.65	3.65	3.65	3.65	3.65	3.66
$d=1.18$	2.95	2.95	2.95	2.95	2.95	2.95	2.95	2.95	2.95	2.95	2.95	2.96
d = 1.21	2.50	2.50	2.49	2.49	2.49	2.49	2.50	2.50	2.50	2.49	2.49	2.50
$d=1.25$	2.09	2.09	2.09	2.09	2.09	2.09	2.09	2.09	2.09	2.09	2.09	2.09
$d=1.3$	1.75	1.75	1.75	1.75	1.75	1.75	1.75	1.75	1.75	1.75	1.75	1.75
$d=1.35$	1.51	1.51	1.51	1.51	1.50	1.50	1.51	1.51	1.51	1.51	1.50	1.50
$d=1.4$	1.32	1.32	1.32	1.32	1.32	1.32	1.32	1.32	1.32	1.32	1.32	1.32
d = 1.45	1.19	1.19	1.19	1.19	1.19	1.19	1.19	1.19	1.19	1.19	1.19	1.19
$d=1.5$	1.10	1.10	1.10	1.10	1.10	1.10	1.10	1.10	1.10	1.10	1.10	1.10

4.2. Weibull model

The second model considered in this section is a Weibull model. Let $Y\sim Exp(\lambda )$ and $T=Y^{1/\beta }$, then

$$T\sim Weibull(\lambda ,\beta ).$$ (8)

The basic exponential model is a special case when $\beta =1$. Retaining the settings of Table 7, that is $P_4=0.01$, $d_0=1.10$, $w_1=2/3$, and n = 10000, ARLs are calculated under a variety of values for β.

As shown in Table 8, the ARL of the Weibull model does not fluctuate much for a sizable range of β. For instance, with d = 1.21, the ARL rises from 2.36 to 2.66 when β ranges from 0.2 to 2. Differences are noticeable for $\beta =4$ but are not so large. Though slightly different in ARLs, it can be concluded that the model is effective when the data follows a Weibull distribution.

Table 8.

ARL of Weibull model. $h=2.458,P_4=0.01,d_0=1.10,w_1=2/3,n=10000$.

	β
	0.2	0.5	0.75	1	2	4
d = 1	116.09	117.36	118.30	120.04	123.27	135.72
d = 1.03	30.40	30.84	31.24	31.69	33.50	38.64
d = 1.06	12.20	12.41	12.61	12.82	13.72	16.24
d = 1.09	6.77	6.90	7.01	7.14	7.67	9.17
d = 1.12	4.56	4.66	4.73	4.83	5.18	6.21
d = 1.15	3.44	3.51	3.57	3.65	3.91	4.68
d = 1.18	2.79	2.84	2.89	2.95	3.15	3.76
d = 1.21	2.36	2.40	2.44	2.50	2.66	3.16
d = 1.25	1.97	2.00	2.04	2.09	2.22	2.63
d = 1.3	1.65	1.67	1.70	1.75	1.86	2.19
d = 1.35	1.42	1.44	1.47	1.51	1.61	1.90
d = 1.4	1.26	1.27	1.29	1.32	1.42	1.68
d = 1.45	1.14	1.15	1.17	1.19	1.27	1.51
d = 1.5	1.07	1.07	1.08	1.10	1.16	1.36

4.3. Gamma model

A gamma model, denoting the failure rate as λ and the shape parameter as γ, is also considered in this paper.

$$T\sim Gamma(\lambda ,\gamma ).$$ (9)

The model reduces to the simple exponential model when $\gamma =1$. We set $\gamma =0.5,0.75,1.2,2.5,4$ and compute the ARLs under the gamma model, as shown in Table 9. The gamma model seems to be robust since the ARL fluctuates sightly as γ changes.

Table 9.

ARL of Gamma model. $h=2.458,P_4=0.01,d_0=1.10,w_1=2/3,n=10000$.

	γ
	0.2	0.5	0.75	1	2	4
d = 1	116.73	117.69	120.03	122.6	126.59	131.77
d = 1.03	30.77	31.22	31.69	33.27	34.36	36.85
d = 1.06	12.40	12.62	12.82	13.62	14.10	15.35
d = 1.09	6.89	7.02	7.14	7.61	7.90	8.64
d = 1.12	4.65	4.74	4.83	5.14	5.34	5.85
d = 1.15	3.51	3.58	3.65	3.88	4.03	4.41
d = 1.18	2.83	2.89	2.95	3.13	3.26	3.55
d = 1.21	2.39	2.44	2.50	2.64	2.75	2.99
d = 1.25	2.00	2.03	2.09	2.20	2.29	2.49
d = 1.30	1.67	1.70	1.75	1.83	1.92	2.08
d = 1.35	1.44	1.46	1.51	1.58	1.66	1.80
d = 1.40	1.27	1.29	1.32	1.39	1.46	1.59
d = 1.45	1.15	1.16	1.19	1.24	1.3	1.42
d = 1.50	1.07	1.08	1.10	1.14	1.18	1.28

The results in Table 7, Table 8 and Table 9, suggest that the CUSUM chart we have constructed is robust to a variety of departures from the latent model on which it is based and that it can perform well in practice.

5. A real example

Customer attrition rate is one the of key business measurements in CRM systems, especially for service industries like banks, telephone service companies, internet service providers and insurance firms. In this section, the proposed latent CUSUM control chart is applied to a customer attrition dataset from a large Chinese insurance company.

5.1. Data description

Endowment Insurance is a type of life insurance that is payable to the insured if he/she is still living on the policy's maturity date, or to a beneficiary otherwise. The dataset includes over 100000 insurance policies signed since 2005 and it is large and thus representative enough to serve as a population for both practice and research. In this paper, we focus on a specific time period of 36 months, from Jan. 2011 to Dec. 2013, during which the monthly sample size is sufficiently large.

Assuming that customer summaries are available on an uneven time scale (i.e. 4 and 5 week ‘business months’). We first standardize the way of presenting the number of weeks for each month, under the direction of ISO 8601¹. It is a date and time format published by the International Organization for Standardization. The criterion is that January 4th always lies in the first week of the year. Equivalently, the $4^{th}$ day of a month indicates the certain first week, which therefore leads to ‘4-week month’ and ‘5-week month’. Take the month of January 2011 as an example. Since 4th January 2011 and 4th February 2011 are Tuesday and Friday respectively, the standardized period of January is from 3rd January to 30th January, which means January in 2011 is a ‘4-week month’. In this way, the whole set of observation dates is divided into two groups consisting of 24 ‘4-week months’ and 12 ‘5-week months’.

When considering Endowment Insurance, it should be mentioned that customer attrition occurs due to either voluntary or involuntary causes. Voluntary attrition occurs due to a customer's decision not to renew an insurance policy, while involuntary attrition may occur because of death or relocation. In this paper, we concentrate on voluntary attrition and define the customer attrition rate of month i:

$$p_i=x_i/n_i,$$ (10)

where $n_i$ is the number of individuals in state at the start of month i, and $x_i$ is the number of transitions out of state during the month.

We randomly select 10000 samples at the start of each month, that is $\{n_i=10000,i=1,2,\dots ,36\}$. Then we count the voluntary attrition samples $x_i,i=1,2,\dots ,36$, and compute the monthly customer attrition rates $p_i,i=1,2,\dots ,36$. As shown in Figure 1, customer attrition rates are low during the first six months of 2011 and gradually increased since late 2011. The two colors stand for ‘4-week month’ and ‘5-week month’, respectively.

Figure 1.

Monthly customer attrition rate.

5.2. Latent CUSUM control chart

The first six monthly records, including four ‘4-week months’ and two ‘5-week months’, are selected to be Phase I (in-control stage). $P_4$ is estimated by means of the MLE procedure and the result is ${\hat{P}}_{4,0}=0.0118$. Referring to Section 3, $d_0=1.1$ and h = 3.085 is recommended in practice when n = 10000 and $P_4=0.012$. Therefore we set the hypothesis for a 4-week month as follows:

$$H_0:P_{4,0}=0.012,H_A:P_{4,A}=d_0{P}_{4,0}$$

where $d_0=1.1$. Similarly for a 5-week month, under $H_0$, $P_{5,0}=1-(1-P_{4,0}{)}^{5/4}$, and under $H_A$, $P_{5,A}=1-(1-d_0{P}_{4,0}{)}^{5/4}$.

The log-likelihood ratio for each month is computed and the latent CUSUM control chart is derived by recursion, with $C_0^{+}=0$ and

$$C_{\nu }^{+}=max\{C_{\nu -1}^{+}+{\mathrm{\Lambda }}_{\nu },0\},\nu =1,2,\dots ,36.$$ (11)

The chart signals when $C_{\nu }^{+}>3.085$.

As shown in Figure 2, the proposed latent CUSUM control chart alerts at February 2012, the ${14}^{th}$ observation. Since the first six monthly records are assumed to be in Phase I and the cumulative statistic is exactly increasing from the second half year of 2011, we conclude that the proposed latent CUSUM performs well.

Figure 2.

Latent CUSUM control chart.

5.3. Latent Shewhart control chart

As for a reference, we also build a latent Shewhart control chart for the insurance dataset. For the standard Shewhart chart, the monitoring statistic is ${\hat{V}}_i={\hat{f}}_i{P}_4/P_{T_i}$, where $f_i$ represents the monthly customer attriton rates in Section 5.1, $P_4=0.0118$ and $P_{T_i}$ follows Equation (2). The control limits are calculated by Equation (3), and then the Shewhart latent chart can be plotted.

As shown in Figure 3, the dashed lines represent the LCL and UCL, respectively. It can be seen that several points lie outside the control limits, which means the chart alerts at the corresponding time points. To be specific, the chart shows 8 signals from year 2012 to the first quarter of 2013 while the first signal appears at the ${15}^{th}$ observation, that is March 2012.

Figure 3.

Latent Shewhart control chart.

Different from the CUSUM chart, the Shewhart type is generally used to detect large shifts. In this example, the latent CUSUM scheme alerts slightly earlier than the Shewhart one. It is expected that in practice, small shifts are much more common in the business world and thus we recommend using the latent CUSUM chart.

6. Conclusion

Customer relationship management (CRM) has gained increasing attention in the modern business field since the 1990s. Maintaining old customers, or in other words, reducing customer attrition, can effectively improve an enterprises profit and thus is regarded of great importance in CRM. The CUSUM chart provides a dynamic procedure to monitor customer attrition. It is available to monitor small and persistent shifts. However in practice, customer summaries are generally measured on an uneven time scale, hampering the usage of conventional CUSUM charts.

This paper proposes a latent CUSUM scheme to detect customer attrition which is summarized as a ‘monthly’ metric. Adapting to different quantities of weeks (that is, 4 weeks or 5 weeks each month), an oversimplified exponential model is developed and the parameters are estimated via both maximum likelihood estimation and the least squares method. It is shown that MLE performs better than LSE in most of the cases, while LSE is advantageous to detect a very small shift. In this paper, we make calibrations for different combinations of parameters, where various magnitudes of shifts are considered. Then we propose a latent CUSUM chart and apply a Markov chain algorithm to present reference tables of the cutoff, under different basic monthly attrition rates. It is recommended to set the shift magnitude parameter $d_0=1.1$ when n = 10000, based on the results.

Also the robustness of our latent CUSUM chart is considered for three more complicated cases, namely a mixture exponential model, a Weibull model and a gamma model. The study shows that the proposed latent CUSUM chart performs well under uneven time scales, and that it is robust to departures from the original assumption of the oversimplified exponential distribution, which is quite inspiring.

A real-world database of an insurance product in China is illustrated to show the superiority of the proposed latent CUSUM scheme. It is shown to perform well when monitoring the ‘monthly’ customer attrition.

Funding Statement

Prof. Wu is supported by the National Natural Science Foundation of China ( 11871324) and the National Bureau of Statistics of China (No. 2020LD03). Prof. MacEachern is supported by NSF grant numbers SES-1921523 and DMS-2015552. The authors thank the editors for reviewing the article.

Disclosure statement

The authors have no conflicts of interest to report.