Model Checking with Right Censored Data Using Relative Belief Ratio

Abstract

Model checking is a topic of special interest in statistics. When data are censored, the problem becomes more difficult. This paper employs the relative belief ratio and the beta-Stacy process to develop a method for model checking in the presence of right-censored data. The proposed method for the given model of interest compares the concentration of the posterior distribution to the concentration of the prior distribution using a relative belief ratio. We propose a computational algorithm for the method and then illustrate the method through several data analysis examples.

1. Introduction

An important problem in statistics is to check whether a chosen statistical model is in agreement with observed data. This problem is known as model checking. The tests used to describe how well a statistical model fits a set of observations are goodness-of-fit tests. If it is determined that the observed data do not contradict the model, the true values of the parameters can be inferred. If the model fails to pass its checks, there is cause for concern about the accuracy of the inferences used in analysis. Thus, checking a proposed model on the basis of observed data is a critical component of statistical inference.

The time until an event occurs is the variable of interest in survival analysis. It is common to refer to this time as the survival time. In this case, the observed data are typically right censored. That is, at the right side, the exact survival time can become incomplete, for example, when a subject leaves the study before an event occurs, or when the study ends before the event occurs, right censoring occurs. This type of data are common in medical studies where the variable of interest is the time to death or time to relapse of a disease, in reliability studies where the variable of interest is the time until a machine part fails, and in social sciences where the variable of interest is the lifetime of the elderly in certain social programs [1]. While there are several well-known methods for model checking, the approach we want to take is Bayesian in nature. There is interest in developing nonparametric Bayesian procedures for hypothesis testing. The majority of this work focuses on uncensored data. The proposed model was embedded as a null hypothesis into a larger family of distributions, which is the main approach of model testing. Then priors were placed on the null and the alternative, and a Bayes factor was computed (see, for example, [2,3,4,5,6]). A second important approach for model testing is to apply a prior to the true distribution generating the data, and then measure the distance between the posterior distribution and the hypothesized distribution (see, for example, [7,8,9]). A third and more recent approach combines the second approach with relative belief ratio ([10]) (for instance, see the work of [11,12,13]).

When data are censored, goodness-of-fit tests become a challenging problem. This explains why there are few and scattered studies on goodness-of-fit tests for right-censored data. In particular, Bayesian goodness-of-fit tests are rarely covered. Some exceptions include the work of [14,15,16] considered the two sample problem by computing the Bayesian factors and beta process priors ([17]). Al-Labadi and Zarepour [15] used the beta-Stacy process and the Kolmogorov–Smirnov distance to test simple hypotheses. Chen and Hanson [16] used Polya tree priors and computed the Bayesian factor. To compute the posteriors, the MCMC procedure was used. Permutation tests were used to compute p-values. These approaches are computationally intensive.

The main problem addressed in this paper is model checking for lifetime distribution functions. A general description of the approach developed in this paper is provided along with a discussion of the benefits that the approach brings to the problem of model checking. The beta-Stacy process $BSP(k(t),F_0(t))$, $t\ge 0$ is considered as a prior on on the space of cumulative distribution functions ([18]). Here, $k(·)>0$ is a function defined on ${\mathbb{R}}^{+}$, and $F_0$ is a cumulative distribution function (cdf) on ${\mathbb{R}}^{+}=[0,\infty )$ and it represents the prior guess for the beta-Stacy process. How closely realizations from the process are clustered around $F_0$ is determined by the value of $k(·)$. The use of the beta-Stacy process is justified because, unlike the Dirichlet process, it is conjugate to both exact and right-censored observations ([19]). The proposed method then compares the posterior and prior distribution concentrations about the model of interest. If the posterior is more concentrated on the model than the prior, this is evidence in favor of the model; if the posterior is less concentrated, this is evidence against the model. This comparison is produced using a relative belief ratio that measures the evidence in the observed data for or against the model and provides a measure of the strength of this evidence; thus, the methodology is based on a direct measure of statistical evidence. The approach is fairly simple to implement and does not necessitate obtaining a closed form of the relative belief ratio. Theoretical results are developed to support appropriate hyperparameter selections $k(·)$ and $F_0$.

The rest of the paper is structured as follows. Section 2 provides the background on the definitions and generic properties of the beta-Stacy process, the relative belief ratio, and the Cramér–von Mises distance between probability measures. Section 3 discusses the proposed methodology and the selection of relevant values for the beta-Stacy process’s hyperparameters. Section 4 develops a computational algorithm for implementing the approach. Section 5 examines the performance of the methodology through a number of examples. This method works well in general and can be used for both censored and uncensored observations. Section 6 wraps up the paper with a summary of the findings. Lastly, datasets and figures are provided in the Appendix A.

2. Notations and Background

Let the random sample $X_1,\dots ,X_n$ be drawn from an unknown cdf F defined on ${\mathbb{R}}^{+}$. Let $(\mathbf{T},\mathit{\delta })=(T_1,{\delta }_1),\dots ,(T_n,$ ${\delta }_n)$ be observed, where $T_i=min(X_i,C_i)$, ${\delta }_i=I\left(X_i\le C_i\right)$ and $C_1,\dots ,C_n$ are the censoring times. When ${\delta }_i=1,$ $X_i$ is observed, and when ${\delta }_i=0,$ $X_i$ is right censored, this type of data can occur, for instance, when $X_i$ is the lifetime of a patient who enrols in a study of a certain disease. If death occurs from the disease before the study ends, $X_i$ is recorded; however, if the patient is still alive, withdraws, or dies for another reason at the end of the study, $C_i$ records the end of the period during which the patient was under observation ([20]).

In survival analysis, cdf F is frequently referred to as the lifetime distribution function. We use the same notation throughout this paper for the probability measure and its corresponding cumulative distribution function, i.e., $F(t)=F((-\infty ,t])$.

2.1. Beta-Stacy Process

Walker and Muliere [18] introduced the beta-Stacy process, which is a nonparametric prior that is widely used in survival analysis. In this section, a relevant introduction about the beta-Stacy process is provided.

The beta-Stacy process is defined by another process known as the log-beta process ([18]). Let $\beta (·)$ be a positive function defined on ${\mathbb{R}}^{+}$, and $\alpha$ be a measure concentrated on ${\mathbb{R}}^{+}$ that is absolutely continuous with respect to the Lebesgue measure such that ${\int }_0^{\infty }{\left(\beta (z)\right)}^{-1}\alpha (dz)=\infty .$ On the basis of [18], ${\left\{Z(t)\right\}}_{t\ge 0}$ is a log-beta process with parameters $\alpha$ and $\beta$, written as $Z(t)\sim logBSP(\alpha (t),\beta (t))$, if ${\left\{Z(t)\right\}}_{t\ge 0}$ is a Lévy process with Lévy measure defined, for $s>0$, by

$$L_t(ds)=\left. [\frac{1}{1-e^{-s}}{\int }_0^t{e}^{-s\beta (z)}\alpha (dz)\right]ds$$

and has the following moment generating function:

$$logE{e}^{-u{Z}_c(t)}={\int }_0^{\infty }\left(e^{-us}-1\right)L_t(ds),t\ge 0,u\in \mathbb{R}.$$

In order to define the beta-Stacy process, let positive function $k(·)$ be defined on ${\mathbb{R}}^{+}$ and the absolutely continuous cdf $F_0$ be defined on ${\mathbb{R}}^{+}$. We call ${\left\{F(t)\right\}}_{t\ge 0}$ a beta-Stacy process with parameters $k(·)$ and $F_0$, written as $F(t)\sim BSP(a(t),F_0(t))$, if $F(t)=1-e^{-Z(t)}$, and ${\left\{Z(t)\right\}}_{t\ge 0}$ is a log-beta process with parameters

$$\alpha (dz)=k(z)F_0(dz)\mathrm{and}\beta (z)=k(z)F_0([z,\infty )).$$

Walker and Muliere [18] showed that

$$E\left. [F(t)\right]=F_0(t)=1-exp\left(-{\int }_0^t{\left(\beta (z)\right)}^{-1}\alpha (dz)\right),$$ (1)

rendering $F_0$ the prior guess. The expression in (1) explains the need for assumption ${\int }_0^{\infty }{\left(\beta (z)\right)}^{-1}\alpha (dz)=\infty$. The beta-Stacy process includes various neutral-to-right processes proposed in the literature. For instance, the beta-Stacy process reduces to the Dirichlet process when $k(t)=k>0$ for all $t\ge 0$. The beta-Stacy process also reduces to a simple homogenous process ([21]) when $\beta (·)$ is constant.

Next, we describe the posterior distributions of ${\left\{Z(t)\right\}}_{t\ge 0}$ and ${\left\{F(t)\right\}}_{t\ge 0}$. Let random sample $X_1,\dots ,X_n$ be drawn from F, and $(\mathbf{T},\mathit{\delta })=(T_1,{\delta }_1),\dots ,(T_n,{\delta }_n)$ be observed, where $T_i=min(X_i,C_i),$ ${\delta }_i=I\left(X_i\le C_i\right)$ and $C_1,\dots ,C_n$ are censoring times. Define the counting process N by

$$N(t)=\sum _{i=1}^{n}I\left(T_i\le t\mathrm{and}{\delta }_i=1\right)$$

and the (left-continuous) at-risk process Y by

$$Y(t)=\sum _{i=1}^{n}I\left(T_i\ge t\right),$$

where I is the indicator function. In particular, let $N\left\{t\right\}=N(t)-N(t^{-})$ (i.e., $N\left\{t\right\}$ is the number of observed $X_i$’s at the exact position t). Let ${\left\{Z(t)\right\}}_{t\ge 0}\sim logBSP(\alpha (t),\beta (t))$ and $F(t)=1-exp\left(-Z(t)\right).$ Given the data $(\mathbf{T},\mathit{\delta })$, the posterior distribution of Z is a log-beta process ([18]) with parameters

$${\alpha }^{*}(t)=\alpha (t)+N(t)$$ (2)

and

$${\beta }^{*}(t)=\beta (t)+Y(t)-N\left\{t\right\}.$$ (3)

The posterior Lévy measure for $Z(t)$ is given by

$$L_t^{☆}(ds)=\left. [\frac{1}{1-e^{-s}}{\int }_0^t{e}^{-s\left(\beta (z)+Y(z)\right)}d\alpha (z)\right]ds.$$

There are fixed points of discontinuity in the posterior process. These extra points appear at the exact (uncensored) observations. If $t_i$ is an exact observation with corresponding jump $S_i$, then

$$1-exp(-S_i)\sim \mathrm{beta}\left(N\left\{t_i\right\},\beta (t_i)+Y(t_i)-N\left\{t_i\right\}\right).$$ (4)

If $N\left\{t_i\right\}=1,$ then the random jump $S_i$ follows an exponential density with mean ${\left. [\beta (t_i)+Y(t_i)-N\left\{t_i\right\}\right]}^{-1}.$

Let $F(t)\sim BSP\left(k(t),F_0(t)\right)$, $t\ge 0$. Given the data $(\mathbf{T},\mathit{\delta })$, the posterior distribution of F is a beta-Stacy process with parameters

$$\begin{array}{c}\hfill k^{*}(t)=\frac{{\beta }^{*}(t)}{F_0^{*}[t,\infty )}=\frac{\beta (t)+Y(t)-N\left\{t\right\}}{F_0^{*}[t,\infty )}\end{array}$$ (5)

and

$$\begin{array}{c}\hfill F_0^{*}(t)=1-\prod _{[0,t]}\left(1-\frac{d{\alpha }^{*}(z)}{{\beta }^{*}(z)+{\alpha }^{*}\left\{z\right\}}\right)=1-\prod _{[0,t]}\left(1-\frac{k(z)d{F}_0(z)+dN(z)}{k(z)F_0[z,\infty )+Y(z)}\right),\end{array}$$ (6)

with ${\alpha }^{*}(s)$ and ${\beta }^{*}(s)$ are defined in (2) and (3), and ∏ stands for the product integral. Note that, as $k(·)$ tends to zero, the nonparametric Kaplan–Meier estimator of the distribution function is obtained. On the other hand, $F^{*}$ becomes the prior guess $F_0$ as $k(·)$ grows large. Thus, parameter $k(t)$ can be viewed as the concentration parameter. The posterior consistency of the beta-Stacy process was addressed by Kim and Lee [22].

Algorithms A and B in Appendix A can be used to sample from prior and posterior beta-Stacy processes. These algorithms were developed by Al-Labadi and Zarepour [15] (see also Lee and Kim [23]).

2.2. Relative Belief Ratio

Evans’ (2015) relative belief ratio has become a widely used tool in statistical hypothesis testing theory. Assume that we have a statistical model defined by the pdf $\{f_{\theta }:\theta \in \mathsf{\Theta }\}$ with respect to the Lebesgue measure on the parameter space $\mathsf{\Theta }$. Let $\pi (\theta )$ to be a prior on $\mathsf{\Theta }$. After observing the data $(\mathbf{T},\mathit{\delta })$, the posterior distribution of $\theta$ is given by the conditional density function

$$\pi \left(\theta |(\mathbf{T},\mathit{\delta })\right)=\frac{f_{\theta }\left((\mathbf{T},\mathit{\delta })\right)\pi (\theta )}{{\int }_{\mathsf{\Theta }}{f}_{\theta }\left((\mathbf{T},\mathit{\delta })\right)\pi (\theta )d\theta },$$

Assume that we are interested in inferring regarding the parameter $\theta$. Let $\pi$ and $\pi (·|(\mathbf{T},\mathit{\delta }))$ be continuous at $\theta .$ Then, the relative belief ratio for a hypothesized value ${\theta }_0$ of $\theta$ is given by

$$R{B}_{\mathsf{\Theta }}({\theta }_0|(\mathbf{T},\mathit{\delta }))=\pi ({\theta }_0|(\mathbf{T},\mathit{\delta }))/\pi ({\theta }_0),$$ (7)

The posterior density to the prior density ratio at ${\theta }_0,$ that is, $RB({\theta }_0|(\mathbf{T},\mathit{\delta })),$ measures how beliefs about ${\theta }_0$ changed from a priori to a posteriori. When $\pi$ and $\pi (·|(\mathbf{T},\mathit{\delta }))$ are discrete, the relative belief ratio is defined through limits. For more information, see Evans (2015).

Quantity $R{B}_{\mathsf{\Theta }}({\theta }_0|(\mathbf{T},\mathit{\delta }))$ is a measure of evidence that ${\theta }_0$ is the true value. If $R{B}_{\mathsf{\Theta }}({\theta }_0|(\mathbf{T},\mathit{\delta }))>1$, then the probability of ${\theta }_0$ being the true value increases from a priori to a posteriori; thus, there is evidence based on the data that ${\theta }_0$ is the true value, and there is, hence, evidence in favor of ${\theta }_0$. If $R{B}_{\mathsf{\Theta }}({\theta }_0|(\mathbf{T},\mathit{\delta }))<1$; then, the probability of ${\theta }_0$ being the true value decreases from a priori to a posteriori. As a result, the data provide evidence that ${\theta }_0$ is not the true value. Case $R{B}_{\mathsf{\Theta }}({\theta }_0|(\mathbf{T},\mathit{\delta }))=1$ implies there is no evidence in either direction.

The ability to calibrate relative belief ratios is an appealing feature that renders it desirable in hypothesis testing problems. After calculating the relative belief ratio, it is critical to determine whether the result represents strong or weak evidence for or against $H_0:\theta ={\theta }_0$. A typical calibration $RB({\theta }_0|(\mathbf{T},\mathit{\delta }))$ is obtained by computing the tail probability (Evans, 2015)

$$St{r}_{\mathsf{\Theta }}({\theta }_0|(\mathbf{T},\mathit{\delta }))=\mathsf{\Pi }(R{B}_{\mathsf{\Theta }}(\theta |(\mathbf{T},\mathit{\delta }))\le R{B}_{\mathsf{\Theta }}({\theta }_0|(\mathbf{T},\mathit{\delta }))|(\mathbf{T},\mathit{\delta })),$$ (8)

where $\mathsf{\Pi }(·|(\mathbf{T},\mathit{\delta }))$ is the posterior distribution of the posterior density $\pi (·|(\mathbf{T},\mathit{\delta }))$. The posterior probability that the true value of $\theta$ has a relative belief ratio no greater than that of the hypothesized value ${\theta }_0$ is represented by Equation (8). When $R{B}_{\mathsf{\Theta }}({\theta }_0|(\mathbf{T},\mathit{\delta }))<1,$ there is evidence against ${\theta }_0,$ and a small value for $St{r}_{\mathsf{\Theta }}({\theta }_0|(\mathbf{T},\mathit{\delta }))$ represents a large posterior probability that the true value has a relative belief ratio greater than $R{B}_{\mathsf{\Theta }}({\theta }_0|(\mathbf{T},\mathit{\delta }))$ and hence there is strong evidence against ${\theta }_0.$ Similarly when $R{B}_{\mathsf{\Theta }}({\theta }_0|(\mathbf{T},\mathit{\delta }))>1,$ indicates a strong evidence in favour of ${\theta }_0,$ while a small value of $St{r}_{\mathsf{\Theta }}({\theta }_0|(\mathbf{T},\mathit{\delta }))$ indicates weak evidence in favour of ${\theta }_0.$ Figure 1 illustrates the strength of the evidence for both cases; $R{B}_{\mathsf{\Theta }}({\theta }_0|(\mathbf{T},\mathit{\delta }))<1$ and $R{B}_{\mathsf{\Theta }}({\theta }_0|(\mathbf{T},\mathit{\delta }))>1.$

Figure 1.

Strength of evidence. The shaded area represents $St{r}_{\mathsf{\Theta }}({\theta }_0|(\mathbf{T},\mathit{\delta })).$ RB represents the relative belief ratio $R{B}_{\mathsf{\Theta }}({\theta }_0|(\mathbf{T},\mathit{\delta })).$ The smaller the shaded area, the stronger the evidence for case (a). The larger the shaded area, the stronger the evidence for case (b).

2.3. Cramér–Von Mises Distance

Let F and G be two cdfs; Cramér–von Mises distance between F and G is defined as

$$d(F,G)={\int }_{-\infty }^{\infty }{\left(F(x)-G(x)\right)}^{2}G(dx).$$

Other distances can be used (see Gibbs and Su (2002)), but d has some computational advantages. The formula for the distance between the beta-Stacy process and a continuous cdf is provided in the following result.

Lemma 1. 

Let $F_0$ be a continuous cdf and $F_{ϵ}=1-exp\left(-Z_{ϵ}(t)\right)$, where $Z_{ϵ}(t)={\sum }_{i=1}^M{J}_i{\delta }_{{\theta }_i}\left(\left. [0,t\right]\right)$, where $t\ge 0$ with ${\theta }_1,\dots ,{\theta }_M\stackrel{i.i.d.}{\sim }{F}_0$. Let ${\theta }_{(1)}\le \dots \le {\theta }_{(M)}$ denote the order statistics of ${\theta }_1,\dots ,{\theta }_M$ and $J_1^{\prime },\cdots ,J_M^{\prime }$ be the associated jump sizes such that $J_i=J_j^{\prime }$ when ${\theta }_i={\theta }_{(j)}.$ Then

$$\begin{array}{ccc}\hfill d(F_{ϵ},F_0)& =& \frac{1}{3}-\sum _{i=0}^M{F}_{ϵ}({\theta }_{(i)})\left. [{(F_0({\theta }_{(i+1)}))}^2-{(F_0({\theta }_{(i)}))}^2\right]\hfill \\ & & +\sum _{i=0}^M{(F_{ϵ}({\theta }_{(i)}))}^2\left. [F_0({\theta }_{(i+1)})-F_0({\theta }_{(i)})\right].\hfill \end{array}$$

where $F_{ϵ}({\theta }_{(i)})=1-exp\left(-{\sum }_{k=1}^i{J}_k^{\prime }\right)$.

Proof. 

Note that

$$F_{ϵ}(x)=\left\{\begin{array}{cc}0\hfill & \mathrm{if}x<{\theta }_{(1)}\hfill \\ F_{ϵ}({\theta }_{(i)})\hfill & \mathrm{if}{\theta }_{(i)}\le x<{\theta }_{(i+1)}(i=1,\dots ,M-1).\hfill \\ 1\hfill & \mathrm{if}x\ge {\theta }_{(M)}\hfill \end{array}\right.$$

Le ${\theta }_{(0)}=0$ and ${\theta }_{(M+1)}=+\infty .$ Then,

$$\begin{array}{ccc}\hfill d(F_{ϵ},F_0)& =& {\int }_{{\theta }_{(0)}}^{{\theta }_{(ϵ+1)}}{\left. [F_{ϵ}(x)-F_0(x)\right]}^2{f}_0(x)dx\hfill \\ \hfill & =& \sum _{i=0}^M{\int }_{{\theta }_{(i)}}^{{\theta }_{(i+1)}}{\left. [F_{ϵ}({\theta }_{(i)})-F_0(x)\right]}^2{f}_0(x)dx.\hfill \end{array}$$

Substituting $y=F_0(x)$ and $U_{(i)}=F_0({\theta }_{(i)})$ gives

$$\begin{array}{ccc}\hfill d(F_{ϵ},F_0)& =& \sum _{i=0}^M{\int }_{U_{(i)}}^{U_{(i+1)}}{\left. [F_{ϵ}({\theta }_{(i)})-y\right]}^{2}dy\hfill \\ \hfill & =& \frac{1}{3}\sum _{i=0}^M\left\{{[F_{ϵ}({\theta }_{(i)})-U_{(i)}]}^3-{[F_{ϵ}({\theta }_{(i)})-U_{(i+1)}]}^3\right\}.\hfill \\ \hfill & =& \frac{1}{3}\sum _{i=0}^M\left. [U_{(i+1)}^3-U_{(i)}^3\right]-\sum _{i=0}^M{F}_{ϵ}({\theta }_{(i)})\left. [U_{(i+1)}^2-U_{(i)}^2\right]\hfill \\ \hfill & & +\sum _{i=0}^M{F}_{ϵ}^2({\theta }_{(i)})\left. [U_{(i+1)}-U_{(i)}\right]\hfill \\ \hfill & =& \frac{1}{3}-\sum _{i=0}^M{F}_{ϵ}({\theta }_{(i)})\left. [U_{(i+1)}^2-U_{(i)}^2\right]+\sum _{i=0}^M((F_{ϵ}){({\theta }_{(i)})}^2\left. [U_{(i+1)}-U_{(i)}\right]\hfill \\ \hfill & =& \frac{1}{3}-\sum _{i=0}^M{F}_{ϵ}({\theta }_{(i)})\left. [{(F_0({\theta }_{(i+1)}))}^2-{(F_0({\theta }_{(i)}))}^2\right]\hfill \\ & & +\sum _{i=0}^M{(F_{ϵ}{\theta }_{(i)})}^2\left. [F_0({\theta }_{(i+1)})-F_0({\theta }_{(i)})\right].\hfill \end{array}$$

□

When considering the prior and posterior distributions of the Cramér–von Mises distance, the following lemma allows for the use of the approximation to $BSP(k(·),F_0)$.

Lemma 2. 

If $F\sim BSP(k(·),F_0)$ and $F_{ϵ}$ is given by Algorithm A, then $d\left(F_{ϵ},F_0\right)\stackrel{a.s.}{\to }d\left(F,F_0\right)$ as $ϵ\to 0.$

Proof. 

By Kim and Lee (2001), $F_{ϵ}(x)\stackrel{a.s.}{\to }F(x)$, where $a.s.$ stands for almost surely convergence. As ${(F_{ϵ}(x)-F_0(x))}^2\le f_0(x)$, where $f_0(x)$ is integrable, the proof follows from the dominated convergence theorem. □

3. Model Checking Using the Relative Belief

Consider the statistical model $\left\{F_{\theta }:\theta \in \mathsf{\Theta }\right\}$ of continuous cdf on ${\mathbb{R}}^{+}$. Let $X_1,\dots ,$ $X_n$ be a random sample drawn from an unknown cdf F defined on ${\mathbb{R}}^{+}$. Assume that $(\mathbf{T},\mathit{\delta })=((T_1,{\delta }_1),\dots ,(T_n,{\delta }_n))$ is observed, where $T_i=min(X_i,C_i),$ ${\delta }_i=I\left(X_i\le C_i\right)$ and $C_1,\dots ,C_n$ are censoring times. The aim is to test the hypothesis ${\mathcal{H}}_0:F\in \left\{F_{\theta }:\theta \in \mathsf{\Theta }\right\}$. Let $BSP(k(·),F_0)$ be the prior on F for some choice of $k(·)$ and $F_0$. Then $F(t)|(\mathbf{T},\mathit{\delta })\sim BSP\left(k^{*}(t),F_0^{*}(t)\right)$, where $k(·)$ and $F_0$ are defined in (5) and (6), respectively. If ${\mathcal{H}}_0$ is true, then the posterior distribution of the distance between F and the proposed model should be more concentrated around 0 than the prior distribution. As a result, this test involves comparing the concentrations of the prior and the posterior distributions of d (see Lemma 1) about 0 using the relative belief ratio with the interpretation as discussed in Section 2.2.

To perform this test, we must measure the distance and then set relevant values for $k(·)$ and $F_0$. To calculate the distance, similar to Al-Labadi and Evans [24], we computed $d(F,F_{\widehat{\theta }})$, where $\widehat{\theta }$ is the relative belief estimate of $\theta ,$ which is always the same as the maximal likelihood estimate (MLE) for the full model parameter. In terms of hyperparameters, we set $F_0=F_{\widehat{\theta }}$ and $k(t)=k$ for all t. There are numerous advantages in setting $F_0=F_{\widehat{\theta }}$. First, it avoids prior-data conflict, a possible contradiction between the data and the prior. This typically happens when the prior places its mass in a region of the parameter space where the data suggest the true value does not lie ([25,26]). In the context of the approach considered in this paper, prior-data conflict arises whenever there is a small overlap between the effective support regions of F and $F_{\widehat{\theta }}$. Note that, by Lemma 1, the distance $d(F,F_{\widehat{\theta }})$ depends on the prior guess $F_0$ through the jump points ${\theta }_i$. If the ${\theta }_i$ lay in one tail of $F_{\widehat{\theta }}$, then we get prior-data conflict between F and $F_{\widehat{\theta }}$ because $F_0$ and F had the same effective support. To avoid this, it is required that ${\theta }_i$ are selected in a region that includes most of the mass of $F_{\widehat{\theta }}$. When $F_0=F_{\widehat{\theta }}$, then $F_{\widehat{\theta }}$ is the prior mean of F; thus, both share the same effective support, which renders it a reasonable choice to avoid prior-data conflict. We refer the reader to Example 1 of Al Labadi and Evans (2018) for an interesting discussion about prior-data conflict. Nevertheless, the choice of $F_0=F_{\widehat{\theta }}$ should also avoid any impacts due to the “double use of the data”. This means that the approach becomes conservative in detecting the model failure when $H_0$ is false. Although setting $F_0=F_{\widehat{\theta }}$ appears to induce a data-dependent prior distribution for d, the following lemma implies that this is not the case; thus, the approach is prior distribution-free with this choice.

Lemma 3. 

If $F\sim BSP\left(k(·),F_{\widehat{\theta }}\right)$, then the distribution of $d\left(F,F_{\widehat{\theta }}\right)$ does not depend on $F_{\widehat{\theta }}$.

Proof. 

Using Lemma 1, since ${({\theta }_i)}_{1\le i\le M}$ is a sequence of i.i.d. random variables with continuous distribution $F_{\widehat{\theta }},$ for $i\ge 1,$ we have $U_i\stackrel{d}{=}{F}_{\widehat{\theta }}({\theta }_i)$, where ${\left(U_i\right)}_{1\le i\le M}$ is a sequence of i.i.d. random variables follow a uniform distribution on $[0,1]$. Thus,

$$\begin{array}{ccc}\hfill d(F_{ϵ},F_{\widehat{\theta }})& \stackrel{d}{=}& \frac{1}{3}-\sum _{i=0}^M{F}_{ϵ}({\theta }_{(i)})\left. [{(U(i+1))}^2-{({\theta }_{(i)})}^2\right]\hfill \\ & & +\sum _{i=0}^M{(F_{ϵ}({\theta }_{(i)}))}^2\left. [U_{(i+1)}-U_{(i)}\right],\hfill \end{array}$$

where $U_{(i)}$ is the i-th order statistic for ${\left(U_i\right)}_{1\le i\le M}$ i.i.d. uniform$[0,1].$ Now, as $ϵ\to 0$, by Lemma 2, we conclude that the distribution of $d(F,F_{\widehat{\theta }})$ does not depend on $F_{\widehat{\theta }}$. □

The following results shows that setting $F_0=F_{\widehat{\theta }}$ prevents any effect due to the double use of the data. Specifically, as the sample size increases, the posterior distribution of $d\left(F,F_{\widehat{\theta }}\right)$ becomes concentrated around 0 if and only if ${\mathcal{H}}_0$ is true. For the proof, see Al-Labadi and Evans (2018).

Lemma 4. 

Let $(\mathbf{T},\mathit{\delta })=((T_1,{\delta }_1),\dots ,(T_n,{\delta }_n))\sim F$, where $F\sim BSP\left(k(·),F_{\widehat{\theta }}\right)$. Suppose that $\widehat{\theta }\stackrel{a.s.}{\to }{\theta }_0,{sup}_y|F_{\widehat{\theta }}(y)-F_{{\theta }_0}(y)|\stackrel{a.s.}{\to }0$ as $n\to \infty .$

• (i)  If ${\mathcal{H}}_0$ is true, then, as $n\to \infty$, $d\left(F|(\mathbf{T},\mathit{\delta }),F_{\widehat{\theta }}\right)\stackrel{a.s.}{\to }0$.
• (ii)  If ${\mathcal{H}}_0$ is false, then, as $n\to \infty$, $liminfd(F|(\mathbf{T},\mathit{\delta }),F_{\widehat{\theta }})\stackrel{a.s.}{>}0.$

Now, concerning the choice of $k(·)$, we considered $k(t)=k$ for all $t>0.$ In general, larger values of k must be chosen to identify smaller deviations. Consequently, it is possible to consider multiple values of k. One way to perform that is, for instance, to start with $k=1$. If a larger (smaller) value of k renders the relative belief ratios to be below (above) 1, ${\mathcal{H}}_0$ is rejected (accepted). As Section 5 shows, when the null hypothesis is correct (not correct), the relative belief ratio always remains above (below) 1 when larger (smaller) values of k are considered. When using the Dirichlet process, Al-Labadi and Zarepour [27] advised using $k\le 0.5n$ for complete data to avoid the prior becoming too influential. Setting k between 1 and 10 is satisfactory for most purposes. As indicated in the introduction, when $k(t)=k$, the beta-Stacy process turns into the Dirichlet process. However, in the presence of right censored data, the posterior distribution of the Dirichlet process becomes beta-Stacy process ([19]). This justifies the necessity of using the beta-Stacy process in the approach.

4. Computational Algorithm

Closed forms of the prior and posterior densities of $D=d(F,F_{\widehat{\theta }})$ are required to compute the relative belief ratio as in (7). This is not usually available. As a result, the relative belief ratio must be approximated through simulation. A particular problem of computing (7) arises here when both ${\pi }_D(0)$ and ${\pi }_{D|(\mathbf{T},\mathit{\delta })}(0)$ are close to 0, where ${\pi }_D(·)$ and ${\pi }_{D|(\mathbf{T},\mathit{\delta })}(·)$ denote the pdf’s of D and $D|(\mathbf{T},\mathit{\delta })$, respectively. In such a case, determining $R{B}_D(0|(\mathbf{T},\mathit{\delta }))$ is difficult. The formal definition of the relative belief ratio, as discussed in Section 2.2, is as a limit that can be approximated at zero by

$$\begin{array}{c}\hfill {\widehat{RB}}_D(0|\mathbf{T},\mathit{\delta })=\frac{{\mathsf{\Pi }}_D([0,d_c))}{{\mathsf{\Pi }}_{D|(\mathbf{T},\mathit{\delta })}([0,d_c))},\end{array}$$ (9)

for a suitably small value $d_c$, where ${\mathsf{\Pi }}_D(·)$ and ${\mathsf{\Pi }}_{D|(\mathbf{T},\mathit{\delta })}(·)$ denote the cdfs of D and $D|(\mathbf{T},\mathit{\delta })$, respectively. From Equation (8), the strength of the evidence based on the relative belief ratio $R{B}_D(0|\mathbf{T},\mathit{\delta })$ can be computed using

$$\begin{array}{c}\hfill St{r}_D(0|(\mathbf{T},\mathit{\delta }))={\mathsf{\Pi }}_{D|(\mathbf{T},\mathit{\delta })}(R{B}_D(d|(\mathbf{T},\mathit{\delta }))\le R{B}_D(0|(\mathbf{T},\mathit{\delta }))).\end{array}$$ (10)

Appendix B provides computational Algorithm C for assessing ${\mathcal{H}}_0$ on the basis of estimates of (9) and (10). A similar algorithm for complete data based on the Dirichlet process was developed by Al-Labadi and Evans [24].

5. Examples

The approach is demonstrated by two main examples in this section. Throughout this section, let Exp $(\lambda )$, Weibull $(k,\lambda )$, and Lognorma l $(\mu ,\sigma )$ denote the exponential distribution with mean $1/\lambda$, the Weibull distribution with shape parameter k and scale parameter $\lambda$, and the log-normal distribution with mean $\mu$ and standard deviation $\sigma$, respectively. In all examples, the sensitivity to choosing k is investigated. We set $ϵ=0.01$, $i_0=1$, $M_0=20$, and $r_1=r_2=1000$ in Algorithms A, B, and C, though other values are also possible. $\mathsf{R}$ package parmsurvfit was used to compute the MLE of the distribution parameters.

Example 1. 

A real dataset from Lee and Wang (2003) based on the remission times (in months) of cancer patients. The dataset is given in Appendix C. We tested hypothesis

${\mathcal{H}}_0:$

F, given in Table 1, is the underlying distribution of the observed data,

where F could be a family of distributions (composite hypothesis) or a specific distribution with known parameters (simple hypothesis). Various values of k were considered to investigate the approach’s sensitivity to concentration parameter selection. The p-value of the (frequentist) log-rank test was also computed for comparison purposes. Table 1 summarizes the findings. When ${\mathcal{H}}_0$ is true, we want $RB>1$ and a strength close to 1, and when ${\mathcal{H}}_0$ is false, we want $RB<1$ and a strength close to 0. According to Table 1, the proposed test performed well in this example.

Table 1.

Model checking of Example 1.

F	${\mathit{F}}_0$	k	RB(Str)	p-Value
Exp$(\lambda )$	Exp$(\widehat{\lambda }=0.0962)$	1	20(1)	0.6521
		5	19.84(1)
		10	15.98(1)
Weibull$(k,\lambda )$	Weibull$(\widehat{k}=1.0536,\widehat{\lambda }=10.1901)$	1	20(1)	0.9986
		5	19.76(1)
		10	17.58(1)
Lognormal$(\mu ,\sigma )$	Lognormal$(\widehat{\mu }=1.8198,\widehat{\sigma }=1.0912)$	1	20(1)	0.7550
		5	18.48(1)
		10	15.52 (1)
Exp$(\lambda =2)$	Exp$(\lambda =2)$	1	0(0)	0
		5	0(0)
		10	0(0)
Weibull$(2,0.5)$	Weibull$(2,0.5)$	1	0(0)	0
		5	0(0)
		10	0(0)
Lognormal$(3,1)$	Lognormal$(2,1)$	1	0(0)	0
		5	0(0)
		10	0(0)

Example 2. 

Simulated data. The primary purpose of this dataset is to investigate how the proposed test performs as the sample size increases. We considered data $(T_1,{\delta }_1),\dots ,(T_n,{\delta }_n)$, where $T_i=min(X_i,C_i)$ where the survival times ${(X_i)}_{1\le i\le n}$ were generated from Lognormal$(1,4)$, while the censored time ${(C_i)}_{1\le i\le n}$ are generated from Lognormal $(4,1)$.

${\mathcal{H}}_0:$

F in Table 2 is the underlying distribution of the observed data.

Table 2 summarizes the results that show that the selected models were accepted. Figure A1, Figure A2 and Figure A3 (see Appendix D) give the plots of $F_0=F_{\widehat{\theta }}$ and 5 sample paths each for the prior beta-Stacy process and the posterior beta-Stacy process for each case in Table 2. These figures clearly show that the plots of the sample paths for the posterior process moved toward the plot of $F_0.$ This supports the previous conclusion regarding the null hypothesis. Furthermore, in this case, the p-values of the (frequentist) log-rank test support the conclusion that the null hypothesis should not be rejected.

Table 2.

Model checking of Example 2.

F	n	${\mathit{F}}_0$	c	$\mathit{RB}(\mathit{Str})$	p-Value
Exp$(\lambda )$	20	Exp$(\widehat{\lambda }=0.03)$	1	3.4(0.83)	0.9995
			5	0.76(0.08)
			10	0.36(0)
	50	Exp$(\widehat{\lambda }=0.03)$	1	0.24(0.02)	0.9977
			5	0.02(0)
			10	0(0)
	100	Exp$(\widehat{\lambda }=0.05)$	1	0.2(0.02)	0.9992
			5	0(0)
			10	0(0)
Weibull$(k,\lambda )$	20	Weibull$(\widehat{k}=0.38,\widehat{\lambda }=20.88)$	1	18.12(1)	1
			5	9.78 (1)
			10	5.22(0.739)
	50	Weibull$(\widehat{k}=0.36,\widehat{\lambda }=19.77)$	1	19.88(1)	0.9992
			5	15.98(1)
			10	11.12(1)
	100	Weibull$(\widehat{k}=0.38,\widehat{\lambda }=14.10)$	1	20(1)	0.9991
			5	19.48(1)
			10	16.46(1)
Lognormal$(\mu ,\sigma )$	20	Lognormal$(\widehat{\mu }=1.14,\widehat{\sigma }=2.40)$	1	11.68(0.416)	0.9867
			5	3.52(1)
			10	1.88(1)
	50	Lognormal$(\widehat{\mu }=1.63,\widehat{\sigma }=3.64)$	1	20(1)	0.9454
			5	16.4(1)
			10	12.4(1)
	100	Lognormal$(\widehat{\mu }=1.34,\widehat{\sigma }=3.44)$	1	20(1)	0.9373
			5	19.36 (1)
			10	16.12(1)

6. Concluding Remarks

The beta-Stacy process and relative belief ratio were used to propose a general approach for model checking. This method could be used for both complete and right-censored data. It could also be used to test composite or simple hypotheses. Several examples demonstrated that the approach works very well.

Though the Cramér–von Mises distance was used here, other distance measures such as the Kolmogorov–Smirnov and Anderson–Darling distances are viable alternatives. Testing for families of multivariate distributions is an important extension of the approach presented in this paper. While conceptually similar, computational and inferential issues must be addressed. This problem can be addressed in future work.

Appendix A. Simulation Algorithms

Algorithm A: Simulation algorithm to approximate the prior of beta-Stacy process.

1. Generate log-beta process with parameters $\alpha (dz)=k(z)F_0(dz)$ and $\beta (z)=k(z)F_0[z,\infty ).$ The following steps are required to accomplish this:

   • (a)Fix a small positive number $ϵ$.
   • (b)Generate the total number of jumps $M\sim Poisson\left({\lambda }_{ϵ}\right),$ where ${\lambda }_{ϵ}=\alpha ([0,\infty ))/ϵ$.
   • (c)For $i=1,\dots M,$ generate the jump times ${\theta }_i$ from the probability density function $\alpha (dz)/\alpha ([0,\infty )).$
   • (d)Generate the jump sizes $J_1,\dots ,J_M$ such that $1-exp(-J_i)|{\theta }_i\sim Beta(ϵ,\beta ({\theta }_i)).$
   • (e)
     Set

     $$\begin{array}{c}\hfill Z_{ϵ}(t)=\sum _{k=1}^M{J}_{k}I({\theta }_k\le t).\end{array}$$ (A1)

2. The approximate prior beta-Stacy process is $F_{ϵ}(t)=1-exp\left(-Z_{ϵ}(t)\right)$.

Algorithm B: Simulation algorithm to approximate the posterior of beta-Stacy process.

1. Generate the posterior log-beta process. The following steps are required to accomplish this:

   • (a)Generate the process ${\left\{Z_{ϵ}^{*}(t)\right\}}_{t\ge 0}$ on the basis of Algorithm A with $\beta ({\theta }_i)$ is replaced by $\beta ({\theta }_i)+Y({\theta }_i).$
   • (b)Generate the process $Z_f^{*}(t)={\sum }_{i=1}^l{S}_i^{*}I\left(T_i\le t\right)$, where the random jumps ${(S_i^{*})}_{1\le i\le l}$ are associated with the fixed points of discontinuity from the distribution given in (4), where $l\le n$. These random jumps occur at points where the data are not right-censored.
   • (c)
     The approximated posterior log-beta process is given by:

     $$\begin{array}{ccc}\hfill Z_{ϵ}^{*}(t)=Z_{ϵ}(t)|(\mathbf{T},\mathit{\delta })& =& Z_{ϵ}^{*}(t)+Z_f^{*}(t)\hfill \\ & =& \sum _{i=1}^M{J}_i^{*}I({\theta }_i\le t)+\sum _{i=1}^l{S}_i^{*}I\left(t_i\le t\right).\hfill \end{array}$$

2. The approximate posterior beta-Stacy is $F_{ϵ}^{*}(t)=F_{ϵ}(t)|(\mathbf{T},\mathit{\delta })=1-exp\left(-Z_{ϵ}^{*}(t)\right).$

Appendix B. Algorithm for Model Checking

Algorithm C (Relative belief algorithm for model checking):

(i) Generate a sample from $F_{ϵ}$, where $F_{ϵ}$ is an approximation of $F\sim BSP(k(·),F_{\widehat{\theta }})$. See Algorithm A.

(ii) Compute $d(pri)=d(F_{ϵ},F_{\widehat{\theta }})$ as described in Lemma 2.

(iii) Repeat Steps (i) and (ii) to obtain a sample of $r_1$ values from the prior of D.

(iv) Generate a sample from $F_{ϵ}|(\mathbf{T},\mathit{\delta })$, where $F_{ϵ}|(\mathbf{T},\mathit{\delta })$ an approximation of F. See Algorithm B.

(v) Compute $d(pos)=d\left(F_{ϵ}|(\mathbf{T},\mathit{\delta }),F_{\widehat{\theta }}\right)$ as described in Lemma 2.

(vi) Repeat Steps (iv) and (v) to obtain a sample of $r_2$ values from the posterior of D.

(vii) For a fixed positive number $M_0$, let ${\widehat{F}}_D$ denote the empirical cdf of D based on the prior sample in (iii) and for $i=0,\dots ,M_0,$ let ${\widehat{d}}_{i/M_0}(pri)$ be the estimate of $d_{i/M_0}(pri),$ the $(i/M_0)$-th prior quantile of $D.$ Here ${\widehat{d}}_0(pri)=0$, and ${\widehat{d}}_1(pri)$ is the largest value of $d(pri)$. Let ${\widehat{F}}_D(·|(\mathbf{T},\mathit{\delta }))$ denote the empirical cdf of D based on the posterior sample of $d(pos)$ in (vi). For $d\in [{\widehat{d}}_{i/M_0}(pri),{\widehat{d}}_{(i+1)/M_0}(pri))$, estimate $R{B}_D(d|(\mathbf{T},\mathit{\delta }))$ by the ratio of the estimates of the posterior and prior contents of $[{\widehat{d}}_{i/M_0}(pri),{\widehat{d}}_{(i+1)/M_0}(pri)).$ Specifically,

$${\widehat{RB}}_D(d|(\mathbf{T},\mathit{\delta }))=M_0\{{\widehat{F}}_D({\widehat{d}}_{(i+1)/M_0}(pri)|(\mathbf{T},\mathit{\delta }))-{\widehat{F}}_D({\widehat{d}}_{i/M_0}(pri)|(\mathbf{T},\mathit{\delta }))\},$$ (A2)

Moreover, estimate (9) by ${\widehat{RB}}_D(0|(\mathbf{T},\mathit{\delta }))=M_0{\widehat{F}}_D({\widehat{d}}_{i_0/M_0}(pri)|(\mathbf{T},\mathit{\delta }))$ where $i_0$ is chosen so that $i_0/M_0$ is not too small (typically $i_0/M_0\approx 0.05)$.

(viii) Estimate (10) by the finite sum

$$\sum _A({\widehat{F}}_D({\widehat{d}}_{(i+1)/M_0}(pri)|(\mathbf{T},\mathit{\delta }))-{\widehat{F}}_D({\widehat{d}}_{i/M}(pri)|(\mathbf{T},\mathit{\delta }))),$$ (A3)

where $A=\{i\ge i_0:{\widehat{RB}}_D({\widehat{d}}_{i/M_0}(pri)|(\mathbf{T},\mathit{\delta }))\le {\widehat{RB}}_D(0|(\mathbf{T},\mathit{\delta }))\}$.

By Al-Labadi and Evans (2018), for fixed $M_0,$ as $r_1\to \infty ,r_2\to \infty ,$ (A2) converge a.s. to $R{B}_D(d|(\mathbf{T},\mathit{\delta }))$ and (A3) converge a.s. to $D{P}_D(R{B}_D(d|(\mathbf{T},\mathit{\delta }))\le R{B}_D(0|(\mathbf{T},\mathit{\delta }))|(\mathbf{T},\mathit{\delta }))$.

Appendix C. Example 1 Dataset

Table A1.

Remission times (months) of 137 cancer patients of the dataset used in Example 1. + denotes a right-censored observation.

4.5	19.13	14.24	7.87	5.49	2.02	9.22	3.82	26.31	4.65+
2.62	0.90	21.73	0.87+	0.51	3.36	43.01	0.81	3.36	1.46
24.80+	10.86+	17.14	15.96	22.69	4.33	7.28	2.46	3.48	4.23
6.54	8.65	5.41	2.23	4.34	32.15	4.87	5.71	7.59	3.02
4.51	1.05	9.47	79.05	2.02	4.26	11.25	10.34	10.66	12.03
2.64	14.74	1.19	8.66	14.83	5.62	18.10	25.74	17.36	1.35
9.02	6.94	7.26	4.70+	3.70	3.64	3.57	11.64	6.25	25.82
3.88	3.02+	19.36+	20.28	46.12	5.17	0.20	36.66	10.06	4.94
5.06	16.62	12.07	6.97	0.08	1.40	2.75	7.32	1.26	6.76
8.60+	7.62	3.52	9.74	0.40	5.41	2.54	2.69	8.26	0.50
5.32	5.09	2.09	7.93	12.02	13.08	5.85	7.09	5.32	4.33+
2.83	8.37	14.77	8.53	11.98	1.76	4.40	34.26	2.07	17.12
12.63	7.66	4.18	13.29	23.63	3.25	7.63	2.87	3.31	2.26
2.69	11.79	5.34	6.93	10.75	13.11	7.39

Appendix D. Example 2 Figures

Figure A1.

Plots of the sample paths for the model, prior and posterior of $BS(a=0.01,F_0)$ with different choices of n and k in Example 2.

Figure A2.

Plots of the sample paths for the model, prior and posterior of $BS(a=0.01,F_0)$ with different choices of n and k in Example 2.

Figure A3.

Plots of the sample paths for the model, prior and posterior of $BS(a=0.01,F_0)$ with different choices of n and k in Example 2.

Author Contributions

Formal analysis, A.A.; Investigation, L.A.-L. and M.A.; Methodology, L.A.-L. and A.A.; Project administration, L.A.-L.; Software, L.A.-L. and M.A.; Writing – review & editing, A.A. and M.A. All authors have read and agreed to the published version of the manuscript.

Institutional Review Board Statement

Not applicable.

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest.

Funding Statement

This research received no external funding.