EVALUATING MULTIPLEX DIAGNOSTIC TEST USING PARTIALLY ORDERED BAYES CLASSIFIER

Abstract

In a diagnostic test using multiplex assay, each individual biomarker is often expected to have monotonic association with the disease outcome and therefore the underlying disease classification rule is partially ordered with respect to the biomarkers. Nonparametric estimation of the classification rule can be accomplished by projecting an unconstrained Bayes estimator onto the partial ordering subspace. However, computing the projection is challenging as it involves performing maximization over a constrained parameter space whose size grows exponentially with the sample size. We introduce a novel sequential update method for projection-based nonparametric estimation of the disease classification rule and propose new recursive algorithms to implement the method. The proposed algorithms yields the exact Bayes solution that maximizes the posterior gain with respect to a classification-type gain function. When compared to an existing algorithm that gives approximate Bayes solution, our algorithms accomplish the same tasks with much reduced elapsed time in simulation study. We apply the sequential update method to evaluate a human papillomavirus test for cervical cancer precursor lesions and derive diagnostic rule that improves accuracy on existing estimation methods including parametric logistic regression and monotone generalized additive models.

1. Introduction.

A multiplex diagnostic assay detects multiple biomarkers and hence its ability to detect a disease depends on how the biomarkers are combined to derive a disease classification rule. To illustrate, Kuhn et al. (2020) conduct a diagnostic accuracy study of a human papillomavirus (HPV) assay for cervical cancer. The assay detects different high-risk HPV types expressed in five fluorescent channels. In the study, cervical biopsies as well as HPV tests were performed in a cohort of women living with and without HIV so that there would be data to estimate the probability $\theta \left(\mathit{x}\right)$ of detecting cervical cancer precursor lesions in biopsy for a given set of HPV channel values ($\mathit{x}$). While the study is designed to assess the assay’s diagnostic performance by evaluating the receiver operating characteristic (ROC) curve (DeLong, DeLong and Clark-Pearson, 1988; Pepe, 1997), it also aims to examine and re-define the disease classification rule based on the HPV values with a motivation to improve specificity of the test to prevent overtreatment in the population. Specifically, we consider disease classification rule that takes the form $\gamma \left(\mathit{x}\right)=I\left\{\theta \left(\mathit{x}\right)>t_0\right\}$ for some threshold $t_0$.

There are extensive discussions on statistical methods that combine multiple biomarkers or diagnostic tests using linear combinations, i.e., $\gamma \left(\mathit{x}\right)$ depends on $\mathit{x}$ only through ${\mathit{c}}^{\prime }\mathit{x}$ for some coefficient $\mathit{c}$; see for example Su and Liu (1993); Richards, Hammitt and Tsevat (1996); Pepe and Thompson (2000); Jin and Lu (2009); Liu, Liu and Halabi (2011). Briefly, these methods estimate the coefficients of the optimal linear combination of the biomarkers to form a composite score, which is then used as the basis of disease classification. For example, under the logistic regression model, Jin and Lu (2009) prove that the optimal $\mathit{c}$ that gives the largest sensitivity for any given specificity is the same as the set of biomarker coefficients in the logistic regression model. Methods based on linear combination of biomarkers apparently rely on the model assumptions about the link function and the linearity of the biomarker effects. Additionally, while interaction effects between biomarkers can in principle be included in the model, the selection of interaction terms can be ad hoc. These methods can also be sensitive to the scale of the biomarkers, and some methods can be computationally intensive when the number of biomarkers is large (e.g., Pepe and Thompson (2000)).

To circumvent the above issues, we consider nonparametric estimation of the classification rule $\gamma \left(\mathit{x}\right)$ assuming only monotonicity. In the motivating example, we expect each individual HPV channel output will have a non-negative association with cervical cancer, i.e., a higher HPV value will not be associated with lower probability of cancer diagnosis while allowing for the possibility of no association. In other words, the underlying $\theta \left(\mathit{x}\right)$ is a monotonic function of individual biomarkers, i.e., all else being fixed, the direction of the effect of each biomarker is known to be nondecreasing (or nonincreasing). This puts the estimation of $\theta \left(\mathit{x}\right)$ and $\gamma \left(\mathit{x}\right)$ as a monotone regression problem. There is a long history of monotone regression with one variable, e.g., see Brunk (1955) and Ayer et al. (1955). The literature on multivariate monotone regression has grown recently with numerous approaches, including additive models (Bacchetti, 1989), splines with monotone basis (Ramsay, 1988), Bayesian mixture modeling (Bornkamp, Ickstadt and Dunson, 2010), monotonic behavior chains (Wan and McAuley, 2018), and projection methods (Lin and Dunson, 2014; Cheung and Diaz, 2023). These methods address some of the issues associated with linear combination based methods, but some still require additional assumptions such as additivity and smoothness. In terms of computation, some of these methods can be formulated as a convex optimization problem that can be solved using existing libraries, e.g., CVXR in R (Fu, Narasimhan and Boyd, 2020) and CVXPY in Python (Diamond and Boyd, 2016). Using these functions require explicit specifications of the constraints, which can be complex as the dimension of biomarkers increases. Hence, most methods have been applied and demonstrated only in low-dimension situations (e.g., with 4 or fewer variables). An exception is a projective Bayes method introduced in Mander and Sweeting (2015) and Cheung, Chandereng and Diaz (2022) and subsequently discussed in Cheung and Diaz (2023) who propose algorithms to give approximate Bayes solution in higher-dimensional problems.

In Section 2, we will formally formulate the estimation problem, illustrate the problem with the HPV assay data, and introduce the projective Bayes method and the computation challenge. In Section 3, we will introduce a novel sequential update method based on the projective Bayes classifier for the estimation of the classification rule under monotonicity. We will propose practical algorithms to implement the estimator in Section 4. In Section 5, we will assess the proposed method’s computation feasibility and performance using simulation. We will revisit the study data in Kuhn et al. (2020) and examine the diagnostic accuracy of the HPV assay in Section 6. The articles ends with a discussion in Section 7 with technical details in a Web appendix.

2. Preliminaries and review of Bayes classifier under partial ordering.

2.1. Notation and partial ordering assumption.

To formally introduce the problem, let ${\mathit{x}}_k=\left(x_{k1},x_{k2},\dots ,x_{kD}\right)$ denote the biomarker panel $k$ for $k=1,\dots ,K$, where $x_{kd}$ is the value of the $d$th biomarker. The dimension $D$ indicates the number of biomarkers detected in the multiplex assay, and $K$ the number of unique values of biomarker combination in the data. Note that $K$ grows linearly with sample size $N$ when the biomarkers take on continuous values. Suppose there are $m_k$ subjects with biomarker panel ${\mathit{x}}_k$. Let $y_{ki}$ denote the outcome of subject $i$ among these $m_k$ subjects, and let ${\mathit{y}}_k=\left(y_{k1},y_{k2},\dots ,y_{k,m_k}\right)$ denote the vector that contains these $m_k$ outcomes. Each $y_{ki}$ follows a distribution indexed by parameter ${\theta }_k=\theta \left({\mathit{x}}_k\right)$, i.e., denoting ${\mathit{y}}_k~\prod _{i=1}^{m_k}f\left(\cdot \mid {\theta }_k\right)$. While the projective Bayes method is applicable to any general distribution $f\left(\cdot \mid {\theta }_k\right)$ of any outcome type, for clarity in exposition, we will describe and illustrate methods assuming the outcome $y_{ki}$ is binary based on the motivating example, i.e., $y_{ki}~\text{Binomial}\left(1,{\theta }_k\right)$ where ${\theta }_k=\text{Pr}\left(y_{ki}=1\mid {\mathit{x}}_k\right)$ is the disease probability for an individual with biomarker panel ${\mathit{x}}_k$.

We assume without loss of generality that $\theta \left(\mathit{x}\right)$ is nondecreasing in $\mathit{x}$ in terms of the partial Euclidean ordering ($\succ$), i.e.,

$$\text{if}{\mathit{x}}_k\succ {\mathit{x}}_{k^{\prime }},\text{then}{\theta }_k\ge {\theta }_{k^{\prime }},$$ (1)

where ${\mathit{x}}_k\succ {\mathit{x}}_{k^{\prime }}$ denotes the event that $x_{kd}\ge x_{k^{\prime }d}$ for each feature $d$ with at least one strict inequality. Note that assumption (1) also holds for any nondecreasing transformation of the biomarker value $x_{kd}$ and the parameter ${\theta }_k$.

2.2. Illustration of data structure.

Our dataset consists of HPV assay data in $N=928$ women. The assay detects HPV types expressed in $D=5$ channels: types 16, 18, 31, 39, and 51. Outputs from the assay measures the viral load of each type: the lowest possible value of each type is 0, and the maximum biomarker values observed in the data are 0.0505, 0.0494, 0.0581, 0.0510, and 0.0485 respectively for the five types. There are a total of $K=400$ unique combinations of HPV values, and Table 1 shows the data of 7 biomarker panels to illustrate the features of the data. First, we note that the HPV values are sparsely distributed: other than the panel with no detected viral load (${\mathit{x}}_1$ in the table), most panels have very few observations; in the entire dataset, 359 HPV panels (out of 400) have only one observation, i.e., $m_k=1$. This renders a completely nonparametric analysis untenable. Second, as we expect that a high viral load is associated with a positive biopsy (i.e., $y_{ki}=1$), data from different HPV panels may be ‘pooled’ due to the partial ordering assumption (1). For example, while ${\mathit{x}}_6\succ {\mathit{x}}_2$ in Table 1, the respective observed positive biopsy rates have the reverse order, suggesting some pooling may increase precision in estimation. In a broad sense, the goal of multivariate monotone regression is to pool these HPV panels with respect to constraints due to partial ordering in a principled way; this can be challenging as these constraints can be immense and intractable even with moderate dimension $D$ and sample size $N$.

Table 1.

A subset of HPV assay data. In this subset, the panels are partially ordered as follows: ${\mathit{x}}_k\succ {\mathit{x}}_1$ for all $k,{\mathit{x}}_6\succ {\mathit{x}}_2,{\mathit{x}}_7\succ {\mathit{x}}_2$ and ${\mathit{x}}_7\succ {\mathit{x}}_5$.

Panel	HPV panel ${\mathit{x}}_k$					$m_k$	${\sum }_{i=1}^{m_k}{y}_{ki}$
$k$	type 16	type 18	type 31	type 39	type 51
1	0	0	0	0	0	470	20
2	0	0	0	0.0260	0	3	1
3	0	0	0	0	0.0306	1	0
4	0	0.0246	0	0	0	1	1
5	0	0	0.0275	0.0246	0	1	1
6	0.0313	0	0	0.0284	0	1	0
7	0.0346	0	0.0326	0.0273	0	1	1

2.3. Unconstrained Bayes classifier.

Let ${\gamma }_k=I\left({\theta }_k>t_0\right)$ for a pre-specified threshold $t_0$ for disease. If no constraint is imposed on $\mathit{\gamma }=\left({\gamma }_1,{\gamma }_2,\dots ,{\gamma }_K\right)$, each ${\gamma }_k$ can be estimated by a Bayes estimator with respect to a classification-type gain function:

$$h_{ϵ}\left(g;{\gamma }_k\right)=ϵg{\gamma }_k+(1-ϵ)(1-g)\left(1-{\gamma }_k\right)\text{for}g=0,1,$$ (2)

where $ϵ\in (0,1)$ is pre-specified. The decision parameter $ϵ$ allows us to devise diagnostic test based on how we weigh true negative diagnosis and true positive diagnosis in a given application. Setting $ϵ<0.5$ implies $h_{ϵ}(1;1)<h_{ϵ}(0;0)$, i.e., giving greater gain to a true negative (yielding a negative diagnosis when in truth there is no disease), whereas setting $ϵ>0.5$ assigns greater gain to a true positive diagnosis. For example, if the disease is very rare among the population, one may choose $ϵ>0.5$ to avoid the trivial classifier that maps all biomarker values to a normal outcome (Bishop and Bishop, 2024). In our application, we will set $ϵ=0.5$, i.e., placing equal gain for the two correct decision types. Using standard computations, we can derive the Bayes estimator for ${\gamma }_k$ that maximizes the posterior gain by integrating (2) with respect to the posterior distribution of ${\theta }_k$. The unconstrained Bayes estimator has a closed form ${\hat{\gamma }}_k^B=I\left(p_k>1-ϵ\right)$, where $p_k=E\left({\gamma }_k\mid {\mathit{y}}_k\right)$ is the expectation of ${\gamma }_k$ taken with respect to the posterior distribution of ${\theta }_k$ given ${\mathit{y}}_k$ and it is easy to compute such as when using the beta-binomial conjugate prior on ${\theta }_k$. An unconstrained Bayes classifier thus maps the biomarker panel ${\mathit{x}}_k$ to test output ${\hat{\gamma }}_k^B$. This is a nonparametric estimator that allows ${\gamma }_k$ to be a free variable without any constraint.

2.4. Projective Bayes classifier under partial ordering.

Let ${\mathit{\gamma }}_{𝒮}=\left({\gamma }_k:k\in 𝒮\right)$ denote the true sub-classifier for a given subset $𝒮\subseteq \{1,\dots ,K\}$. Because $\gamma \left(\mathit{x}\right)$ is a nondecreasing transformation of $\theta \left(\mathit{x}\right)$, partial ordering (1) implies that ${\gamma }_{𝒮}$ lives in $\Gamma \left(𝒮\right)$ which is defined as

$$\Gamma (𝒮)=\{\mathit{g}\in \{0,1{\}}^{\left|𝒮\right|}:g_k\ge g_{k^{\prime }}\text{for}{\mathit{x}}_k\succ {\mathit{x}}_{k^{\prime }};k,k^{\prime }\in 𝒮\}.$$ (3)

Projective Bayes method estimates ${\mathit{\gamma }}_{𝒮}$ using data $\left\{\left({\mathit{x}}_k,{\mathit{y}}_k\right):k\in 𝒮\right\}$ by solving a constrained maximization problem:

$${\stackrel{\mathit{ˆ}}{\mathit{\gamma }}}_{𝒮}=\text{arg}\underset{\mathit{g}\in \Gamma \left(𝒮\right)}{\text{max}}{H}_{𝒮}(\mathit{g})$$ (4)

where

$$H_{𝒮}(\mathit{g})=\prod _{k\in 𝒮}{\left[{\left(ϵp_k\right)}^{g_k}{\left\{(1-ϵ)\left(1-p_k\right)\right\}}^{1-g_k}\right]}^{w_k}=\prod _{k\in 𝒮}{\varphi }_k^{g_k}{\rho }_k^{1-g_k}$$ (5)

with ${\varphi }_k={\left(ϵp_k\right)}^{w_k}$ and ${\rho }_k={\left\{(1-ϵ)\left(1-p_k\right)\right\}}^{w_k}$. The objective function $H_{𝒮}\left(\mathit{g}\right)$ is a weighted product of posterior gains with respect to (2) over $k\in 𝒮$, with weight $w_k>0$ defined to reflect the information content about ${\gamma }_k$ in the data, e.g., setting $w_k=m_k$; see Proposition 1 in Cheung and Diaz (2023). The function is maximized by the unconstrained Bayes classifier ${\hat{\gamma }}_{𝒮}^B=\left({\hat{\gamma }}_k^B:k\in 𝒮\right)$ on the unconstrained space $\{0,1{\}}^{\left|𝒮\right|}$. Thus, the maximizer ${\hat{\gamma }}_{𝒮}$ of (4) takes on the value of the unconstrained Bayes estimator if ${\hat{\gamma }}_{𝒮}^B\in \Gamma \left(𝒮\right)$. Generally, the estimator ${\hat{\mathit{\gamma }}}_{𝒮}$ is a projection of ${\hat{\mathit{\gamma }}}_{𝒮}^B$ on the partial ordering subspace $\Gamma \left(𝒮\right)$. This estimator imposes partial ordering of the HPV panel ${\mathit{x}}_k$ by mapping it to the output ${\hat{\gamma }}_k$ according to the constraint (3), but makes no additional assumption.

When the cardinality $\left|𝒮\right|$ is small, one may evaluate ${\stackrel{\mathit{ˆ}}{\mathit{\gamma }}}_{𝒮}$ by enumerating $\mathit{g}\in \{0,1{\}}^{\left|𝒮\right|}$, checking if $\mathit{g}\in \Gamma \left(𝒮\right)$, and if so saving the value of $H_{𝒮}\left(\mathit{g}\right)$. Brute force enumeration becomes quickly infeasible as $\left|𝒮\right|$ increases. In particular, the number of operations involved in the evaluation of $\hat{\gamma }≔{\hat{\gamma }}_{\{1,\dots ,K\}}$ using full data is of the order $O\left(2^K\right)$. To address computation when $K$ is large, Cheung and Diaz (2023) propose a sequential random subset maximization algorithm (SRSM). The idea of SRSM is to reduce the original maximization problem (4) into smaller computational tasks defined by sequentially and randomly selected subsets. Briefly, SRSM starts by selecting a subset ${𝒮}_1\subset 𝒮$, evaluating ${\hat{\mathit{\gamma }}}_{{𝒮}_1}$, and imputing ${\hat{\gamma }}_k$ for $k\in {ℳ}_1\setminus {𝒮}_1$ where ${ℳ}_1$ is the index set for which the value of ${\hat{\gamma }}_k$ is implied by ${\hat{\mathit{\gamma }}}_{{𝒮}_1}$ per partial ordering. It continues by selecting another set ${𝒮}_2\subset 𝒮\setminus \left({𝒮}_1\cup {ℳ}_1\right)$, evaluating ${\hat{\mathit{\gamma }}}_{{𝒮}_2}$, and imputing ${\hat{\gamma }}_k$ for $k\in {ℳ}_2\setminus \left({𝒮}_1\cup {ℳ}_1\cup {𝒮}_2\right)$ where ${ℳ}_2$ is the index set for which ${\hat{\gamma }}_k$ is implied per partial ordering. The algorithm proceeds in a similar manner until ${\hat{\gamma }}_k$ is determined for all $k$. The resulted classifier ${\hat{\mathit{\gamma }}}^{\mathit{\text{SRSM}}}$ is formed by stacking ${\hat{\mathit{\gamma }}}_{{𝒮}_1},{\hat{\mathit{\gamma }}}_{{ℳ}_1},{\hat{\mathit{\gamma }}}_{{𝒮}_2},{\hat{\mathit{\gamma }}}_{{ℳ}_2},\dots$ To ensure computation is feasible, the selected subsets are small, e.g., with $\left|{𝒮}_1\right|=5$. As there is no theoretical guarantee that the resulted classifier will be the same as the true maximizer ${\hat{\mathit{\gamma }}}_{𝒮}$, we may repeat SRSM $N_{\mathit{\text{rep}}}$ times and select ${\hat{\mathit{\gamma }}}^{\mathit{\text{SRSM}}}$ with the largest $H_{𝒮}(\cdot )$ among all replicates. Running repeated SRSM may only give an approximate Bayes solution. While SRSM will find the maximizer ${\hat{\mathit{\gamma }}}_{𝒮}$ as the number of replicates increases, there is no theoretical guidance on how large $N_{\mathit{\text{rep}}}$ should be. Also, the computation can still be burdensome in practice especially with a large $N_{\mathit{\text{rep}}}$. Cheung and Diaz (2023) consider a scenario with dimension $D=12$ and $K>3000$, where it would take hours to run SRSM with $N_{\mathit{\text{rep}}}=50$.

Overall, while the projective Bayes classifier $\hat{\mathit{\gamma }}$ is a principled nonparametric estimator for the disease classification rule $\gamma \left(\mathit{x}\right)$, efficient computation approaches for the optimization problem (4) are needed to render the estimator feasible and broadly applicable in practice.

3. A sequential update estimation method.

We consider estimating the disease classification rule $\gamma \left(\mathit{x}\right)$ using the projective Bayes classifier $\hat{\mathit{\gamma }}={\hat{\mathit{\gamma }}}_{\{1,\dots ,K\}}$ with the full data set $\left\{\left({\mathit{x}}_k,{\mathit{y}}_k\right):k=1,\dots ,K\right\}$. To avoid brute force computation of $\stackrel{\mathit{ˆ}}{\mathit{\gamma }}$, we approach this problem as a sequential update problem: Suppose we know the projective Bayes sub-classifier ${\hat{\mathit{\gamma }}}_{𝒮}$ for some $𝒮\subset \{1,\dots ,K\}$; then we seek to evaluate ${\hat{\mathit{\gamma }}}_{{𝒮}^{*}}$ where ${𝒮}^{*}=𝒮\cup \left\{i\right\}$ for some $i\notin 𝒮$ by updating ${\hat{\mathit{\gamma }}}_{𝒮}$ with “new” data ($x_i,y_i$). Algorithm 1 outlines a general sequential update algorithm that operationalizes this idea.

Algorithm 1.

Sequential update method to estimate $\mathit{\gamma }$ using projective Bayes $\hat{\mathit{\gamma }}$

1. Initiate $𝒮=\left\{1,\dots ,K_0\right\}$ for some $K_0\le K$ and solve ${\stackrel{\mathit{ˆ}}{\mathit{\gamma }}}_{𝒮}$ per (4).

2. for $i=K_0+1$ to $K$ do

a) set ${𝒮}^{*}=𝒮\cup \left\{i\right\}$

b) evaluate ${\stackrel{\mathit{ˆ}}{\mathit{\gamma }}}_{𝒮*}$ by updating ${\stackrel{\mathit{ˆ}}{\mathit{\gamma }}}_{𝒮}$ with new data $\left(x_i,y_i\right)$

c) $𝒮\leftarrow {𝒮}^{*}$ and ${\hat{\mathit{\gamma }}}_{𝒮}\leftarrow {\hat{\mathit{\gamma }}}_{{𝒮}^{*}}$

We can initiate Algorithm 1 with a small $K_0$ (e.g., 10) so brute force enumeration is feasible in step 1. The practicality of the algorithm then relies on the key that the updating step (step 2b) will be computationally tractable. Updating is indeed very simple when the data meet certain conditions:

Theorem 3.1. If the stacked vector $\left({\hat{\mathit{\gamma }}}_{𝒮},{\hat{\gamma }}_i^B\right)\in \Gamma \left({𝒮}^{*}\right)$, then ${\hat{\mathit{\gamma }}}_{{𝒮}^{*}}=\left({\hat{\mathit{\gamma }}}_{𝒮},{\hat{\gamma }}_i^B\right)$.

Theorem 3.1 is quite intuitive in that ${\hat{\mathit{\gamma }}}_{{𝒮}^{*}}$ can be formed by simply stacking the current classifier ${\hat{\mathit{\gamma }}}_{𝒮}$ and the unconstrained Bayes estimator ${\hat{\gamma }}_i^B$ when they concur in terms of partial ordering. As a special case, if the unconstrained Bayes classifier ${\hat{\mathit{\gamma }}}^B\in \Gamma (\{1,\dots ,K\})≔\Gamma$ (i.e., having no violators in the data such as in a perfectly accurate assay), Algorithm 1 will be sequentially stacking individual ${\hat{\gamma }}_i^B$, and the number of operations will grow with $K$ linearly, as opposed to exponentially.

To deal with situations when $\left({\hat{\mathit{\gamma }}}_{𝒮},{\hat{\gamma }}_i^B\right)\notin \Gamma \left({𝒮}^{*}\right)$, i.e., when the condition in Theorem 3.1 is not met, we further define

$${\Gamma }_{il}=\left\{\mathit{g}\in \Gamma \left({𝒮}^{*}\right):g_i=l\right\}\text{and}{\stackrel{\mathit{ˆ}}{\mathit{\gamma }}}_{il}^{*}=\text{arg}\underset{\mathit{g}\in {\Gamma }_{il}}{\text{max}}{H}_{{𝒮}^{*}}\left(\mathit{g}\right)\text{for}l=0,1.$$ (6)

Since ${\Gamma }_{i0}\cup {\Gamma }_{i1}=\Gamma \left({𝒮}^{*}\right)$, the overall maximizer ${\hat{\mathit{\gamma }}}_{{𝒮}^{*}}$ will either be ${\hat{\mathit{\gamma }}}_{i0}^{*}$ or ${\hat{\mathit{\gamma }}}_{i1}^{*}$, the respective maximizers over subspaces ${\Gamma }_{i0}$ and ${\Gamma }_{i1}$ per (6). More precisely, we have

$${\hat{\mathit{\gamma }}}_{{𝒮}^{*}}={\hat{\mathit{\gamma }}}_{il}^{*}\text{if}{H}_{{𝒮}^{*}}\left({\hat{\mathit{\gamma }}}_{il}^{*}\right)>H_{{𝒮}^{*}}\left({\hat{\mathit{\gamma }}}_{i,1-l}^{*}\right).$$

Next, we will describe how to evaluate ${\hat{\mathit{\gamma }}}_{i0}^{*}$ and ${\hat{\mathit{\gamma }}}_{i1}^{*}$ by updating the current ${\hat{\mathit{\gamma }}}_{𝒮}$. We consider two ways to partition the current set $𝒮$ for a given $i\notin 𝒮$. First, we have $𝒮={ℒ}_i\cup {𝒰}_i\cup {ℰ}_i$ where

$${ℒ}_i=\left\{k\in 𝒮:{\mathit{x}}_k\prec {\mathit{x}}_i\right\},{𝒰}_i=\left\{k\in 𝒮:{\mathit{x}}_k\succ {\mathit{x}}_i\right\},\text{and}{ℰ}_i=𝒮\setminus \left({ℒ}_i\cup {𝒰}_i\right).$$

Second, let ${\hat{\gamma }}_k$ denote the $k$th element of the current classifier ${\hat{\mathit{\gamma }}}_{𝒮}$ and define

$${𝒮}_l=\left\{k\in 𝒮:{\hat{\gamma }}_k=l\right\}\text{for}l=0,1.$$

Further let ${\hat{\gamma }}_{ilk}^{*}$ denote the $k$th element of ${\hat{\mathit{\gamma }}}_{il}^{*}$ for $k\in {𝒮}^{*}$.

Theorem 3.2. Suppose $\left({\hat{\mathit{\gamma }}}_{𝒮},{\hat{\gamma }}_i^B\right)\notin \Gamma \left({𝒮}^{*}\right)$ and ${\hat{\gamma }}_i^B=1$. Then ${\hat{\mathit{\gamma }}}_{i0}^{*}$ and ${\hat{\mathit{\gamma }}}_{i1}^{*}$ can be evaluated as follows: (a) ${\hat{\mathit{\gamma }}}_{i0}^{*}=\left({\hat{\mathit{\gamma }}}_{𝒮},0\right)\in \Gamma \left({𝒮}^{*}\right)$, and (b) ${\hat{\mathit{\gamma }}}_{i1}^{*}=\left({\hat{\mathit{\gamma }}}_{{\widetilde{𝒮}}_0},{\mathbf{1}}_{{𝒮}_1\cup {𝒰}_i\cup \left\{i\right\}}\right)$, i.e., setting $\left\{{\hat{\mathit{\gamma }}}_{i1k}^{*}:k\in {\widetilde{𝒮}}_0\right\}$ to be equal to ${\hat{\mathit{\gamma }}}_{{\widetilde{𝒮}}_0}$ and ${\hat{\gamma }}_{i1k}^{*}=1$ for $k\in {𝒮}_1\cup {𝒰}_i\cup \left\{i\right\}$, where ${\widetilde{𝒮}}_0={𝒮}_0\cap \left({ℒ}_i\cup {ℰ}_i\right)$. The dependence of ${\widetilde{𝒮}}_0$ on $i$ is suppressed for brevity in notation.

Theorem 3.2 prescribes how ${\hat{\mathit{\gamma }}}_{i0}^{*}$ and ${\hat{\mathit{\gamma }}}_{i1}^{*}$ may be evaluated given the current ${\hat{\mathit{\gamma }}}_{𝒮}$. Specifically, the update involves maximization over a smaller subspace $\Gamma \left({\widetilde{𝒮}}_0\right)$ than $\Gamma$. This underlies the advantage of formulating the computation of $\hat{\mathit{\gamma }}$ as a sequential update problem.

Theorem 3.3. Suppose $\left({\hat{\mathit{\gamma }}}_{𝒮},{\hat{\gamma }}_i^B\right)\notin \Gamma \left({𝒮}^{*}\right)$ and ${\hat{\gamma }}_i^B=0$. Then ${\hat{\mathit{\gamma }}}_{i0}^{*}$ and ${\hat{\mathit{\gamma }}}_{i1}^{*}$ can be evaluated as follows: (a) ${\hat{\mathit{\gamma }}}_{i1}^{*}=\left({\hat{\mathit{\gamma }}}_{𝒮},1\right)\in \Gamma \left({𝒮}^{*}\right)$, and (b) ${\hat{\mathit{\gamma }}}_{i0}^{*}=\left({\hat{\mathit{\gamma }}}_{{\widetilde{𝒮}}_1},{\mathbf{0}}_{{𝒮}_0\cup {ℒ}_i\cup \left\{i\right\}}\right)$, i.e., setting $\left\{{\hat{\gamma }}_{i0k}^{*}:k\in {\widetilde{𝒮}}_1\right\}$ to be equal to ${\hat{\mathit{\gamma }}}_{{\widetilde{𝒮}}_1}$ and ${\hat{\gamma }}_{i0k}^{*}=0$ for $k\in {𝒮}_0\cup {ℒ}_i\cup \left\{i\right\}$ where ${\widetilde{𝒮}}_1={𝒮}_1\cap \left({𝒰}_i\cup {ℰ}_i\right)$. The dependence of ${\widetilde{𝒮}}_1$ on $i$ is suppressed for brevity in notation.

Theorem 3.3, together with Theorems 3.1 and 3.2, cover all possibilities how step 2b in Algorithm 1 may be implemented. Proofs of the theorems are provided as Supplementary Material in a Web Appendix.

4. Computational algorithms.

4.1. Comb-down algorithm.

For the sequential update estimation method to be practical, Algorithm 1 needs to be fine-tuned by leveraging the results of Theorems 3.1–3.3 effectively. First, it is advantageous to keep ${\widetilde{𝒮}}_0$ and ${\widetilde{𝒮}}_1$ small in each iteration, or to avoid computing ${\hat{\mathit{\gamma }}}_{{\widetilde{𝒮}}_0}$ and ${\hat{\mathit{\gamma }}}_{{\widetilde{𝒮}}_1}$ altogether. This may be accomplished with a comb-down implementation of Algorithm 1 whereby we pre-arrange the labels $1,\dots ,K$ such that $\left\{\sum _d{x}_{kd}:k=1,\dots ,K\right\}$ is a nonincreasing sequence, i.e., having

$$\sum _{d=1}^D{x}_{1d}\ge \sum _{d=1}^D{x}_{2d}\ge \cdots \ge \sum _{d=1}^D{x}_{Kd}$$ (7)

and secondarily $p_k$ in descending order, i.e.,

$$p_k\ge p_{k+1}\ge \cdots p_{k+l}\text{on the set}\left\{\sum _{d=1}^D{x}_{kd}=\sum _{d=1}^D{x}_{k+1,d}=\cdots \sum _{d=1}^D{x}_{k+l,d}\right\}.$$ (8)

A comb-down algorithm solves and updates ${\hat{\gamma }}_1,{\hat{\gamma }}_2,\dots ,{\hat{\gamma }}_K$ sequentially by combing through the biomarker panel ${\mathit{x}}_k$ per the descending order of $\sum _d{x}_{kd}$. An implication of rearranging labels per (7) and (8) is that $x_i\nsucc x_k$ for $k\in 𝒮$ in each iteration, i.e., ${ℒ}_i=\varnothing$. Consequently, when ${\hat{\gamma }}_i^B=0$, the condition in Theorem 3.1 will be met and the update in step 2b can be quickly achieved by stacking ${\hat{\mathit{\gamma }}}_{𝒮}$ and ${\hat{\gamma }}_i^B$. When ${\hat{\gamma }}_i^B=1$ and the condition of Theorem 3.1 does not hold, the main computation task ${\stackrel{\mathit{ˆ}}{\mathit{\gamma }}}_{{\widetilde{𝒮}}_0}$ will be evaluated over a reduced subspace as a result of ${ℒ}_i=\varnothing$, i.e., having ${\widetilde{𝒮}}_0={𝒮}_0\cap \left({ℒ}_i\cup {ℰ}_i\right)={𝒮}_0\cap {ℰ}_i$. Algorithm 2 details the steps in a comb-down algorithm.

Algorithm 2.

Comb-down algorithm to evaluate $\stackrel{\mathit{ˆ}}{\mathit{\gamma }}$

0. Rearrange the labels $1,\dots ,K$ with descending $\sum _d{x}_{kd}$ per (7) and (8)

1. Initiate $𝒮=\left\{1,\dots ,K_0\right\}$ for some $K_0\le K$ and solve ${\stackrel{\mathit{ˆ}}{\mathit{\gamma }}}_{𝒮}$ per (4)

2. for $i=K_0+1$ to $K$ do

a) set ${𝒮}^{*}=𝒮\cup \left\{i\right\}$

b) if $\left({\stackrel{\mathit{ˆ}}{\mathit{\gamma }}}_{𝒮},{\hat{\gamma }}_i^B\right)\in \Gamma \left({𝒮}^{*}\right)$, set ${\stackrel{\mathit{ˆ}}{\mathit{\gamma }}}_{{𝒮}^{*}}=\left({\stackrel{\mathit{ˆ}}{\mathit{\gamma }}}_{𝒮},{\hat{\gamma }}_i^B\right)$; else set ${\hat{\mathit{\gamma }}}_{i0}^{*}=\left({\hat{\mathit{\gamma }}}_{𝒮},0\right)$ and ${\stackrel{\mathit{ˆ}}{\mathit{\gamma }}}_{i1}^{*}=\left({\stackrel{\mathit{ˆ}}{\mathit{\gamma }}}_{{\widetilde{𝒮}}_0},1_{{𝒮}_1\cup {𝒰}_i\cup \left\{i\right\}}\right)$

c) $𝒮\leftarrow {𝒮}^{*}$ and ${\hat{\gamma }}_{𝒮}\leftarrow {\hat{\gamma }}_{{𝒮}^{*}}$

To further reduce the computational task in step 2b of Algorithm 2, we introduce the extension of ${ℒ}_i$ and ${𝒰}_i$. For a given subset $ℐ\subseteq \{1,\dots ,K\}$, let

$$ℒ\left(ℐ\right)=\left\{k\in 𝒮:{\mathit{x}}_k\prec {\mathit{x}}_j\text{for some}j\in ℐ\right\}\text{and}𝒰\left(ℐ\right)=\left\{k\in 𝒮:{\mathit{x}}_k\succ {\mathit{x}}_j\text{for some}j\in ℐ\right\}.$$

For a given $i$ and an initial set ${ℐ}_0={𝒮}_0\setminus {\widetilde{𝒮}}_0={𝒮}_0\cap {𝒰}_i$, we define the recursion

$${ℐ}_r=\left\{{ℐ}_{r-1}\cup ℒ\left({ℐ}_{r-1}\right)\cup 𝒰\left({ℐ}_{r-1}\right)\right\}\cap {\widetilde{𝒮}}_0\text{for positive integer}r.$$ (9)

Theorem 4.1 (Refinement of Theorem 3.2). In step 2b of the comb-down Algorithm 2, we can set ${\hat{\mathit{\gamma }}}_{{\widetilde{𝒮}}_0}=\left({\hat{\mathit{\gamma }}}_{{ℐ}_{\infty }},{\mathbf{0}}_{{\widetilde{𝒮}}_0\setminus {ℐ}_{\infty }}\right)$ where ${ℐ}_{\infty }={\text{lim}}_{r\to \infty }{ℐ}_r$.

Because of how ${ℐ}_r$ is defined per (9), the limit ${ℐ}_{\infty }\subseteq {\widetilde{𝒮}}_0$. That is, the computation of ${\hat{\mathit{\gamma }}}_{{\widetilde{𝒮}}_0}$ can be accomplished with a further reduced computation task ${\hat{\mathit{\gamma }}}_{{ℐ}_{\infty }}$ per Theorem 4.1.

4.2. Comb-up algorithm.

Analogous to the comb-down algorithm, we can define a comb-up algorithm that solves and updates ${\hat{\gamma }}_1,\dots ,{\hat{\gamma }}_K$ sequentially by combing through the biomarker panel per an ascending $\left\{\sum _d{x}_{kd}\right\}$. Precisely, we first rearrange the labels $1,\dots ,K$ such that

$$\sum _{d=1}^D{x}_{1d}\le \sum _{d=1}^D{x}_{2d}\le \cdots \le \sum _{d=1}^D{x}_{Kd}$$ (10)

and secondarily $p_k$ in ascending order, i.e.,

$$p_k\le p_{k+1}\le \cdots p_{k+l}\text{on}\text{the}\text{set}\left\{\sum _{d=1}^D{x}_{kd}=\sum _{d=1}^D{x}_{k+1,d}=\cdots \sum _{d=1}^D{x}_{k+l,d}\right\}.$$ (11)

The comb-up algorithm is outlined detailed in Algorithm 3 below. Refinement analogous to Theorem 4.1 can also be derived for the comb-up algorithm.

Algorithm 3.

Comb-up algorithm to evaluate $\hat{\mathit{\gamma }}$

0. Rearrange the labels $1,\dots ,K$ with ascending $\sum _d{x}_{kd}$ per (10) and (11)

1. Initiate $𝒮=\left\{1,\dots ,K_0\right\}$ for some $K_0\le K$ and solve ${\hat{\mathit{\gamma }}}_{𝒮}$ per (4)

2. for $i=K_0+1$ to $K$ do

a) set ${𝒮}^{*}=𝒮\cup \left\{i\right\}$

b) if $\left({\stackrel{\mathit{ˆ}}{\mathit{\gamma }}}_{𝒮},{\hat{\gamma }}_i^B\right)\in \Gamma \left({𝒮}^{*}\right)$, set ${\stackrel{\mathit{ˆ}}{\mathit{\gamma }}}_{{𝒮}^{*}}=\left({\stackrel{\mathit{ˆ}}{\mathit{\gamma }}}_{𝒮},{\hat{\gamma }}_i^B\right)$; else set ${\stackrel{\mathit{ˆ}}{\mathit{\gamma }}}_{i1}^{*}=\left({\stackrel{\mathit{ˆ}}{\mathit{\gamma }}}_{𝒮},1\right)$ and ${\stackrel{\mathit{ˆ}}{\mathit{\gamma }}}_{i0}^{*}=\left({\stackrel{\mathit{ˆ}}{\mathit{\gamma }}}_{{\widetilde{𝒮}}_1},{\mathbf{0}}_{{𝒮}_0\cup {ℒ}_i\cup \left\{i\right\}}\right)$

c) $𝒮\leftarrow {𝒮}^{*}$ and ${\hat{\gamma }}_{𝒮}\leftarrow {\hat{\gamma }}_{{𝒮}^{*}}$

4.3. Recursive comb.

In the comb-down algorithm, the set ${\widetilde{𝒮}}_0$ can be arbitrarily large in a data set rendering a single use of Algorithm 2 infeasible. For the sequential update method to be broadly applicable, we may apply the algorithm recursively. Specifically, a recursive comb-down (DD) algorithm will call Algorithm 2 recursively to evaluate ${\hat{\mathit{\gamma }}}_{{\widetilde{𝒮}}_0}$ when needed; see Algorithm 4 below.

In a similar manner to recursive comb-down (DD), a recursive comb-up (UU) algorithm is based on Algorithm 3 and will call itself recursively to evaluate ${\stackrel{\mathit{ˆ}}{\mathit{\gamma }}}_{{\widetilde{𝒮}}_1}$ when needed. In fact, the single-use comb-down and comb-up algorithms can each serve as the building blocks of recursive algorithms. Specifically, a recursive down-and-up (DU) algorithm starts with Algorithm 2 and then alternates between Algorithm 3 to evaluate ${\stackrel{\mathit{ˆ}}{\mathit{\gamma }}}_{{\widetilde{𝒮}}_0}$ and Algorithm 2 to evaluate ${\stackrel{\mathit{ˆ}}{\mathit{\gamma }}}_{{\widetilde{𝒮}}_1}$. Likewise, a recursive up-and-down (UD) algorithm starts with Algorithm 3 and then alternates between Algorithm 2 and Algorithm 3.

Algorithm 4.

Recursive comb-down (DD) algorithm to evaluate $\hat{\mathit{\gamma }}$

0. Rearrange the labels $1,\dots ,K$ with descending $\sum _d{x}_{kd}$ per (7) and (8)

1. Initiate $𝒮=\left\{1,\dots ,K_0\right\}$ for some $K_0\le K$ and solve ${\hat{\mathit{\gamma }}}_{𝒮}$ per (4)

2. for $i=K_0+1$ to $K$ do

a) set ${𝒮}^{*}=𝒮\cup \left\{i\right\}$

b) if $\left({\stackrel{\mathit{ˆ}}{\mathit{\gamma }}}_{𝒮},{\hat{\gamma }}_i^B\right)\in \Gamma \left({𝒮}^{*}\right)$, set ${\stackrel{\mathit{ˆ}}{\mathit{\gamma }}}_{{𝒮}^{*}}=\left({\stackrel{\mathit{ˆ}}{\mathit{\gamma }}}_{𝒮},{\hat{\gamma }}_i^B\right)$; else set ${\stackrel{\mathit{ˆ}}{\mathit{\gamma }}}_{i0}^{*}=\left({\stackrel{\mathit{ˆ}}{\mathit{\gamma }}}_{𝒮},0\right)$ and ${\stackrel{\mathit{ˆ}}{\mathit{\gamma }}}_{i1}^{*}=\left({\stackrel{\mathit{ˆ}}{\mathit{\gamma }}}_{{\widetilde{𝒮}}_0},1_{{𝒮}_1\cup {𝒰}_i\cup \left\{i\right\}}\right)$ by calling Algorithm 4 to evaluate ${\stackrel{\mathit{ˆ}}{\mathit{\gamma }}}_{{\widetilde{𝒮}}_0}$

c) $𝒮\leftarrow {𝒮}^{*}$ and ${\hat{\gamma }}_{𝒮}\leftarrow {\hat{\gamma }}_{{𝒮}^{*}}$

5. Simulation studies.

5.1. Simulation setting 1: Comparison of recursive algorithms.

We run simulations to examine the computation feasibility of the proposed recursive algorithms and compare them with the SRSM algorithm. We consider a setting with binary outcomes with $D=7$ ordinal biomarkers. Briefly, four of the 7 biomarkers take on the value of 0, 1, or 2, and three take on 0 or 1, thus resulting in a total of $K=3^4\times 2^3=648$ possible combinations of biomarker panel values. A simulated data set has a sample size of 10,000. For a given biomarker panel ${\mathit{x}}_k$, we draw $y_{ki}$ from a Bernoulli distribution with probability ${\theta }_k$ for $i=1,\dots ,m_k$ where $\left\{m_k:k=1,\dots ,648\right\}$ is multinomial with size 10,000 and equal probabilities, and

$$\text{logit}\left({\theta }_k\right)=-5+1.5{\left(\sum _{d=1}^7{x}_{kd}\right)}^{1/2}+0.75x_{k1}^2+0.5x_{k2}{x}_{k3}.$$

This is the data generation model considered in Cheung and Diaz (2023). The algorithms are implemented with a uniform(0,1) prior for each ${\theta }_k$ so that the individual Bayes classifier ${\hat{\gamma }}_k^B$ can be quickly computed based on conjugacy. We initiate the proposed recursive algorithms with $K_0=10$. For the SRSM algorithm, we select random subsets of size 5, i.e., $\left|{𝒮}_1\right|=5$; and we repeat the algorithm $N_{\mathit{\text{rep}}}=100$ times. Table 2 summarizes the elapsed times of the five algorithms based on 100 simulated data sets. The proposed recursive algorithms have shorter average elapsed time than SRSM on average. The results also suggest that alternating between comb-down and comb-up in the recursive calls not only reduces the average time but it also reduces the variability across data sets. Specifically, while recursive comb-down (DD) has a smaller median elapsed time than SRSM (8.9 seconds vs 25 seconds), it takes a long time to complete the task for some data sets resulting in a relatively large mean and a large standard deviation. Similar observations, perhaps to a lesser extent, are also noted for the recursive comb-up (UU) which has the shortest median elapsed time of 0.76 seconds among all methods. In contrast, the recursive down-and-up (DU) and up-and-down (UD) algorithms have the smallest means and standard deviations among all recursive algorithms.

Table 2.

Comparison of the elapsed time (in seconds) of five algorithms

Algorithm	Mean	SD	Median	IQR
Recursive comb-down (DD)	20.8	25.0	8.9	3.7 – 22.2
Recursive comb-up (UU)	6.1	18.4	0.76	0.37 – 1.35
Recursive down-and-up (DU)	4.9	2.4	5.1	2.5 – 6.5
Recursive up-and-down (UD)	4.7	3.4	3.7	2.5 – 6.3
SRSM	24.7	1.5	25.0	24.2 – 25.9

Finally, while all four recursive algorithms give the exact Bayes solution $\hat{\gamma }$ that maximizes the posterior gain (4), SRSM gives an approximation and is in fact correct in only 31% of the simulated data sets.

5.2. Simulation setting 2: The effects of $D$ and $K$ with binary biomarkers.

In the second simulation study, we consider settings where the assay detects $D$ binary biomarkers, i.e., $x_{kd}\in \{0,1\}$ for each $d=1,\dots ,D$, where $D=8,9,\dots ,13$, with total sample size $N=2000$, 4000 or 8000. The first goal of this simulation study is to examine the computation feasibility of the projective Bayes classifier by the recursive DU algorithm and the SRSM algorithm as the assay dimension $D$ grows. With $2^D$ possible biomarker combinations, the number $K$ of unique biomarker combinations in the sample will grow exponentially with $D$.

In the simulation, we first generate $m_k’\text{s}$ from a multinomial distribution with size $N$ and probability $1/2^D$ for each $k=1,\dots ,2^D$ and keep $m_k’\text{s}$ fixed across simulation replicates. In each replicate, for biomarker panel ${\mathit{x}}_k$, we draw $y_{ki}$ from a Bernoulli distribution with probability ${\theta }_k=\theta \left({\mathit{x}}_k\right)$ for $i=1,\dots ,m_k$, where

$${\theta }_k=\left\{\begin{array}{l}0.1\text{if}{\sum }_{d=4}^D{x}_{kd}<0.3,\\ 0.7\text{if}0.7\le {\sum }_{d=4}^D{x}_{kd}<0.8,\\ 0.8\text{if}{\sum }_{d=4}^D{x}_{kd}\ge 0.8.\end{array}\right.$$ (12)

The DU algorithm and SRSM are implemented with a uniform(0, 1) prior for each ${\theta }_k$. We set $K_0=10$ to initiate the DU algorithm. For the SRSM algorithm, we select random subsets of size 5 and repeat the algorithm $N_{\mathit{\text{rep}}}=50$ times.

Table 3 shows the elapsed times of the two algorithms under different $D,N$, and $K$. The DU algorithm is consistently faster than SRSM, and the difference increases substantially as $D$ increases. In the scenario when $D=13,N=8000$, and $K=5095$, SRSM takes almost half an hour on average to compute the classifier while the DU algorithms takes less than 2 minutes on average on the same computer. While the mean elapsed time of both algorithms increases as $K$ increases, Figure 1 suggests that the mean elapsed time of DU grows linearly with $K$, whereas that of SRSM grows at a faster rate than linear.

Table 3.

Comparison of the elapsed time (in seconds) of DU and SRSM algorithms under data generation model (12) for different assay dimension $D$ and sample size $N$

$D$	$N$	$K$	$N/K$	SRSM mean (SD)	DU mean (SD)
8	2000	256	7.8	4.8 (0.20)	0.51 (0.53)
	4000	256	15.6	4.9 (0.14)	0.20 (0.01)
	8000	256	31.3	4.9 (0.13)	0.20 (0.01)
9	2000	502	4.0	10.2 (0.34)	4.2 (1.6)
	4000	512	7.8	11.4 (0.21)	0.66 (0.46)
	8000	512	15.6	12.4 (0.26)	0.48 (0.02)
10	2000	879	2.3	22.2 (1.0)	10.7 (3.8)
	4000	1002	4.0	30.7 (0.75)	5.9 (3.0)
	8000	1024	7.8	35.2 (0.79)	1.6 (0.57)
11	2000	1285	1.6	42.1 (1.6)	19.7 (5.9)
	4000	1767	2.3	97.3 (2.6)	28.4 (4.8)
	8000	2002	4.0	160 (4.9)	20.7 (6.4)
12	2000	1577	1.3	60.7 (1.7)	12.9 (3.4)
	4000	2559	1.6	203 (12.6)	50.3 (13.7)
	8000	3516	2.3	719 (70.1)	74.0 (15.9)
13	2000	1774	1.1	81.0 (2.8)	12.6 (4.2)
	4000	3138	1.3	294 (8.7)	67.1 (39.1)
	8000	5095	1.6	1634 (176)	95.0 (25.6)

Fig 1.

Mean elapsed time (in seconds) of SRSM and DU algorithms against $K$ in simulation setting 2. The dotted line is the least squares fit of linear regression of mean elapsed time on $K$.

The second goal of the simulation study is to compare the accuracy of the approximate Bayes solution (SRSM) and the exact solution (DU) of the projective Bayes classifier, along with other methods. To define a performance metric, we note that the classification error rate for future data due to a classifier $\mathit{g}=\left(g_1,\dots ,g_K\right)$ can be evaluated as

$$\mathit{\text{Err}}(\mathit{g})=\frac{1}{K}\left[\sum _{k=1}^K{m}_k\left\{g_k\left(1-{\theta }_k\right)+\left(1-g_k\right){\theta }_k\right\}\right]\text{for}{g}_k=0,1.$$ (13)

It is easy to verify that $\text{Err}\left(\mathit{g}\right)$ attains its minimum when $g_k={\gamma }_k=I\left({\theta }_k>0.5\right)$. Therefore, while $\mathit{\gamma }=\left({\gamma }_1,\dots ,{\gamma }_K\right)$ depends on the unknown probability ${\theta }_k$, it may serve as a benchmark classifier in the simulation study. Correspondingly, we will set $g_k={\hat{\gamma }}_k$ with $t_0=0.5$ for the projective Bayes classifier, where ${\hat{\gamma }}_k’\text{s}$ are obtained either by DU or SRSM. We will calculate error ratio $Err\left(\mathit{\gamma }\right)/Err\left(\stackrel{\mathit{ˆ}}{\mathit{\gamma }}\right)$ for each simulated trial, and the relative efficiency of $\stackrel{\mathit{ˆ}}{\mathit{\gamma }}$ with respect to the benchmark $\mathit{\gamma }$ will be estimated by averaging the error ratio across simulation replicates.

For comparison purposes, we will include a classification rule based on additive logistic regression fitted value ${\tilde{\theta }}_k$ for ${\theta }_k$, i.e., setting $g_k=I\left({\tilde{\theta }}_k>0.5\right)$ and a nonparametric rule that based on the observed disease proportion of each biomarker panel ${\mathit{x}}_k$ in the data, i.e., setting $g_k=I\left(\sum _{i=1}^{m_k}{y}_{ki}/m_k>0.5\right)$. Additionally, we will consider the unconstrained Bayes classifier, i.e., setting $g_k={\hat{\gamma }}_k^B$, which is another nonparametric approach to estimate ${\gamma }_k$; see Section 2.3.

Figure 2 displays the relative efficiency of these five classifiers. The approximate projective Bayes by SRSM has consistently lower efficiency compared to the exact Bayes solution by DU, and that the efficiency gap widens as the dimension $D$ increases. As expected, the nonparametric classifier and the unconstrained Bayes classifier have poorer performance than the projective Bayes classifier, as the former do not leverage the partial ordering information in the biomarkers. The performance of additive logistic regression is somewhat erratic: it is competitive with the projective Bayes classifier in some scenarios (e.g., when $D=13$), but it is worse than the nonparametric classifiers in some scenarios; importantly, its efficiency may be lower with a larger $N$ likely due to bias caused by model misspecification, which has increasing influence as the variance diminishes with a large $N$. As efficiency is defined in terms of classification error rate (13) relative to the benchmark, the simulation results show that the projective Bayes classifier consistently improves classification accuracy compared to the other methods especially in situations with large $N$ and $D$.

Fig 2.

Relative efficiency of projective Bayes classifier (exact and approximate), additive logistic regression, nonparametric classifier, and unconstrained Bayes classifier.

5.3. Simulation setting 3: Continuous biomarkers.

The third simulation study considers the accuracy of the projective Bayes classifier $\hat{\mathit{\gamma }}$ in settings where the assay gives continuous biomarker values. In the simulation, each biomarker is standardized and follows the uniform distribution, i.e., $x_{kd}~\text{uniform}(0,1)$, for each $d=1,\dots ,D$. We consider $D=5\text{and}6$, and sample size $N=400,800,1200,1600,2000,4000,\text{and}8000$. Because of the continuous biomarker values, each biomarker panel ${\mathit{x}}_k$ is unique (i.e., $m_k=1$) and the number $K$ of unique biomarker combination in the sample is the same as the sample size $N$. For panel ${\mathit{x}}_k$, the outcome $y_k$ is drawn from Bernoulli with probability

$${\theta }_k=0.15+0.7I\left(x_{k1}>0.5\right)I\left\{x_{k4}^{1-x_{k3}}+\mathit{\text{log}}\left(x_{k2}+1\right)>1\right\}$$ (14)

so that biomarker $x_{k5}$ and $x_{k6}$ are not predictive of the outcome. That is, including them as input features for a classifier will only add noise.

We evaluate the projective Bayes classifier $\hat{\mathit{\gamma }}$ based on the classification error rate (13) relative to the benchmark classifier $\mathit{\gamma }$ in the same manner as in the previous subsection. In addition, we include the classification rule based on logistic regression that assumes each biomarker has an additive, logit-linear relationship with the outcome. Since all biomarker panel has only $m_k=1$ observation, completely nonparametric rule is not feasible. Instead, we include a classification rule based on monotone generalized additive model (MGAM) which assumes nondecreasing, additive relationship between a biomarker and the outcome, without the logit-linearity assumption. MGAM allows additional flexibility in estimating ${\theta }_k$ and is implemented using the LogisticGAM model in the pyGAM library in Python (Servén and Brummitt, 2018). Figure 3 shows the efficiency of these three classifiers relative to the benchmark. Logistic regression is the least efficient likely due to the fact that the model is misspecified. MGAM performs better than logistic regression generally, as the former model is more flexible than the latter. Notably, both methods do not improve as $N$ increases. By contrast, the projective Bayes classifier does not have the limitation of model misspecification and its efficiency increases with the sample size. When $N$ is ‘small’ and the biomarker values are sparse, the variability in the data affects the performance of the projective Bayes method, which does not leverage any parametric assumption; this is indicated by the noticeable drop in efficiency from $D=5$ and $D=6$, i.e., when more non-predictive features are used in constructing the classifier. This suggests the importance of feature selection when applying the projective Bayes classifier and other classification methods in general.

Fig 3.

Relative efficiency of projective Bayes classifier, logistic regression, and monotone generalized additive model (MGAM).

6. Real data example: HPV assay for cervical cancer.

6.1. Analytical approaches.

We analyzed the HPV assay data in $N=928$ women using five analytical approaches to estimate the classification rule ${\gamma }_k=I\left({\theta }_k>0.5\right)$ based on this assay:

• Approach 1 (Logistic classifier): Use ${\tilde{\gamma }}_k^{\mathit{\text{Cont}}}=I\left({\tilde{\theta }}_k^{\mathit{\text{Cont}}}>0.5\right)$ where ${\tilde{\theta }}_k^{\mathit{\text{Cont}}}$ is the fitted value for ${\theta }_k$ based on logistic regression with the biomarker values as additive continuous predictors.

• Approach 2 (Manufacturer cutoffs): Use the cutoffs provided by the assay manufacturer for high viral loads to dichotomize the biomarkers:

  $$\begin{array}{rr}& x_{k1}=I(\text{Type}16>0.0250),\\ & x_{k2}=I(\text{Type}18>0.0250),\\ & x_{k3}=I(\text{Type}31>0.0263),\\ & x_{k4}=I(\text{Type}39>0.0263),\\ & x_{k5}=I(\text{Type}51>0.0263),\end{array}$$ (15)

  and use the classification rule ${\tilde{\gamma }}_k^{\mathit{\text{Dich}}}=I\left({\tilde{\theta }}_k^{\mathit{\text{Dich}}}>0.5\right)$ where ${\tilde{\theta }}_k^{\mathit{\text{Dich}}}$ is the fitted value for ${\theta }_k$ based on logistic regression with the dichotomized values (15) as additive predictors.

• Approach 3 (Projective Bayes classifier): Use projective Bayes classifier $\hat{\mathit{\gamma }}$ with the biomarker values as nondecreasing continuous predictors.

• Approach 4 (Projective Bayes classifier based on manufacturer cutoffs): Use projective Bayes classifier $\hat{\mathit{\gamma }}$ with the dichotomized values (15) as nondecreasing binary predictors.

• Approach 5 (MGAM): Use ${\tilde{\gamma }}_k^{\mathit{\text{MGAM}}}=I\left({\tilde{\theta }}_k^{\mathit{\text{MGAM}}}>0.5\right)$ where ${\tilde{\theta }}_k^{\mathit{\text{MGAM}}}$ is the fitted value for ${\theta }_k$ based on monotone generalized additive model with the biomarker values as additive continuous predictors.

6.2. Numerical results: classification rules.

This subsection presents the numerical results of the classification rules using the five analytical approaches.

For the logistic classifier (Approach 1) that uses biomarker values as continuous predictors, a positive diagnosis is defined if

$$-2.46+96.4\text{type}16+42.6\text{type}18+78.5\text{type}31-3.09\text{type}39+1.06\text{type}51>0.$$ (16)

We note that the logistic classifier does not impose monotonicity and gives a negative coefficient for type 39. However, its coefficient is small compared to the others and it is not statistically significantly different from 0. Similarly, the coefficient of type 51 is small and not statistically significant. In other words, the classification rule (16) is mainly driven by outputs of types 16 and 31, and to a lesser extent by type 18. The large positive coefficients indicate the plausibility of nondecreasing association between these biomarkers and the biopsy outcomes.

For Approach 2 that uses the manufacturer’s cutoffs as dichotomized predictors (15), a positive diagnosis is defined if

$$-2.33+3.13x_{k1}+1.22x_{k2}+2.59x_{k3}+0.049x_{k4}+0.345x_{k5}>0.$$ (17)

Rules (16) and (17) concur in that the coefficients of type $16\left(x_{k1}\right)$ and type $31\left(x_{k3}\right)$ are largest and therefore drive the classification. In fact, since each $x_{kd}\in \{0,1\}$ in (17), the rule effectively excludes types 18, 39, and 51 from being relevant in classifying the outcomes. (It is because their coefficients—individually and their sum—are smaller than 2.33.) Iterating all possible combinations of ${\mathit{x}}_k$, we can verify that a positive diagnosis per (17) is defined by having a high viral load of types 16 or 31 per the manufacturer’s cutoffs (15), i.e., $x_{k1}=1$ or $x_{k3}=1$.

For the projective Bayes classifier using continuous biomarkers (Approach 3), Table 4 prescribes 11 threshold sets of panel values that define a positive diagnosis. Three of the 11 threshold sets are based on a single type (types 16, 18, and 31), and the other require more nuanced combinations of the output values (threshold sets 2.1–2.3 and 3.1–3.5). These rules can be easily tabulated for transparency and programmed for implementation in practice. Additionally, we note that all threshold values in Table 4 are greater than the corresponding manufacturer cutoffs (15) thus potentially leading to a more specific test.

Table 4.

Thresholds for positive diagnosis per the projective Bayes classifier

Diagnostic threshold	Type 16 >	Type 18 >	Type 31 >	Type 39 >	Type 51 >
1.1	0.0310	—	—	—	—
1.2	—	0.0353	—	—	—
1.3	—	—	0.0312	—	—
2.1	—	0.0315	—	—	0.0452
2.2	—	—	0.0306	—	0.0270
2.3	—	—	—	0.0375	0.0500
3.1	0.0278	—	0.0272	—	0.0313
3.2	0.0260	—	—	0.0305	0.0291
3.3	0.0260	—	0.0274	0.0270	—
3.4	0.0260	—	0.0278	—	0.0408
3.5	—	0.0266	0.0274	0.0380	—

For the projective Bayes classifier using dichotomized biomarkers (Approach 4), iterating all $2^D=32$ possible combination of biomarker values, we find that combinations yielding a positive diagnosis are defined by either $x_{k1}=1$ or $x_{k3}=1$ regardless of the other biomarker types. In other words, the approach gives the same classification rule as Approach 2. This suggests that the analytical approach used to generate classification rule may be less of a consideration if dichotomized biomarkers, rather than continuous biomarkers, are as used input features.

For MGAM (Approach 5), the marginal biomarker-outcome relationships are plotted in Figure 4. The model fits indicate clear monotonicity between each biomarker and a positive diagnosis (except perhaps for type 39). The classification rule, however, cannot be explicitly specified or tabulated as in the other approaches.

Fig 4.

Monotone generalized additive model fit of the HPV assay data in Kuhn et al. (2020).

6.3. Performance comparison.

In the overall sample of 928 women, projective Bayes using continuous biomarkers (Approach 3) has comparable sensitivity to using manufacturer cutoffs (Approaches 2 and 4) with superior specificity (an 8-percentage-point difference); see Table 5. Also, the projective Bayes method gives better sensitivity than MGAM with similar specificity; and it is generally more accurate than logistic regression using continuous biomarkers (Approach 1) in terms of correct classification, specificity, and sensitivity.

Table 5.

Comparison of classification rules in the overall sample and in subgroups: correct classification proportion (correct), specificity (spec), and sensitivity (sens)

Method	Overall			HIV negative subgroup			HIV positive subgroup
	correct	spec	sens	correct	spec	sens	correct	spec	sens
Approach 1: Eq. (16)	0.84	0.89	0.75	0.89	0.94	0.73	0.80	0.82	0.77
Approach 2+: Eq. (17)	0.84	0.85	0.81	0.89	0.92	0.80	0.78	0.76	0.81
Approach 3: Table 4	0.89	0.93	0.80	0.91	0.96	0.77	0.87	0.89	0.83
Approach 5: Figure 4	0.86	0.92	0.73	0.89	0.96	0.70	0.82	0.87	0.74

⁺Approach 4 yields the same classification rule as Approach 2.

In addition to the entire population, the study investigators are interested in evaluating the assay in subgroups of women by HIV status. In the HIV negative subgroup, all methods have comparable accuracy. However, the projective Bayes classifier (Approach 3) is more accurate than the others in HIV positive women. We note that the projective Bayes classifier does not leverage the HIV status in predicting biospy outcomes. Therefore, its consistent performance in the subgroups as well as in the overall sample is reassuring in that the method may have captured the underlying association between the assay and biopsy that is common across subgroups.

As the study aims to assess the overall diagnostic value of the assay, we also estimate ${\gamma }_k=I\left({\theta }_k>t_0\right)$ for $t_0=0.001,0.002,\dots ,0.999$ and evaluate the sensitivity and specificity for each $t_0$. Figure 5 shows the ROC curves of the classification rules based on the five different analytical approaches. While all analytical methods give area under the curve (AUC) close to or above 0.90, projective Bayes classifier using continuous biomarkers depicts the most favorable case for the assay, not only because it gives largest AUC in the overall sample, but it also performs consistently in both subgroups.

Fig 5.

ROC curves of different disease classification rules based on HPV assay data in Kuhn et al. (2020). The overall sample consists of 928 women, of whom 482 are HIV negative and 446 are HIV positive.

7. Discussion.

In diagnostic assay, the direction of association (if there is any) between a biomarker and the disease outcome is often known. Thus, monotonicity is a well suited assumption when we estimate the underlying disease classification rule. In this article, we introduce a novel sequential update method that yields an estimate $\hat{\mathit{\gamma }}$ of the classification rule by projecting a nonparametric Bayes classifier onto a partial ordering subspace (3). The proposed method offers several practical advantages. First, the projection method is invariant to any monotone transformation of the biomarkers, i.e., solving the maximization problem (4) will give the same classification rule whether it is applied to the biomarkers on their original scale or log scale. It thus alleviates some of the issues of analytical methods being sensitive to feature engineering. Also, in many situations, an assay will give an undetectable readout, which is often imputed as zero or an arbitrarily small, positive value. While the choice of this undetectable limit may affect parametric methods such as logistic regression, the projective Bayes classifier is not sensitive to its choice. Second, the classifier can work with panel of ordinal biomarkers, continuous biomarkers, or mixed types. Third, the method naturally leverages interactions of biomarkers in classification without imposing strong assumptions; and at the same time, the resulted classification rule can be tabulated (Table 4) providing investigators with clear rationales of the rule. Fourth, as shown in our simulation studies, the projective Bayes classifier is efficient compared to completely unconstrained methods, and it guards against accuracy loss due to model misspecification (cf. Table 5).

Despite the advantages of projection-based method, challenges in computation can limit its practical utility. While the projective Bayes classifier is defined mathematically in (4) and is a principled approach to estimate $\mathit{\gamma }$, computing it by brute force involves $O\left(2^K\right)$ operations, where $K$ is the number of unique biomarker panel values and grows linearly with sample size. The idea of the sequential update method (Algorithm 1) is that the original maximization problem (4) can be replaced with sequentially defined smaller and computationally tractable tasks, which is also the rationale behind the SRSM algorithm in Cheung and Diaz (2023). A contribution of this article is that it provides the theoretical results (Section 3) and new practical algorithms (Section 4) that facilitate the computation of these sequentially defined tasks that will result in the Bayes solution, as opposed to an approximated solution by the SRSM algorithm. We have also demonstrated in simulation that the sequential update method takes shorter time than SRSM. The sequential update method, implemented using one of the proposed recursive algorithms, is indeed computationally feasible in the numerous settings considered in our simulation studies and real data example. Thus, our work has enhanced the practicality of the projective Bayes classifier under partial ordering and broadened its use.

In our simulation studies, for certain data sets, it could take an algorithm an unusually long time to complete the task, as demonstrated by the right-skewed elapsed time of DD in Table 2. However, we have not experienced scenarios where the algorithms were ‘stuck’ in our simulation, although computation could progress slowly when $K$ was large. Computation speed can be further improved in two pragmatic ways in data analysis. Firstly, while any algorithm may take a long time to run for a given data set, the other algorithms are often able to compute much more quickly. Our experience in particular indicates that DU and UD complement each other well. Thus, a practical approach to data analysis is to run the recursive algorithms in parallel sessions and take the results from the first finished task. This can indeed be formalized by bundling the algorithms for parallel computing such as using the ‘future’ library in R. Secondly, when biomarkers are measured on a continuous scale, we could bin the biomarkers (say) into deciles according to the level of resolution required by the applications (Jang et al., 2021). This will reduce computation time quite substantially by reducing $K$ in the data.

For clarity in exposition, in this article, we focus on classification of binary outcomes. The method is readily applicable to general situations where the outcome $y_{ki}$ is a continuous or ordinal outcome such as a risk score. In addition, Cheung and Diaz (2023) point out that $\theta \left(\mathit{x}\right)$ can be evaluated by inverting the classifier $\gamma \left(\mathit{x}\right)$ defined across different $t_0$ (cf. Section 6.3). Thus, the algorithms proposed here can be applied to situations where we are interested in estimating the underlying parameter $\theta \left(\mathit{x}\right)$ in addition to the classification rule $\gamma \left(\mathit{x}\right)$.

Supplementary Material

Technical details and code

Proofs of the theorems are provided in Web Appendix A. To facilitate dissemination and reproducibility, R code for implementation is given in Web Appendix B.