Optimizing Sequential Decision Rules for Prostate Cancer Biopsy Management: A Multi-Objective Statistical Framework

Abstract

Binary medical decision-making increasingly demands sequential diagnostic strategies that optimize accuracy while minimizing patient burden and healthcare costs. In prostate cancer diagnosis, many patients undergo unnecessary biopsies despite existing biomarkers and imaging tests that already inform risk stratification. Sequential testing, where tests are selectively administered based on prior results, offers a promising approach to balance diagnostic power with procedural efficiency. We propose a novel framework for deriving optimal sequential decision rules using a multi-objective optimization perspective. Specifically, we aim to (1) minimize unnecessary invasive procedures while maintaining sensitivity to underlying disease, and (2) reduce procedural costs by limiting the number of subsequent tests. Rather than collapsing multiple goals into a single weighted score, which forces subjective choices about trade-off weights, we optimize one target while requiring the others to meet prespecified standards. This constrained formulation can be solved efficiently using Lagrange multipliers, yielding a family of optimal sequential rules and a trade-off curve that summarizes the best achievable balance among sensitivity, specificity, and testing burden for clinical protocol design. In the prostate cancer diagnostic data with biomarker and imaging measurements and biopsy-confirmed outcomes, the sequential strategies identify testing pathways that reduce unnecessary procedures while preserving diagnostic quality.

1. Introduction

In binary medical decision-making scenarios for early detection, there is a critical need for reliable decision rules that can accurately identify patients with malignant outcomes while avoiding unnecessary invasive diagnostic procedures for those at low risk. For example, in prostate cancer (PCa) diagnosis, patients presenting with elevated Prostate-specific antigen (PSA) are typically referred for a prostate biopsy, an invasive procedure associated with potential complications. However, due to the relatively low prevalence of clinically significant PCa, many biopsies ultimately prove unnecessary. To address this issue, the National Comprehensive Cancer Network (NCCN) currently recommends six pre-biopsy tests based on blood- and urine-based biomarkers. These guidelines recommend risk stratification using either biomarker testing or multiparametric magnetic resonance imaging (mpMRI), when available, before deciding on biopsy. With a growing variety of diagnostic tools, each differing in cost, side effects, accessibility, and effectiveness, efforts are needed to balance diagnostic accuracy, minimize harm, and manage healthcare resources efficiently. While employing all available tests may improve disease detection, the associated risks and costs are often unjustified in low-risk cases. A more cost-effective and personalized solution lies in sequential testing, in which diagnostic procedures are selectively administered based on intermediate results. For instance, a three-step protocol for PCa diagnosis might operate as follows:

Example (PCa).

Sequential biopsy necessity assessment using biomarker and MRI.

1. Patients with a model score using routine clinical information below a lower threshold are spared further testing; those exceeding an upper threshold are referred directly for biopsy; those in the intermediate range proceed to novel biomarker testing.

2. Among patients who receive a biomarker test, those with the score below a lower threshold are spared further testing; those exceeding an upper threshold are referred directly for biopsy; those in the intermediate range proceed to mpMRI.

3. Among those receiving mpMRI, a combined assessment of biomarker and imaging results informs the final decision on whether to proceed with biopsy.

Such personalized diagnostic pathways have the potential to reduce unnecessary interventions, lower costs, and lessen patient burden. However, careful selection of thresholds is essential to preserve overall clinical effectiveness and prevent delays in care.

Our goal is to develop statistical methods for deriving such sequential decision rules. The analytical task is two-fold: (1) minimizing unnecessary biopsies (false positives) while maintaining an acceptably high rate of detecting aggressive cancers (true positives); and (2) minimizing the overall procedural costs by limiting the number of patients assigned to subsequent steps. The two tasks involve a trade-off: overall performance outlined in (1) may, in general, be compromised by limiting testing to reduce costs in (2). Thus, developing an effective sequential rule requires quantifying the trade-offs between these competing interests and optimizing the configuration of each step. Note that we focus here on a rule-out setting. In rule-in applications, one could instead prioritize detection through the true positive rate while constraining overtreatment (false positive rate) to a clinically meaningful level.

Existing statistical literature on deriving benefit-risk balanced medical decision rules has primarily focused on non-sequential settings. For example, individualized treatment strategies using all selection tools have been proposed to balance treatment efficacy and risk (Laber et al., 2014; Wang et al., 2018). The Neyman–Pearson (NP) framework provides a natural paradigm for controlling one type of error while minimizing the other type. The NP classifiers are applied to the diagnostic testing setting, for example, to minimize the false positive rate (FPR) under the constraint of the true positive rate (TPR) (e.g., Neyman et al., 1933; Scott and Nowak, 2005; Wang et al., 2020, 2022). However, existing NP-based procedures assume a single decision point and do not extend to multi-step diagnostic pathways.

To address the second task of reducing procedural costs under a sequential setting, one approach is to adopt the sequential probability ratio test (SPRT). It was originally developed for rapid hypothesis testing under fixed Type I/II error constraints (Wald, 1947). In classification settings, the approach can be adopted for deriving sequential decision rules, aligning type I/II errors with the corresponding classification errors (e.g., Sochman and Matas, 2005). However, the SPRT does not guarantee achieving the prescribed error rates, nor does it ensure a decision within a fixed number of steps. In our setting, these limitations become pronounced, making the SPRT inadequate.

In this work, we aim to optimize multiple objectives simultaneously and cast the multiple tasks in a multi-objective optimization (MOO, e.g., Miettinen, 1999; Deb et al., 2016) framework. A popular approach for MOO is scalarization, which combines, in our case, cost and accuracy metrics (e.g., the misclassification rate, such as the $F_1$ score, TPR, and FPR), into a single objective function via a weighted sum. In sequential settings, the weighted objective couples decisions across steps and can be optimized using dynamic programming by rewriting it as a recursion over step-specific value functions. Applications of such an approach include cost-sensitive feature selection (Trapeznikov and Saligrama, 2013; Wu and Zhu, 2015); longitudinal diabetic risk prediction (Pan et al., 2023); and diagnostic panel selection (Yu et al., 2023). However, the scaling approach has key limitations that hinder its applicability in practical settings. It requires prior specification of objective weights, which may lack consensus or clinical interpretability. This requirement also restricts exploration of the full accuracy-cost spectrum, resulting in a single-point solution rather than a comprehensive characterization of trade-offs.

Beyond scalarization, multi-objective reinforcement learning methods such as Pareto Q-learning (Moffaert and Nowé, 2014; Ruiz-Montiel et al., 2017) and explicit constraint-based approaches (Huang et al., 2022) have been proposed to simultaneously obtain sets of policies without requiring hand-tuned scalarization weights. These methods are typically formulated within a Markov Decision Process (MDP) framework, in which an agent’s actions influence future states and cumulative returns over time. In contrast, we consider diagnostic testing, where actions reveal information about an underlying disease state but do not alter its occurrence. Moreover, our analysis is based on offline observational trajectories with no opportunity for intervention, and our objectives are defined directly through operating characteristics (TPR, FPR, and testing burden) rather than cumulative rewards. Consequently, an explicit MDP formulation is unnecessary. Instead, we enforce these clinically meaningful targets directly via an $\epsilon$-constraint formulation, which provides a more direct and practical approach in our setting.

We propose a novel statistical framework for deriving optimal sequential decision rules. It converts the multi-objective optimization into a constrained problem by treating all but one objective as constraints with user-specified bounds ($\epsilon$ values). This strategy directly encodes clinically meaningful performance targets and yields a family of decision rules indexed by the specified bounds.

Leveraging Lagrangian duality, we derive a closed-form solution for the constrained optimization problem. Under mild regularity conditions (e.g., continuity and boundedness of risk functions), the resulting rule is shown to be Pareto optimal, the optima of the MOO. That is, no objective can be improved without worsening at least one other. Varying the $\epsilon$ constraint values generates the Pareto front, which offers a visualization of the trade-offs between diagnostic accuracy and procedural burden.

Section 2 introduces the data and motivates our method development. In Section 3, we formally define the MOO problem and derive the Pareto optimal rule. In Section 4, we propose a flexible plug-in estimation procedure that accommodates both simple linear models and more complex non-parametric alternatives, whose asymptotic optimality is discussed in Section 5. Section 6 evaluates the method through simulation. Section 7 illustrates how sequential strategies can enhance efficiency without compromising diagnostic performance, addressing our motivating questions in the prostate cancer detection setting.

2. The Prostate MRI (pMRI) Data and Motivation

The pMRI study is an ongoing prospective cohort study conducted by the National Cancer Institute’s Early Detection Research Network. Its primary objective is to evaluate the combined utility of pMRI and panels of laboratory biomarkers for the detection of prostate cancer (PCa) in contemporary clinical settings. To date, 311 men with suspected prostate cancer referred for diagnostic evaluation, including biopsy, have been enrolled. Among them, only 154 (50%) were diagnosed with clinically significant PCa, defined as Gleason Grade Group 2 or higher on biopsy. The study aims to develop a clinical decision rule that reduces unnecessary biopsies while ensuring that men with clinically significant disease are accurately identified for biopsy. Hence, the key objective is to maintain a high TPR, set at a minimum of 90%, to avoid under-detection of invasive cancer, while minimizing the FPR to reduce unnecessary testing and associated burden.

Three diagnostic tools, summarized in Table 2.1, are being evaluated to support biopsy decision-making: (1) Prostate Cancer Prevention Trial (PCPT) risk calculator, which is based on routinely collected clinical factors including age, race, family history, prior biopsy, prostate volume, digital rectal exam (DRE) findings, and PSA levels; (2) MyProstateScore 2.0 (MPS2), a novel urine-based biomarker panel that incorporates the expression of 18 genes along with clinical variables (Tosoian et al., 2024; Clarke, 2025); and (3) pMRI, interpreted using the Prostate Imaging Reporting and Data System (PI-RADS) on a scale of 0–5, with scores of 4 or 5 considered positive. In addition to their diagnostic performance, these tests vary considerably in terms of cost and ease of clinical implementation. Based on reported average provider costs, we estimated $50 for PCPT, $760 for MPS2, and $4,419 for MRI. A de-identified and slightly perturbed dataset from the study was used for the analysis, as it is still ongoing. Specifically, Gaussian noise with standard deviation equal to 10% of the original variable’s standard deviation was added to each continuous variable (PCPT and MPS2). For each binary variable (MRI and outcome), 5% of the labels were randomly flipped. The findings remain consistent with those obtained from the original data.

Table 2.1:

Biomarkers (median, (Q1, Q3)) and MRI-positive counts (%), stratified by clinically significant or low-grade prostate cancer (PCa).

	Low-grade PCa N = 154	Clinically Significant PCa N = 157
PCPT	0.08 (0.05, 0.11)	0.12 (0.08, 0.18)
MPS2	0.06 (0.02, 0.19)	0.37 (0.17, 0.64)
MRI positive	86 (56%)	143 (91%)

Table 2.2 presents the cross-validated performance of each test individually and in combination using simultaneous decision rules based on logistic regression. Specifically, for each subset of tests (e.g., PCPT + MPS2), we fit a logistic regression using the corresponding test results as covariates to obtain a scalar risk score, and choose a threshold to attain the target TPR in the training fold; we then report the resulting performance on held-out folds.

Table 2.2:

Average performance (and 95% intervals) of non-sequential protocols over 100 repeated experiments. For example, “PCPT + MPS2” denotes a logistic-regression risk score fitted on tests PCPT and MPS2, thresholded to achieve the target TPR.

Protocol	Cost	FPR%	TPR%
PCPT only	50	74 (64, 82)	89 (88, 91)
MPS2 only	760	52 (37, 70)	90 (88, 91)
MRI only	4419	56 (49, 62)	91 (86, 95)
PCPT + MPS2	810	52 (39, 68)	90 (87, 91)
PCPT + MRI	4469	52 (39, 65)	90 (88, 91)
MPS2 + MRI	5179	46 (34, 60)	89 (87, 91)
PCPT + MPS2 + MRI	5229	46 (34, 62)	89 (87, 91)

Combining multiple test modalities generally improves diagnostic accuracy, but at increased burden and cost. In our data, using all three tests together yields the lowest FPR, 46% (95% bootstrapping confidence interval (CI): 34%, 62%), at 89% TPR, but also incurs the highest expected cost ($5,229). By contrast, the MPS2-only protocol, which is cheaper and more accessible, achieves an FPR of 52% (95% CI: 37%, 70%) at the same TPR. These results suggest that much of the performance gain from multi-modality testing may be attainable without applying the most costly procedure (e.g., MRI) to everyone.

Motivated by this observation, we seek to construct sequential protocols that combine the biomarker tests and MRI to answer the following scientific questions:

1. Can a sequential protocol maintain the diagnostic performance of multi-modality testing, in the sense of achieving a prespecified TPR while keeping FPR close to the three-test strategy?

2. How much can such a protocol reduce the utilization of the more costly procedure (e.g., MRI) and the associated expected cost, by reserving it for individuals whose earlier results are inconclusive?

These questions directly motivate the methodological development in Section 3–Section 5.

3. Trackable Pareto Optimality via Lagrangian Duality

We first formally specify the MOO problem and derive a closed-form solution via the $\epsilon$-constraint approach and Lagrangian duality. We then provide a theoretical justification for the existence and Pareto optimality of such a solution.

3.1. The MOO Problem and Associated Lagrangian Dual

For a sequential $T$-step protocol, denote the associated outcome as $Y\in \left\{\mathrm{0,1}\right\}$, and the covariate information gathered at step $t$ as $X_t\in R$ for $t=1,\dots ,T$. In the context of the PCa example in Section 2, we have $T=3$. $X_1,X_2,X_3$, and $Y$ would represent PCPT, MPS2, MRI result, and the presence of the aggressive disease necessitating biopsy, respectively. Denote a sequential decision rule as

$$\phi \left(X_{1:T}\right)=1\left(f_1>0\right)+1\left(f_1=0,f_2>0\right)+\cdots +1\left(f_{1:T-1}=0,f_T>0\right),$$ (3.1)

where $1(\cdot )\in \left\{\mathrm{0,1}\right\}$ is the indicator function taking 1 if the condition is true, and 0 otherwise (see Figure 3.1 for a visualization). To simplify notation, for $t=1,\dots ,T$, we write $f_t=f_t\left(X_{1:t}\right)\in R$ as the step-wise rule at step $t$ based on the history $X_{1:t}=\left(X_1,\dots ,X_t\right)$; write $f_{1:t}=f_{1:t}\left(X_{1:t}\right)=\left(f_1\left(X_1\right),\dots ,f_t\left(X_{1:t}\right)\right)$ as the step-wise rules till the $t$th step, so $f_{1:T-1}=0$ means $f_1\left(X_1\right),\dots ,f_{T-1}\left(X_{1:T-1}\right)$ are all zero.

Figure 3.1:

A flow chart for the sequential decision rule (3.1).

An early conclusion is made for the smallest $t_0$ such that $f_{t_0}\ne 0$ for any $t_0<T$, under which circumstances subsequent covariates $X_{t_0+1,T}$ are unnecessary, and the conclusion is determined by the sign of $f_{t_0}$.

In the prostate cancer application, the protocol is designed primarily to rule out an unnecessarily invasive procedure (biopsy). Accordingly, our primary goal is to avoid recommending biopsy for men without clinically significant cancer, while ensuring those with aggressive disease are still identified at a clinically acceptable rate. This rationale motivates our choice to minimize the FPR subject to maintaining the TPR above a pre-specified threshold. The reverse direction would be appropriate for rule-in applications where missed cases are the major concern.

Denote the FPR as $\mathbb{E}\left[\phi \left(X_{1:T}\right)\mid Y=0\right]$, TPR as $\mathbb{E}\left[\phi \left(X_{1:T}\right)\mid Y=1\right]$, and step-inconclusiveness as $\mathbb{P}\left(f_{1:t}\left(X_{1:t}\right)=0\right)$ for all $t<T$, representing the proportion of cases remain inconclusive till step $t$. The target MOO problem then seeks to, subject to a TPR constraint, minimize the FPR while simultaneously minimizing the step-inconclusiveness at each step. Given a desired TPR value of $\beta \in \left(\mathrm{0,1}\right)$, we aim to find $\phi$ in the form (3.1) that

$$\begin{array}{l}\text{minimize}\left(\mathbb{E}\left[\phi \left(X_{1:T}\right)\mid Y=0\right],\mathbb{P}\left(f_1\left(X_1\right)=0\right),\dots ,\mathbb{P}\left(f_{1:T-1}\left(X_{1:T-1}\right)=0\right)\right)\text{,}\\ \text{subject}\text{to}\mathbb{E}\left[\phi \left(X_{1:T}\right)\mid Y=1\right]=\beta \text{.}\end{array}$$ (3.2)

Note that there are $T-1$ inconclusiveness for a $T$-step protocol. Together with the FPR objective, it constitutes a $T$-objective problem. Recall that the optima of multi-objective problems are called Pareto optimal, where none of the objectives can be further improved without sacrificing some other objectives. In general, multiple optima exist, the collection of which is called the Pareto front.

We employ the $\epsilon$-constraint approach to simplify the MOO to a constrained single-objective optimization problem. It constrains all but one objectives by some constraint values, and the Pareto front is computed by looping over such values. Subsequently, problem (3.2) translates to, over all $\phi$ of form (3.1),

$$\begin{array}{l}\mathrm{m}\mathrm{i}\mathrm{n}\mathrm{i}\mathrm{m}\mathrm{i}\mathrm{z}\mathrm{e}\mathbb{E}\left[\phi \left(X_{1:T}\right)\mid Y=0\right]\text{,}\\ \text{subject}\text{to}\mathbb{P}\left(f_1\left(X_1\right)=0\right)\le {\gamma }_1,\dots ,\mathbb{P}\left(f_{1:T-1}\left(X_{1:T-1}\right)=0\right)\le {\gamma }_{T-1}\text{,}\\ \text{and}\mathbb{E}\left[\phi \left(X_{1:T}\right)\mid Y=1\right]=\beta \text{,}\end{array}$$ (3.3)

for some $1\ge {\gamma }_1\ge \cdots \ge {\gamma }_{T-1}\ge 0$.

By putting one of the $T$ objectives as the main objective and the remaining set of the $T-1$ objectives into constraints, a total of $T$ variants of the $\epsilon$-constraint problems exist for our $T$-objective problem (3.2). We can also formulate the $\epsilon$-constraint problem by minimizing a single-step inconclusiveness under the constraints on the inconclusiveness of other steps, FPR, and TPR (see, e.g., problem (3.10)). We choose to translate step-inconclusiveness to constraints rather than the FPR in the form of (3.3) due to its interpretability and simplicity in specifying the values of ${\gamma }_1,\dots ,{\gamma }_{T-1}$.

By introducing Lagrangian multipliers, we derive the following proposition, which provides a tractable formulation of the objectives and the optimal decision rule via the associated dual function.

Proposition 3.1.

The Lagrangian function of (3.3) is

$$ℒ\left(f_{1:T},\mathit{a}\right)=a_T\beta -\sum _{t=1}^{T-1}{a}_t{\gamma }_t-\mathbb{E}\left[{\eta }_{1}1\left(f_1>0\right)\right]-\sum _{t=2}^T\mathbb{E}\left[1\left(f_{1:t-1}=0\right)\left({\eta }_{t}1\left(f_t>0\right)-a_{t-1}\right)\right]$$ (3.4)

for some Lagrangian multipliers $\mathit{a}=\left(a_1,\dots ,a_T\right)$ with $a_1,\dots ,a_{T-1}\ge 0$ and $a_T\in R$, where $\eta$ is defined by

$${\eta }_t={\eta }_t\left(X_{1:t};a_T\right)=\left(a_T{p}_1^{-1}+p_0^{-1}\right){\mu }_t-p_0^{-1},$$ (3.5)

with $p_1:=\mathbb{P}(Y=1)>0,p_0:=1-p_1>0$, and ${\mu }_t:=\mathbb{E}\left[Y\mid X_{1:t}\right]$ for $t=1,\dots ,T$.

Define the Lagrangian dual function as $G(\mathit{a})=\underset{f}{\mathrm{i}\mathrm{n}\mathrm{f}}ℒ(f,\mathit{a})$ with the infimum taken over all $ℱ$-adapted $f_{1:T},\mathrm{i}.\mathrm{e}.,f_t\mathrm{i}\mathrm{s}{ℱ}_t$-measurable for all $t=1,\dots ,T$, where $ℱ=\left({ℱ}_1,\dots ,{ℱ}_T\right)\mathrm{i}\mathrm{s}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{n}\mathrm{a}\mathrm{t}\mathrm{u}\mathrm{r}\mathrm{a}\mathrm{l}\mathrm{f}\mathrm{i}\mathrm{l}\mathrm{t}\mathrm{r}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}\mathrm{s}\mathrm{p}\mathrm{a}\mathrm{n}\mathrm{n}\mathrm{e}\mathrm{d}\mathrm{b}\mathrm{y}{X}_{1:T}$. We have

$$G(\mathit{a})=a_T\beta -\sum _{t=1}^{T-1}{a}_t{\gamma }_t-\mathbb{E}\left[S_1\right],$$ (3.6)

where the $ℱ$-adapted sequence $S_{1:T}$ is defined by $S_T=S_T\left(X_{1:T};a_{T-1}\right):={\eta }_T^{+}-a_{T-1}$ and

$$S_t=S_t\left(X_{1:t};a_{t-1},\dots ,a_{T-1}\right):=\mathrm{m}\mathrm{a}\mathrm{x}\left({\eta }_t^{+},v_t^{+}\right)-a_{t-1},v_t:=\mathbb{E}\left[S_{t+1}\mid X_{1:t}\right]$$ (3.7)

for $t=1,\dots ,T-1$ with the convention that $a_0=0$. Note that $\left(S_t,v_t\right)$ are defined recursively relying on $\left(S_{t+1},v_{t+1}\right)$. Here ${\eta }_t^{+}:={\eta }_{t}1\left({\eta }_t\ge 0\right)$, so as for $v_t,t=1,\dots ,T$.

Here the quantities $\eta$, $v$, and $S$ depend on $\mathit{a}$, though we suppress the notation for simplicity. See Section S1 in the Supplement for proof, especially the derivation of (3.6). In particular, the infimum of $ℒ$ over $f$ is achieved at ${\tilde{f}}_{1:T}(\mathit{a})$ with ${\tilde{f}}_T\left(X_{1:T};\mathit{a}\right)={\eta }_T$, and for $t=1,\dots ,T-1$,

$${\tilde{f}}_t\left(X_{1:t};\mathit{a}\right)=\left\{\begin{array}{ll}1,& \text{if}{\eta }_t>v_t^{+},\\ 0,& \text{if}{v}_t\ge {\eta }_t^{+},\\ -1,& \text{if}\mathrm{m}\mathrm{a}\mathrm{x}\left(v_t,{\eta }_t\right)<0.\end{array}\right.$$ (3.8)

The step-wise rule ${\tilde{f}}_t$ in (3.8) takes action to maximize benefit. Intuitively, ${\eta }_t$ quantifies the benefit of immediate action at step $t$ with a positive decision, which takes the exact form of the non-sequential Neyman–Pearson classifier at the $t$th-step based on history $X_{1:t}$, e.g., $1\left({\mu }_t\left(X_{1:t}\right)\ge c\right)$ for some constant cutoff $c$. Meanwhile, $v_t$ embodies the anticipated benefit of delaying till the next step, should the subsequent steps follow the same policy, similar to the value iteration in reinforcement learning. Therefore, it chooses the option, either making an immediate positive decision $\left({\tilde{f}}_t=1\right)$ or waiting $\left({\tilde{f}}_t=0\right)$, that offers the greater benefit. Lastly, a negative immediate decision (${\tilde{f}}_t=-1$) is granted, presented with negative ${\eta }_t$ and $v_t$, i.e., neither a positive immediate decision nor waiting is beneficial.

3.2. Dual Optimal is Pareto Optimal

We now focus on the existence of Pareto optimal rules and the optimality of the proposed step-wise rule (3.8). The previous Proposition 3.1 greatly simplifies the problem by parameterizing (3.3) with $\mathit{a}$, so that it suffices to optimize for the finite-dimensional Lagrangian multipliers instead of all sequential rules. This enables discussion on the continuity and differentiability of the objectives with respect to $\mathit{a}$, which eventually leads to the existence and Pareto optimality. We now show (3.8) is also Pareto optimal for problem (3.2) under the optimal Lagrangian multipliers that optimize the following dual problem:

$$\underset{\mathit{a}}{\mathrm{m}\mathrm{a}\mathrm{x}}G(\mathit{a})\text{for}{a}_1,\dots ,a_{T-1}\ge 0\text{and}{a}_T\in R.$$ (3.9)

Below is a summary of the main ideas of the proof. Recall that problem (3.3) effectively optimizes along the FPR direction. By swapping the objective and constraint roles, we obtain different variants of the $\epsilon$-constraint problem that target other directions, such as TPR or inconclusiveness. A solution to a multi-objective problem is Pareto optimal if and only if it is optimal in all these directions and satisfies the constraints with equality (Miettinen, 1999, part II, section 3.2). Therefore to show the solution to the primal-dual problems (3.3) and (3.9) is also Pareto optimal for problem (3.2), we first show there exists a feasible rule in the form of (3.8) satisfying the constraints of (3.3) with equality (Proposition S1.1), which is also the dual optimum utilizing the derivatives of the dual function (Proposition S1.2). The rule is then shown to minimize all $\epsilon$-constraint problems, making it a Pareto optimum (Proposition 3.2). We refer to Section S1 of the Supplement for theoretical details.

Proposition 3.2 (Existence and Pareto optimality).

Suppose for $t=1,\dots ,T$, the $X_t$ is a continuous random variable admits density, and $X_{1:t}\mapsto {\mu }_{1:t}$ is a continuous function bounded away from zero, then there always exists some ${\mathit{a}}^{\mathrm{*}}$ that maximize the dual problem (3.9). Denote ${\tilde{f}}^{\mathrm{*}}(X)=\tilde{f}\left(X;{\mathit{a}}^{\mathrm{*}}\right)$ following (3.8) and ${\phi }^{\mathrm{*}}$ by plugging-in ${\tilde{f}}^{\mathrm{*}}$ to (3.1), then ${\phi }^{\mathrm{*}}$ optimizes the primal problem (3.3) satisfying the constraints with equality. Moreover, ${\phi }^{\mathrm{*}}$ also optimizes, for all $t_0=1,\dots ,T-1$ with $\alpha =\mathbb{E}\left[{\phi }^{\mathrm{*}}\mid Y=0\right]$,

$$\begin{array}{l}\underset{\phi }{\mathrm{m}\mathrm{i}\mathrm{n}}\mathbb{P}\left(f_{1:t_0}\left(X_{1:t_0}\right)=0\right),\\ \text{subject}\text{to}\mathbb{P}\left(f_{1:t}\left(X_{1:t}\right)=0\right)\le {\gamma }_t\text{for}t\ne t_0,\\ \mathbb{E}\left[\phi \left(X_{1:T}\right)\mid Y=0\right]\le \alpha ,\text{and}\mathbb{E}\left[\phi \left(X_{1:T}\right)\mid Y=1\right]=\beta ,\end{array}$$ (3.10)

satisfying the constraints with equality. Therefore, ${\phi }^{\mathrm{*}}$ is a Pareto optimum for the MOO problem (3.2).

Proposition 3.2 shows that (3.8) is Pareto optimal. It transforms the multi-objective problem (3.2) for sequential decision rules into a finite-dimensional optimization of the dual problem (3.9). Once ${\mathit{a}}^{\mathrm{*}}$ is obtained by, e.g., numeric optimization in practice, plugging it into (3.8) provides the Pareto optimal sequential rule.

Proposition 3.2 also gives a sufficient condition for the Pareto optimality: a sequential rule ${\phi }_0$ is Pareto optimal for problem (3.2) if $\mathbb{E}\left[{\phi }_0\mid Y=1\right]=\beta$ and in each step it takes the form of (3.8). It suffices to set ${\gamma }_1,\dots ,{\gamma }_{T-1}$ equal to the step-inconclusiveness of ${\phi }_0$ in problem (3.3) to see this.

4. Plug-in Estimation of the Optimal Rule

We next discuss how to obtain the Pareto solutions in practice presented with sample $\left(X_{i,1:T},Y_i\right)$, $i=1,\dots ,n$ that are independent copies of ($X_{1:T},Y$), by tackling an empirical version of problem (3.9). It suffices to tackle the processes ${\mu }_{1:T}$ and $v_{1:T-1}$ recursively and plug in the estimates to definitions of $S$ in (3.7) for an empirical dual function ($G$ in (3.6)), whose maximum point estimates the optimal Lagrangian multipliers (${\mathit{a}}^{\mathrm{*}}$). Subsequently, plugging in the estimated processes and optimal Lagrangian multipliers to $\tilde{f}$ of (3.8) provides an estimate for the optimal rule. In the end, the Pareto front is obtained by solving a family of problems (3.9) looping over all constraint values $\mathit{\gamma }$ satisfying $1>{\gamma }_1\ge \cdots \ge {\gamma }_{T-1}>0$. In particular, the surface of the optimal FPR with respect to inconclusiveness characterizes the optimal trade-off between diagnostic power and operational burden.

Recall by definition ${\mu }_t=\mathbb{E}\left[Y\mid X_{1:t}\right]$, it suffices to “project” $Y$ to covariates $X_{1:t}$. Similar logic applies to $v$ following their definition: $v_t=\mathbb{E}\left[S_{t+1}\mid X_{1:t}\right]$ for $t=T-1,\dots ,1$ and $S_T={\eta }_T^{+}-a_{T-1}$. It suffices to “project” ${\hat{S}}_{t+1}$ to $X_{1:t}$ in an inverse time order, where ${\hat{S}}_{t+1}$ is a function of ${\hat{v}}_{t+1}$ and ${\hat{\mu }}_{t+1}$. As summarized in Algorithm 1, such a recursive procedure provides $G(\mathit{a})$ for a specific $\mathit{a}$, which is used for optimization with respect to $\mathit{a}$.

Algorithm 1:

Estimate $G$ given $\mathit{a}$

Input: $X_{i,1:T},Y_i$ for $i=1,\dots ,n$ and the Lagrangian multipliers $\mathit{a}$.

Output: Estimated value of the dual function $G(\mathit{a})$

1 Estimate ${\hat{\mu }}_T$ by projecting $Y_i$ to $X_{i,1:T}$, for $i=1,\dots ,n$;

2 By (3.5), ${\hat{\eta }}_{i,T}\leftarrow \left(a_T{p}_1^{-1}+p_0^{-1}\right){\hat{\mu }}_T\left(X_{i,1:T}\right)-p_0^{-1}$ and ${\hat{S}}_{i,T}\leftarrow {\hat{\eta }}_{i,T}^{+}-a_{T-1}$ for $i=1,\dots ,n$;

3 for $t$ in $T-1,\dots ,1$ do

4 Estimate ${\hat{v}}_t$ by projecting ${\hat{S}}_{i,t+1}$ to $X_{i,1:t},i=\dots ,n$;

5 Estimate ${\hat{\mu }}_t$ by projecting $Y_i$ to $X_{i,1:t}$, for $i=1,\dots ,n$;

6 ${\hat{\eta }}_{i,t}\leftarrow \left(a_t{p}_1^{-1}+p_0^{-1}\right){\hat{\mu }}_t\left(X_{i,1:t}\right)-p_0^{-1}$ for $i=1,\dots ,n$;

7 ${\hat{S}}_{i,t}\leftarrow \mathrm{m}\mathrm{a}\mathrm{x}\left({\hat{\eta }}_{i,t}^{+},{\hat{v}}_{i,t}^{+}\right)-a_{t-1}$ for $i=1,\dots ,n$ by (3.7);

8 end

9 $G(\mathit{a})\leftarrow a_T\beta -\sum _{t=1}^{T-1}{a}_t{\gamma }_t-n^{-1}\sum _{i=1}^n{\hat{S}}_{i,1}$ by (3.6);

10 return $G(\mathit{a})$

Hence, the empirical dual function is

$$G(\mathit{a})=a_T\beta -\sum _{t=1}^{T-1}{a}_t{\gamma }_t-\frac{1}{n}\sum _{i=1}^n{\hat{S}}_{i,1}$$ (4.1)

by plugging-in the $\hat{\mu }$ and $\hat{v}$ to (3.5) and (3.7) for the estimator ${\hat{S}}_{i,1}$. Recall that all those estimates depend on $\mathit{a}$, yet we suppress the notation for simplicity in the right-hand side of (4.1). We then maximize $G$ using a standard numerical optimization routine to obtain an estimate of ${\mathit{a}}^{\mathrm{*}}$. Denote such maximizer subject to $a_1,\dots ,a_{T-1}\ge 0$ and $a_t\in R$ as $\mathit{a}$, then the optimal step-wise scores are ${\hat{f}}_T\left(x_{1:T};\mathit{a}\right)={\hat{\eta }}_T\left(x_{1:T},\mathit{a}\right)$, and

$${\hat{f}}_t\left(x_{1:t};\mathit{a}\right)=\left\{\begin{array}{ll}1,& \text{if}{\hat{\eta }}_t\left(x_{1:t};\mathit{a}\right)>{\hat{v}}_t{\left(x_{1:t};\mathit{a}\right)}^{+},\\ 0,& \text{if}{\hat{v}}_t\left(x_{1:t};\mathit{a}\right)\ge {\hat{\eta }}_t{\left(x_{1:t};\mathit{a}\right)}^{+},\\ -1,& \text{if}\mathrm{m}\mathrm{a}\mathrm{x}\left({\hat{v}}_t\left(x_{1:t};\mathit{a}\right),{\hat{\eta }}_t\left(x_{1:t};\mathit{a}\right)\right)<0,\end{array}\right.$$ (4.2)

for $t=1,\dots ,T-1$.

We now discuss how to tackle the “projection”, i.e., the conditional expectations. We propose two ways for estimating $\mu$ and $v$, namely local likelihood (see, e.g., Fan and Gijbels, 1996) and linear regression. Adoption of local likelihood is motivated by its flexibility, since an otherwise parametric regression of $2T-1$ models tracking ${\mu }_{1:T}$ and $v_{1:T-1}$ could be tedious and prone to misspecification. The recursive $v$ also accumulates estimation errors rapidly. On the other hand, linear regression could serve as a simple and interpretable alternative as long as it is not severely misspecified, as we later illustrate in Section 6.

For the local likelihood estimates, for any Lagrangian multiplier $\mathit{a}$, we recursively fit multiple models for each time step $t=T,\dots ,1$: at one given $x_{1:t}=\left(x_1,\dots ,x_t\right)$, we set ${\hat{\mu }}_t\left(x_{1:t}\right)={\hat{\mu }}_t\left(x_{1:t}\right)={\hat{\theta }}_{\mu ,t0}$ by

$$\left({\hat{\theta }}_{\mu ,t0},\dots ,{\hat{\theta }}_{\mu ,tt}\right)\in \underset{\theta }{\mathrm{a}\mathrm{r}\mathrm{g}\mathrm{m}\mathrm{i}\mathrm{n}}\sum _{i=1}^n{\left({\theta }_0+\sum _{s=1}^t{\theta }_s\left(X_{i,s}-x_s\right)-Y_i\right)}^2{K}_h\left(X_{i,1:t}-x\right);$$ (4.3)

set ${\hat{v}}_t\left(x_{1:t}\right)={\hat{v}}_t\left(x_{1:t};\mathit{a}\right)={\hat{\theta }}_{v,t0}$ by

$$\left({\hat{\theta }}_{v,t0},\dots ,{\hat{\theta }}_{v,tt}\right)\in \underset{\theta }{\mathrm{a}\mathrm{r}\mathrm{g}\mathrm{m}\mathrm{i}\mathrm{n}}\sum _{i=1}^n{\left({\theta }_0+\sum _{s=1}^t{\theta }_s\left(X_{i,s}-x_s\right)-{\hat{S}}_{i,t+1}\right)}^2{K}_h\left(X_{i,1:t}-x\right);$$ (4.4)

where we let $K_0$ be a kernel function, e.g., the density of the standard normal distribution, and define $K_h\left(\cdot \right):=h^{-1}{K}_0(\cdot /h)$ with bandwidth $h>0$. The ${\hat{S}}_{i,t+1}$ is obtained by plugging-in ${\hat{\mu }}_{t+1}\left(X_{i,1:t+1}\right)$ and ${\hat{v}}_{t+1}\left(X_{i,1:t+1}\right)$ into (3.5) and (3.7). Note that (4.4) is only necessary for $t=T-1,\dots ,1$. To estimate the entire functions ${\mu }_t$ and $v_t$, we solve the above weighted least squares problem for all points $x_{1:t}$ in the domain of interest.

Despite providing consistent estimates, the local likelihood estimates may not yield sensible decision regions when plugged into $\hat{f}$, especially in small sample settings. An intuitive example is to consider $T=2$ with univariate $X_1$, $X_2$, where higher covariate values are associated with higher risk. Naturally ${\tilde{f}}_1^{\mathrm{*}}$ would take the form of two cutoff thresholds: ${\tilde{f}}_1^{\mathrm{*}}=-1$ if $X_1<c_1$, ${\tilde{f}}_1^{\mathrm{*}}=1$ if $X_1>c_2$, and ${\tilde{f}}_1^{\mathrm{*}}=0$ otherwise. However, this is not necessarily the case for plug-in estimates that are nonlinear functions of the covariates, such as local likelihood estimates or even logistic regression. Since the estimates ${\hat{\eta }}_1$ and ${\hat{v}}_1$ could cross each other multiple times, especially when ${\eta }_1\approx v_1$, fragmented and counter-intuitive decision regions are possible. To aid interpretability, the local likelihood (as in (4.3) and (4.4)) may be replaced by linear regression. At each time step $t$, set ${\hat{\mu }}_t\left(x_{1:t}\right)={\hat{b}}_{\mu ,t0}+{\hat{b}}_{\mu ,t1}{x}_1+\cdots +{\hat{b}}_{\mu ,t}{x}_t$ and ${\hat{v}}_t\left(x_{1:t}\right)={\hat{b}}_{v,t0}+{\hat{b}}_{v,t1}{x}_1+\cdots +{\hat{b}}_{v,t}{x}_t$ for all $x_{1:t}=\left(x_1,\dots ,x_t\right)$, where

$$\left({\hat{b}}_{\mu ,t0},\dots ,{\hat{b}}_{\mu ,tt}\right)\in \underset{b}{\mathrm{a}\mathrm{r}\mathrm{g}\mathrm{m}\mathrm{i}\mathrm{n}}\sum _{i=1}^n{\left(b_0+\sum _{s=1}^t{b}_s{X}_{i,s}-Y_i\right)}^2$$

and

$$\left({\hat{b}}_{v,t0},\dots ,{\hat{b}}_{v,tt}\right)\in \underset{b}{\mathrm{a}\mathrm{r}\mathrm{g}\mathrm{m}\mathrm{i}\mathrm{n}}\sum _{i=1}^n{\left(b_0+\sum _{s=1}^t{b}_s{X}_{i,s}-{\hat{S}}_{i,t+1}\right)}^2.$$

While the framework is presented under univariate covariates $X_t\in R$ for all steps $t=1,\dots ,T$, it is straightforward to generalize to multivariate situation where $X_t\in R^{p_t}$ for $t=1,\dots ,T$. It suffices to replace the models for $\mu$ and $v$ by, e.g., multiple regressions. See Subsection S5.2 of the Supplement for details.

4.1. Translating Inconclusiveness to Cost for Ease of Inspection

When $T>2$, the additional objectives make it challenging to visualize the trade-offs between FPR and inconclusiveness due to the high dimensionality of the Pareto surface. To address this, we can use a consolidated criterion, such as a combined cost. For instance, step-wise inconclusiveness can be turned into a single cost value using a weighted sum. Suppose the cumulative cost for a patient underwent the first $t$ steps is $C_t$ and that $\mathbb{P}\left(f_{1:t}=0\right)={\gamma }_t$, then the expected cost to reach a diagnosis is $C_1\left(1-{\gamma }_1\right)+\dots +C_t\left({\gamma }_{t-1}-{\gamma }_t\right)+\dots +C_T{\gamma }_{T-1}$. Therefore, the equal-cost hyperplane $E_C:=\left\{\left({\gamma }_1,\dots ,{\gamma }_{T-1}\right):C=C_1\left(1-{\gamma }_1\right)+\dots +C_t\left({\gamma }_{t-1}-{\gamma }_t\right)+\dots +C_T{\gamma }_{T-1}\right\}$ represents all optimal sequential rules with expected cost $C$.

To find the optimal rule at a given cost $C$, it suffices to find the one along $E_C$ that minimizes FPR at $\mathrm{T}\mathrm{P}\mathrm{R}=\beta$. This is equivalent to problem (3.2) but replacing all inconclusiveness constraints by a cost constraint, whose solution we denote as ${{\phi }^{\mathrm{*}}}_C$. Therefore, drawing the FPR of ${{\phi }^{\mathrm{*}}}_C$ versus the cost over $C_1<C<C_T$ yields a one-dimensional Pareto front, that illustrates the optimal trade-off between FPR and cost, subject to a TPR constraint.

5. Performance Guarantee of the Estimated Rules

We now establish asymptotic performance guarantees for the plug-in rule, deriving convergence rates for its constraint violation and its excess FPR relative to the true optimal. Proofs and technical details are deferred to Section S2 of the Supplement.

Given any constraint values ${\gamma }_1,\dots ,{\gamma }_{T-1}$, and $\beta$, denote the constraint equations as

$$\begin{array}{l}{\mathrm{\Gamma }}_t(\phi )=\mathbb{P}\left(f_{1:t}\left(X_{1:t}\right)=0\right)-{\gamma }_t\text{for}t=1,\dots ,T-1,\\ {\mathrm{\Gamma }}_T(\phi )=\beta -\mathbb{P}\left(\phi \left(X_{1:T}\right)=1\mid Y=1\right).\end{array}$$

It can be shown that ${\mathrm{\Gamma }}_t(\tilde{\phi }(\cdot ,\mathit{a}))=\partial G/\partial a_t(\mathit{a})$ (see Proposition S1.2 in the Supplement), so that maximizing the dual function is equivalent to solving estimation equations ${\mathrm{\Gamma }}_t(\tilde{\phi }(\cdot ,\mathit{a}))=0$. For simpler theoretical development, we randomly split the sample into an estimation set and a validation set, each of size $m$ and $n$ respectively. We construct the plug-in estimators using the estimation set, then obtain the Lagrange multiplier by solving the empirical constraint equations on the validation set. That is, $\mathit{a}$ is picked so that ${\stackrel{\mathrm{ˆ}}{\mathrm{\Gamma }}}_t(\hat{\phi }(\cdot ,\mathit{a}))=0$ with

$$\begin{array}{ll}{\stackrel{\mathrm{ˆ}}{\mathrm{\Gamma }}}_t(\hat{\phi }(\cdot ,\mathit{a}))& ={\mathbb{P}}_n({\hat{f}}_{1:t}\left(X_{1:t},\mathit{a}\right)=0)-{\gamma }_t\text{for}t=1,\dots ,T-1,\\ {\stackrel{\mathrm{ˆ}}{\mathrm{\Gamma }}}_T(\hat{\phi }(\cdot ,\mathit{a}))& =\beta -{\mathbb{P}}_n\left(\hat{\phi }\left(X_{1:T},\mathit{a}\right)=1\mid Y=1\right),\end{array}$$

where $\left(X_{1:T},Y\right)\sim {\mathbb{P}}_n(\cdot )$ is the empirical distribution of the validation set that is independent from the estimation set used to fit $\hat{f}$ and $\hat{\phi }$. A direct implication of Hoeffding’s inequality indicates that the constraints are satisfied up to an arbitrarily small error.

Proposition 5.1.

For any $\epsilon >0,\mathbb{P}\left(\left|{\mathrm{\Gamma }}_t(\hat{\phi }(\cdot ,\mathit{a}))\right|\ge \epsilon \right)\le 2\mathrm{e}\mathrm{x}\mathrm{p}\left(-2n{\epsilon }^2\right)$ so that ${\mathrm{\Gamma }}_t(\hat{\phi }(\cdot ,\mathit{a}))=O_p(\sqrt{\mathrm{l}\mathrm{o}\mathrm{g}n}/n)$ as $n\to \mathrm{\infty }$, for $t=1,\dots ,T$.

Denote a sensible domain of $\mathit{a}$ as $𝒜$, so that for any $\mathit{a}\in 𝒜$, the TPR, FPR, and inconclusiveness of the corresponding rule $\tilde{\phi }(\cdot ,\mathit{a})$ are bounded away from 0 or 1. Denote $R_0(\phi )=\mathbb{P}\left(\phi \left(X_{1:T}\right)=1\mid Y=0\right)$ as the achieved FPR of any rule $\phi$. For an estimated rule $\hat{\phi }$, probability in $R_0(\hat{\phi })$ is taken under the conditional law of $X_{1:T}\mid Y=0$ given $\hat{\phi }$, so that $R_0(\hat{\phi })$ is a measurable function of the training sample. We assume:

(A1) $v_t-{\eta }_t^{+}$ and ${\eta }_t-v_t^{+}$ satisfy the margin condition of order $\bar{\rho }$ uniformly over $\mathit{a}\in 𝒜$, that is, there exist positive constants $M_0$ and $\bar{\rho }$, such that for any $\delta \ge 0$,

$$\underset{a\in 𝒜}{\text{sup}}\mathbb{P}\left(\left|{\nu }_t\left(X,\mathit{a}\right)-{\eta }_t^{+}\left(X,\mathit{a}\right)\right|\le \delta \right)\le M_0{\delta }^{\overline{\rho }},$$

so as for ${\eta }_t-v_t^{+},t=1,\dots ,T$, with the convention that $v_T\equiv 0$.

(A2) There exist $M,\underset{\_}{\rho }>0$, and a neighborhood $U$ of ${\mathit{a}}^{\mathrm{*}}$ such that ${‖\mathit{a}-{\mathit{a}}^{\mathrm{*}}‖}_{\mathrm{\infty }}\le M\Vert \mathrm{\Gamma }(\mathit{a}){\Vert }_{\mathrm{\infty }}^{\underset{\_}{\rho }}$ for all $\mathit{a}\in U$.

(A3) $\underset{\mathit{a}\in 𝒜}{\mathrm{s}\mathrm{u}\mathrm{p}}{‖{\hat{\eta }}_t(\cdot ,\mathit{a})-{\eta }_t(\cdot ,\mathit{a})‖}_{\mathrm{\infty }}=O_p\left(r_m\right)$ and $\underset{\mathit{a}\in 𝒜}{\mathrm{s}\mathrm{u}\mathrm{p}}{‖{\hat{v}}_t(\cdot ,\mathit{a})-v_t(\cdot ,\mathit{a})‖}_{\mathrm{\infty }}=O_p\left(r_m\right)$ as $m\to \mathrm{\infty }$ for all $t=1,\dots ,T$.

(A4) Conditioning on the estimation set, ${𝒵}_{1,t}=\left\{x\mapsto 1\left(\left({\hat{v}}_t-{\hat{\eta }}_t^{+}\right)(x,\mathit{a})>0\right):\mathit{a}\in 𝒜\right\}$ and ${𝒵}_{2,t}=\left\{x\mapsto 1\left(\left({\hat{\eta }}_t-{\hat{v}}_t^{+}\right)(x,\mathit{a})>0\right):\mathit{a}\in 𝒜\right\}$ are Vapnik–Chervonenkis (VC) classes whose VC dimensions are bounded by $V_m$ for $t=1,\dots ,T$.

Theorem 5.1.

Under (A1), (A2), (A3), (A4), and conditions of Proposition 3.2, as $n\to \mathrm{\infty }$ and $m\to \mathrm{\infty }$,

$$R_0(\hat{\phi })\le R_0\left({\phi }^{\mathrm{*}}\right)+O_p\left({\left(\frac{V_m\mathrm{l}\mathrm{o}\mathrm{g}{V}_m}{n}\mathrm{l}\mathrm{o}\mathrm{g}\frac{n}{V_m\mathrm{l}\mathrm{o}\mathrm{g}{V}_m}\right)}^{\underset{\_}{\rho }\bar{\rho }/2}+r_m^{\mathrm{m}\mathrm{i}\mathrm{n}(\bar{\rho },\underset{\_}{\rho }{\bar{\rho }}^2)}\right).$$ (5.1)

Condition (A1) is a Tsybakov’s margin condition to ensure only a small fraction of observations are “borderline”, so that small estimation errors in scores only alter the results for a small fraction of cases. Condition (A2) is a regularity assumption that links gaps in constraints to multipliers’ accuracy: if ${\mathrm{\Gamma }}_t(\mathit{a})$ are nearly zero, then $\mathit{a}$ must be close to ${\mathit{a}}^{\mathrm{*}}$. Similar conditions have appeared in performance analysis of plug-in Neyman–Pearson classifiers (e.g., Tong, 2013). Conditions (A3) and (A4) characterize the consistency and complexity of the plug-in estimators. They ensure both the plug-in rule and constraint equations are well estimated without overfitting. The first term in the rate of (5.1) arises from the standard VC complexity penalty, while the second term is tied to the consistency of the plug-in estimators. For example, if we adopt the Nadaraya–Watson estimator, which is a special case of the proposed local likelihood estimator, then $V_m=O(m\mathrm{l}\mathrm{o}\mathrm{g}m)$, and under the conditions of Theorem 8 in Hansen (2008), $r_m=h^2+\sqrt{\mathrm{l}\mathrm{o}\mathrm{g}m/\left(m{h}^T\right)}$ with bandwidth $h$. When the sample sizes $m,n\to \mathrm{\infty }$ and bandwidth $h\to 0$ are chosen in a way such that $V_m\mathrm{l}\mathrm{o}\mathrm{g}{V}_m/n\to 0$ and $r_m\to 0$, the FPR of the estimated rule is arbitrarily close to $R_0\left({\phi }^{\mathrm{*}}\right)$, the true optimum.

6. Simulation

We further evaluate the proposed method on simulated data with $T=2$ to illustrate (i) optimality of the learned Pareto front, (ii) constraint satisfaction for the target operating characteristics, and (iii) practical estimator selection. Covariates $\left\{X_t:t=1,\dots ,T\right\}$ were generated from a stationary Gaussian AR(1) process with autocorrelation 0.6. Outcomes were generated under two mechanisms (logistic and unimodal links). Unless otherwise specified, we trained on $n=\mathrm{1,000}$ observations with target TPR $\beta =0.9$ and a grid $\gamma \in \{0.05,\dots ,0.95\}$ of desired step-inconclusiveness, and evaluated achieved TPR, FPR, and step-inconclusiveness on independent test samples of size 10,000. Each setting was replicated 100 times. See Section S5 of the Supplement for setup details.

We compare the estimated Pareto front to the true optimum obtained by numerical integration. In Figure 6.1 (left panels), the local likelihood plug-in estimators adequately approximate the components ${\mu }_t$ and $v_t$, and the corresponding Pareto fronts closely track the true optimal tradeoff between FPR and the step-2 rate. As a built-in check, performance near $\gamma =0$ and $\gamma =1$ aligns with the non-sequential NP baselines using only $X_1$ and using $X_{1:2}$ respectively. Moreover, the second and third columns of Figure 6.1 show that the learned rules closely track the target constraints across $\gamma$. The achieved TPR remains close to $\beta =0.9$, and the achieved step-2 rates remain close to the prespecified levels.

Figure 6.1:

Under the logistic (top panels) or unimodal (bottom panels) link, the estimated Pareto front (left), achieved control over step 2 ratio (middle) and true positive rate (right), via either local likelihood or linear regression fitting, in comparison to the truth as well as Wald’s SPRT. All rules aimed for a TPR of 0.9. The vertical error bars at the margins of the left and right panels show the performance of non-sequential rules, utilizing only $X_1$ (at $\gamma =0$) or both $X_1$ and $X_2$ (at $\gamma =1$). Error bars and ribbons show 95% intervals over 100 repeated experiments, while the dots and lines are the corresponding means.

We also compare against Wald’s SPRT (see Section S3 of the Supplement). In this setting, SPRT can miss the target TPR because dependence between steps and the imposed finite horizon violate Wald’s assumptions, making its apparent FPR not directly comparable.

To guide the choice between linear regression and local likelihood plug-ins, we use the generalized cross-validation (GCV) score (e.g., Loader, 1999, Section 2.4), as a component-wise goodness-of-fit diagnostic (Figure 6.2). Under the logistic link (top row), the two estimators have very similar GCV across components, indicating that the linear model is adequate; correspondingly, linear plug-ins recover a reasonable Pareto front and stable constraint tracking in Figure 6.1. Under the unimodal link (bottom row), linear regression shows substantially larger GCV for ${\mu }_1$ and ${\mu }_2$, whereas local likelihood maintains lower GCV. This mismatch manifests as a distorted Pareto front, inflated variability, and even under-performance relative to non-sequential baselines in Figure 6.1. We note that GCV for $v_1$ under linear regression may appear overly optimistic because $v$ is constructed from the $\mu$-components, so misspecification in ${\mu }_2$ can propagate. Overall, we recommend using the simplest plug-in when GCV indicates adequate fit, and otherwise adopting local likelihood, with a quick sanity check that the learned Pareto front is sensible (correct behavior at $\gamma \in \left\{\mathrm{0,1}\right\}$, non-increasing FPR in $\gamma$, and small constraint gaps).

Figure 6.2:

Generalized cross-validation (GCV) scores comparing local likelihood and linear regression fits for ${\mu }_1$, ${\mu }_2$, and $v_1$ (columns) under the logistic (top row) and unimodal (bottom row) links. A point corresponds to one repeated experiment (for $v_1$, GCV is averaged over a grid of $\gamma$). Boxplots summarize 100 repeats. All rules targeted a TPR of 0.9.

7. Efficient Detection for Clinically Significant PCa

We now return to our motivating data, and aim to reduce the overall testing burden and cost by optimizing the administration of tests while attaining a low under-detection risk, targeting a TPR of 90%. Given differences in cost and ease of implementation, we consider a class of sequential testing strategies structured with the following steps: (1) all patients are first assessed with PCPT; (2) those without a conclusive decision are referred for MPS2; (3) those still unresolved are referred for MRI to make the final decision on biopsy. In sum, a proportion of ${\gamma }_1$ patients would only undergo PCPT and MPS2, while a proportion of ${\gamma }_2$ patients would undergo PCPT, MPS2, and MRI. Therefore, such inconclusiveness can be interpreted as the utilization of MPS2 and MRI, respectively. Since additional testing improves diagnostic accuracy but increases cost and burden, we adopt the proposed multi-objective Pareto optimization framework to balance competing goals: maximizing diagnostic accuracy and minimizing total cost using MPS2 and MRI. For ease of interpretation, we use linear regression plug-in estimators (see Section 7.3 for justification). Model performance was assessed using 5-fold cross-validation. Average values and 95% CIs were calculated and reported based on 100 bootstrap samples.

7.1. Findings

Figure 7.1 summarizes the performance of the derived 3-step sequential strategies. Figure 7.1a displays the FPR across combinations of MPS2 and MRI utilization rates (${\gamma }_1$ and ${\gamma }_2$) as background raster, overlaid with cost contours (dotted lines). The cost contours were drawn following Section 4.1: when $T>2$, there are multiple step-utilization rates, which are hard to visualize jointly. For easier inspection, we summarize them by a single expected cost, computed as a weighted average of cumulative costs, where the weights are the fractions of patients who stop at each step. From this, we identify the optimal (${\gamma }_1$,${\gamma }_2$)-combination that yields the lowest FPR while achieving approximately 90% TPR for each cost level. An example is shown in Figure 7.1c, where at the cost level around $2,000, the FPR-minimizing rule corresponds to 95% MPS2 utilization (red cross). Pooling these optimal points across different cost levels forms the Pareto front in Figure 7.1b, which traces the trade-off between cost and FPR. An observation is the identification of a “sweet spot” (red crosses in Figure 7.1), where a 3-step strategy on average achieves an FPR of 0.48 at a cost of $2,031. This is comparable to the non-sequential protocol using all three tests (the last row of Table 2.2) at a substantially lower cost.

Figure 7.1:

3-step protocols utilizing PCPT, MPS2, and MRI. The red crosses in the three panels indicate the sweet spot: an optimal sequential rule that achieves oracle performance at minimal cost.

Figure 7.2 illustrates the 3-step sequential decision logic, comparing the benefit of immediate action versus waiting at each step, using the utility scores ${\eta }_t$ and $v_t$ defined in (3.5)–(3.8). The coefficients for computing these scores as linear functions of the test results are presented in Table 7.1.

Figure 7.2:

Visualization of the 3-step decision process via comparisons between $\eta \mathrm{s}$ and $v\mathrm{s}$. Each quantity is a linear combination of test results at the corresponding step, using coefficients detailed in Table 7.1.

Table 7.1:

Coefficients (average values and 95% CIs) for 3-step strategies using PCPT, 95% MPS2 and 28.5% MRI over 100 repeated experiments. $\eta$ and $v$ at step $t$ are linear scores of combining related variables with corresponding coefficients.

Component	Step t	Variable	Coefficient
ηt	1	(Intercept)	−0.13 (−0.7, 0.3)
		PCPT	13.88 (9.5, 20.8)
	2	(Intercept)	−0.77 (−1.3, −0.4)
		MPS2	5.54 (3.9, 7.3)
		PCPT	6.01 (2.1, 11.6)
	3	(Intercept)	−1.94 (−2.5, −1.3)
		MPS2	4.65 (3.2, 6.4)
		MRI	2.07 (1.4, 2.7)
		PCPT	5.22 (1.8, 10.4)
vt	1	(Intercept)	0.03 (−0.3, 0.3)
		PCPT	13.32 (9.0, 20.0)
	2	(Intercept)	−0.59 (−1.1, −0.3)
		MPS2	4.83 (3.2, 6.5)
		PCPT	5.51 (2.0, 10.7)

7.2. A Parsimonious Two-Step Protocol

Interestingly, the selected 3-step “sweet spot” suggests that PCPT is marginally informative, reflected by ${\gamma }_1=0.95$, indicating that 95% of patients proceed to MPS2. This aligns with the non-sequential results in Table 2.2, where PCPT adds little value over a two-test combination of MPS2 and MRI. These findings motivate further analysis of simplified 2-step strategies using only MPS2 and MRI.

Figure 7.3 presents the estimated Pareto front (left panel), achieved TPR (middle), and variations in achieved MRI utilization (right) across a range of desired MRI utilization. The Pareto front exhibits an elbow pattern, suggesting diminishing marginal returns: beyond 25% MRI utilization, further reductions in FPR are minimal. This trend is confirmed by the results in Table 2.2 (second-to-last row) and Table 7.2 (last two rows), where FPRs remain comparable across 2-step protocols with MRI proportions ranging from 25% to 100%. Notably, near-oracle performance can be achieved with roughly half the patients undergoing MRI. A similar elbow pattern is also visible in the 3-step protocols (Figure 7.1b), suggesting that we can halve the cost without a significant impact on the decision quality.

Figure 7.3:

For the 2-step sequential protocol utilizing MPS2 and MRI, the average values (lines) and point-wise 95% confidence bands (ribbons) of the estimated Pareto front (left), achieved TPR (middle), and error between achieved and desired MRI utilization (right). Vertical bars at the margins of the left and middle panels indicate the performance of non-sequential protocols, e.g., utilizing MPS2 alone (MRI utilization = 0%, left margins) or both MPS2 and MRI (MRI utilization = 100%, right margins). Results are based on 100 repeated experiments.

Table 7.2:

Average performance (and 95% intervals) of sequential protocols over 100 repeated experiments. For example, “MPS2 + 25% MRI” denotes a proposed protocol that measures MPS2 for all patients and send 25% of them to further MRI.

Protocol	Cost	FPR%	TPR%
PCPT + 95% MPS2 + 28.5% MRI	2027 (1865, 2199)	48 (35, 63)	90 (88, 92)
MPS2 + 25% MRI	1857 (1707, 2022)	48 (34, 61)	90 (89, 92)
MPS2 + 40% MRI	2516 (2374, 2641)	47 (36, 57)	90 (88, 92)

Moreover, because the protocol has only two steps, the decision logic rule can be written in a particularly transparent form. In step 1, it simply compares the MPS2 value against two cutoffs, with values between the cutoffs treated as inconclusive and sent to MRI. For those receiving MRI, the final decision is based on a single linear score, i.e., a weighted sum of the available predictors, compared to a cutoff. The corresponding thresholds and coefficients are summarized in Table 7.3.

Table 7.3:

Results for two selected 2-step protocols with 25% or 40% MRI utilization. Lower and upper thresholds (95% CIs) for step 1 and coefficients (95% CIs) in ${\eta }_2$ are presented based on 100 repeated experiments.

	Step 1: compare MPS2 to thresholds		Step 2: η2 > 0 ?
MRI%	Lower	Upper	Intercept	MPS2	MRI
0.25	0.04 (0, 0.1)	0.15 (0.1, 0.2)	−1.59 (−2, −1.1)	5.18 (3.6, 6.9)	2.08 (1.4, 2.7)
0.40	0.03 (0, 0.1)	0.23 (0.2, 0.3)	−1.59 (−2, −1)	5.15 (3.5, 7)	2.06 (1.4, 2.6)

These results highlight the potential of a simple yet cost-efficient diagnostic strategy: a sequential rule that uses the biomarker(s) up front and reserves MRI for patients whose initial results fall within an “uncertain” middle range. In terms of diagnostic accuracy, the learned sequential protocol can be tuned to meet the prespecified TPR while keeping the FPR close to the full non-sequential three-test strategy. At the same time, because MRI is triggered only for the subset of patients who remain inconclusive after biomarkers, the protocol substantially reduces MRI utilization and therefore the testing burden, with an average cost of $1,857 per patient.

7.3. Stability and Model Specification Checks

Our plug-in framework permits flexible choices for estimating the intermediate regression components used to construct the sequential decision rule. In the real-data analysis, we conducted two sensitivity analyses to evaluate the robustness of our approach.

We first examined the methods’ sensitivity to potential outliers, considering both the fitted model components and the resulting sequential rule, with a focus on the two biomarkers. Because PCPT is positive and MPS2 takes values in (0,1), we applied a log transform to PCPT and a centered log-ratio transform ($x\mapsto \mathrm{l}\mathrm{o}\mathrm{g}x/2-\mathrm{l}\mathrm{o}\mathrm{g}(1-x)/2$) to MPS2, and denote the transformed biomarkers vector as $Z\in R^2$. After such transformation, the resulting bivariate distribution was well-approximated by a bi-variate Gaussian model (Figure 7.4a, Henze–Zirkler test $p=0.11$).

Figure 7.4:

Observations with squared Mahalanobis distance exceeding the 0.99 quantile of ${\chi }_2^2$ are flagged as potential outliers (red).

Under a bi-variate Gaussian model, the squared Mahalanobis distance, $d=(Z-\bar{Z}{)}^T{\mathrm{\Sigma }}^{-1}(Z-\bar{Z})$, where $\bar{Z}$ and $\mathrm{\Sigma }$ denote the sample mean vector and covariance matrix respectively, is asymptotically ${\chi }_2^2$-distributed. We flagged observations with $d$ exceeding the 0.99 quantile of ${\chi }_2^2$ as outliers (Figure 7.4b). We re-fit the procedure with and without those potential outliers. The resulting fitted components and held-out-sample performance (Figure 7.6) were stable, and the selected rules were essentially unchanged (see Section S6 of the Supplement).

Figure 7.6:

Average differences in absolute constraint gaps relative to the LM baseline with all observations retained.

Second, we compared a simple linear model-based estimator (LM) with more flexible local likelihood-based estimators (Locfit, using local polynomials of degree 0 or 1) to assess whether the added flexibility leads to materially different results. Across bootstrap samples, the GCV scores were largely similar across the three estimators (Figure 7.5), indicating that additional estimator flexibility yields only limited improvement on this dataset. To assess whether modeling choices affect decision-relevant quantities, we compared the empirical constraint gaps on held-out data in Figure 7.6. For example, let gap_TPR denote the out-of-sample absolute TPR constraint gap, the leftmost panel summarizes ${\mathrm{\Delta }}_{\text{TPR}}=\mid {\mathrm{g}\mathrm{a}\mathrm{p}}_{\mathrm{T}\mathrm{P}\mathrm{R}}^{\mathrm{l}\mathrm{o}\mathrm{c}\mathrm{f}\mathrm{i}\mathrm{t}}|-|{\mathrm{g}\mathrm{a}\mathrm{p}}_{\mathrm{T}\mathrm{P}\mathrm{R}}^{\text{l}\mathrm{m}}\mid$, where ${\mathrm{\Delta }}_{\text{TPR}}>0$ indicates Locfit has a larger TPR constraint gap than LM, and vice versa. The empirical gaps in the MPS2 and MRI utilization were again largely similar under LM and Locfit, while LM tended to satisfy the TPR constraint more closely.

Figure 7.5:

Generalized cross-validation (GCV) scores comparing linear regression (LM, top row) and local likelihood with different polynomial degree (Locfit, bottom two rows) fits for ${\mu }_{1:3}$ and $v_{1:2}$ (columns). A point corresponds to one bootstrapping experiment (for $v_{1:2}$, GCV is averaged over a grid of $\gamma$). Boxplots summarize 100 repeats. All rules targeted a TPR of 0.9.

These comparisons were qualitatively unchanged when repeating the analysis with flagged outliers retained versus excluded (red versus blue in Figure 7.5 and Figure 7.6). We therefore report results using all observations and the LM estimator, which is simpler, yields more interpretable fits, and performs at least as well as local likelihood on both fit adequacy and constraint satisfaction.

8. Discussion

Our proposed sequential testing framework, developed with a multi-objective optimization perspective, naturally extends the Neyman–Pearson paradigm to sequential decision-making. By imposing separate objectives on clinical performance and step-wise inconclusiveness, it enables more nuanced control over the diagnostic process without requiring prespecified weighting parameters. The closed-form solution offers an interpretable, step-by-step protocol that balances confidence in existing evidence against the potential benefit of deferring decisions to later stages with additional information. The plug-in estimation scheme further enhances flexibility, accommodating models ranging from simple linear forms to more complex nonparametric alternatives.

Visualization of the resulting Pareto front illustrates the incremental value of each test, revealing opportunities to forgo unnecessary steps without sacrificing diagnostic accuracy. Our application to prostate cancer diagnosis demonstrates that data-driven sequential strategies can significantly reduce diagnostic cost and burden while preserving high accuracy. In particular, a simplified 2-step protocol using MPS2 and selective MRI performs comparably to more complex approaches, underscoring its clinical practicality and efficiency.

Our approach is broadly adoptable in many settings. For example, for disease active surveillance, a single diagnostic test may be performed repeatedly to monitor an adverse condition. An accurate decision rule with the earliest decision is desired. The strong correlation among sequential longitudinal results might incur additional complexity. Exploring machine learning methods, such as deep learning, may offer a flexible tool for dimensionality reduction in long serial measurements. Additionally, unlike dynamic treatment regimes, in this framework, we assume a decision rule for a diagnostic test that detects but does not alter the existing disease condition. However, in prognostic settings where detection may trigger early intervention and subsequently affect future disease outcomes, further development under a causal framework is warranted.

Supplementary Material

Supplement to “Optimizing Sequential Decision Rules for Prostate Cancer Biopsy Management: A Multi-Objective Statistical Framework”: Proofs of the theoretical propositions. Additional details of the simulation and real-world examples. (PDF file)

Data Availability Statement

The data that support the findings of this study include de-identified preliminary results with added perturbations from an ongoing study conducted by the National Cancer Institute’s Early Detection Research Network. The original study data are not publicly available due to data use agreements. A synthetic dataset and the code to reproduce the simulation analyses are openly available at https://github.com/jiamingqiu/seqdx_jasa_repo.