Patient-Preference Diagnostics: Adapting Stated-Preference Methods to Inform Effective Shared Decision Making

INTRODUCTION

Shared decision making involves providing evidence-based information about treatment options, systematic patient-preference elicitation, and decision support consistent with patient concerns.^{1, 2} While clinical practice guidelines and guidance statements underscore the need to incorporate patient preferences in clinical decision making, meaningful assessment of patient preferences in clinical encounters presents providers with significant challenges.^{3, 4} These challenges include the ever-increasing clinical and administrative tasks required within the constraints of short clinic visits.^{5, 6} Other challenges include lack of training or experience in assessing patient preferences and potential for unconscious biases that can affect how providers discuss treatment alternatives.^{7, 8} From the patients’ perspective, lack of prior knowledge about alternatives or unfamiliarity with medical terminology can cause them to defer to provider preferences.⁹ This may be problematic as clinicians cannot reliably predict individual patient preferences for treatment processes, treatment outcomes, or potential treatment-related risks.^{10, 11} This is even more relevant when evidence-based treatment recommendations are based on medical indications and physicians are likely to focus only on patient symptoms, health history, and risk factors, ignoring the role of preferences that need not correlate with these characteristics.

Approaches that combine patient preferences and clinical evidence can facilitate effective patient-provider communication and more patient-centric healthcare decisions.^{12, 13} Some interactive, electronic decision tools have used stated-preference or conjoint-analysis methods to elicit patient preferences and to derive preference weights for treatment features and outcomes. Such quantitative preference measures can help identify an evidence-based choice of treatment that also incorporates patient concerns.^13–16 The use of conjoint analyses in decision tools is relatively new and clinicians report challenges in the time and effort required for implementation.¹⁷ There is, however, increased interest in using these tools to more formally derive individual-level preferences to support shared decision making.¹⁸

Eliciting individual-level preferences using conjoint analysis has two important limitations. First, assuming providers start with little or no information on an individual’s preferences, the elicitation tool must ask multiple questions to derive definitive preference weights with acceptable measurement error. In the interest of speed and simplicity, researchers often collect too little preference information, which can limit its reliability and usefulness to support clinical decisions.¹⁹ Others employ sophisticated adaptive algorithms that extract more information per question, but still may require numerous questions to obtain reliable results.²⁰

For some conditions there already is evidence about the distribution of patient preferences in the population or clinically-relevant patient types that can be used to make this problem more tractable. We propose a structured approach to efficiently measure how distant individual patient preferences are from what is known about preferences across the patient population, much in the same way tools are used to diagnose specific disease conditions. This approach leverages prior information on patient preferences to generate an adaptive, personalized series of questions. The adaptive process generates questions to identify a patient-preference phenotype by limiting questions to only those that are most informative about the proximity of a patient’s preferences to a known preference phenotype. This information can be used for measuring sensitivity and specificity, much like any other diagnostic procedure.

Our objective is to develop a pragmatic approach that is simple enough to use in clinical settings. To achieve that goal requires a means of diagnosing the preference phenotype for a patient with a minimum number of questions and with a transparent indication of the confidence level for the diagnosis. In this article we derive a diagnostic algorithm and provide an example to illustrate the first step in developing a preference-diagnostic tool. We used results from a preference study of patients with first-time anterior shoulder dislocation (FTASD). This work is critical for future research as there is a lack of a formal procedure to diagnose patient preferences. The diagnostic tool will be validated clinically in a subsequent study.

The relevance of an adaptive design based on the preferences of the population

The fundamental premise of our approach is that we learn about the distribution of preferences in a population through valid preference research or that the context of clinical decision making provides a relevant preference phenotype to diagnose. With that as a starting point, we can focus the questions in a preference-elicitation instrument such as a discrete-choice experiment (DCE) to identify the likely location of an individual’s preferences within that distribution or their proximity to the relevant phenotype. The approach can support shared decision-making in at least three important areas.

First, the approach could narrow the preference-concordant treatment options in situations where no alternative is clearly superior from a clinical standpoint. In such a case, a preference-diagnostic tool can help prioritize options based on patient views about the more desirable or acceptable treatment features. This perhaps is the most common application of currently available shared-decision aids.²¹

Second, the approach can cue a provider to proactively address issues that are considered most important to patients. For example, if a treatment has a relatively high incidence of a side effect that is particularly worrisome to some patients, the diagnostic tool could identify these patients and alert the providers that additional action may be needed to promote treatment adherence and to maximize treatment effectiveness.

Third, the approach can document the correspondence between patient preferences and treatment decisions in the context of regulatory or reimbursement requirements. For example, suppose a regulatory agency accepts evidence on patient preferences to determine that the balance of benefits and risks of a treatment are acceptable to some patients. The preference-diagnostic tool then could help document the diagnosis of an acceptable balance of benefits and risks.

Constructing a preference-diagnostic tool

Figure 1 presents the steps of the adaptive process to construct the proposed preference-diagnostic tool. First, we identify relevant preference phenotypes. For this we can use known information about the distribution of preferences from a valid quantitative preference study of the patient population or a preference phenotype that is considered relevant given the clinical context.¹³ When using previous empirical work, we identify groups of patients with similar preferences using analytic tools such as k-means cluster analysis, latent-class analysis, or hierarchical cluster analysis (HCA).^22–24 Based on the phenotype(s) identified, we identify a small number of choice questions that maximize discrimination among groups. We account for potential random errors that patients could make in answering these questions. Determining whether any given level of accuracy is sufficient is beyond the scope of this manuscript and would require validation of the tool with users. The validation process would entail determining the diagnostic receiver-operating characteristics (ROC) curve for the tool with members of the relevant population. If diagnostic accuracy is considered sufficient, the preference-diagnostic tool could be implemented in clinical-care pathways to support shared decision making.

Figure 1.

Steps in developing a preference diagnostic tool based on population-level preferences

To better explain the process of constructing a design for a preference-diagnostic tool, consider a two-alternative scenario such as that shown in Figure 2 where alternative A is a dominant option. This means that all the treatment-feature levels in A are strictly superior to those in B, and therefore, we expect that the choice between A and B would be the same regardless of respondent’s preferences (i.e., respondents always should choose A). Thus, this set of alternatives offers very poor discrimination between types of preferences. An answer to the dominated question tells us very little about whether a patient is more concerned about greater efficacy, lower risk, or lower cost. To use choices to effectively discriminate among kinds of patient preferences, we would need a question for which alternative A is closely related to the preferences of one type of patient and alternative B is closely related to the preferences of a different type of patient. This way, respondents with the first type of preferences would nearly always choose A and patients with the second type of preferences nearly always would choose B.

Figure 2:

Example choice question

Formally, assume that choice between alternatives A and B is determined probabilistically by the relative utility of the alternatives following a simple conditional logit probability density,²⁵ such that:

$$P(C=A\mid A,B)=\frac{e^{U_A}}{{\sum }_i{e}^{U_i}}\text{for}i=A,B$$ (1)

Further, assume that utility of alternative $i$, ${\text{U}}_i$, is a function of the attributes of the alternatives. We assume this function is a linear combination of a set of attributes in each alternative, such that:

$$U_i={\sum }_k{\beta }_k{x}_{ik}$$ (2)

where betas are attribute-specific weights representing the marginal utility of k attributes. For simplicity, we assume that all attributes are continuous and have constant marginal effects. This assumption, however, does not affect the generalizability of our argument. We assume the functional form of ${\text{U}}_i$ matches the clinically-relevant preference phenotypes. The relative size of these marginal utilities is a measure of preferences. We can calculate the probability of choosing between A and B for each preference phenotype $j$ by applying Equation (2) using betas that match the preferences of phenotype $j^1\left({\beta }_{kj}\right)$.

$$U_{ij}=\sum _k{\beta }_{kj}{x}_{ik}$$ (3)

and applying Equation (1) for each $U_{ij}$.

With the relevant choice probability at hand, it is possible to evaluate the probability that someone is a member of that phenotype $j$ based on Bayes Rule as presented in Equation (4).

$$P(J=j\mid C=i)=\frac{P(J=j)P(C=i\mid J=j)}{P(C=i)}$$ (4)

Where $P(J=j\mid C=i)$ is the probability of having phenotype $j$ given a specific choice $i$. This is a function of $P(J=j)$ which is the prior probability of having preference phenotype $j$ and $P(C=i\mid J=j)$ which refers to the probability of choice $i$ conditional on having preference phenotype $j$. $P(C=i)$ is the unconditional probability of choices between specific alternatives. Conceptually, Equation (4) uses the information on specific preference phenotypes to understand the choices expected from such phenotypes. This information in turn makes it possible to evaluate how likely it is that an individual belongs to a specific preference phenotype if they make a given choice among alternatives.

We obviate the notation for the alternatives in Equation (4), but each choice probability also is conditional on the alternatives presented to the respondent. This is because the probability of choice given a preference phenotype is defined by Equation (1). Thus, the conditional choice probabilities are a function of the preference weights, but also the attributes of the alternatives. A more detailed derivation of this result is included in Appendix A.

Priors for probabilities of an individual phenotype $(P(J=j))$ can be uninformed or informed. If we assume a naïve prior about an individual’s phenotype, then they can be assumed to have the same probability of being in any of the relevant phenotypes before choices are observed. An alternative approach would be to inform that prior using results from previous preference work or expert opinion. It also is possible to account for known correlations between observable respondent characteristics and preferences to adjust prior probabilities for individuals.

Designing questions to diagnose preference phenotypes

To maximize the diagnostic power of choice questions we need a formal discrimination criterion that guides the search for the most appropriate sets of constructed alternatives represented in the choice questions in the choice questions. We define this criterion to be the following:

$${\lambda }_{ii}=(P(J=j\mid C=i))(P(J=j\mid C=i))={(P(J=j\mid C=i))}^2$$ (5)

Note that this is the squared value of the phenotype probability given choice $i$. If we are changing the alternatives in a choice question, what we are modifying is the likelihood of choice given phenotype in Equation (4). Thus, the criterion must be sensitive and have useful properties for changes in the likelihood. We derived this criterion based on the properties of Bayesian classification (see Appendix A).

Note that when the likelihood is 0.5, the posterior probability equals the prior, as no new information is provided. Also note that the expectation of classification for phenotype $j$ tends to 1 when the likliehood of choice approximates 0 or 1. In words, the more certain we are about someone’s choice between alternatives, the more more certain we are about their phenotype. Hence, this form of discrimination is bounded between $P(J=j)$ (the prior probability of a phenotype or no additional discrimination) and $1$ (complete discrimination). When the score is at its minimum, choosing alternative $i$ is uninformative as it does not change the probability that someone presents a phenotype of interest. When the score is 1 choosing $i$ increases the probability that someone presents a specific phenotype to 1. For multiple phenotypes, one can simply add all individual phenotype scores.

While the score for a particular alternative indicates the probability that someone presents a phenotype, it does not preclude the possibility that the same alternative is associated with multiple phenotypes. To account for this, we can organize relationships across phenotypes with the following I x I matrix:

$$\text{Λ}=\left[\begin{array}{ccc}{\lambda }_{11}& \dots {\lambda }_{1i}\dots & {\lambda }_{1I}\\ ⋮& \ddots & ⋮\\ {\lambda }_{I1}& \dots {\lambda }_{Ii}\dots & {\lambda }_{II}\end{array}\right]$$ (9)

Where $I$ is the number of choice alternatives. Off-diagonal elements of $\text{Λ}$,

$${\lambda }_{AB}={\lambda }_{BA}={\sum }_j(P(J=j\mid C=A))(P(J=j\mid C=B))$$ (10)

are cross-phenotype scores that range between 0 (alternatives are not associated with the same phenotype) and 1 (alternatives are perfectly concordant with the same phenotypes).

Also note that we can construct $\Lambda$ by calculating $M^{\prime }M$, where $M$ is an $I\times J$ matrix with the conditional probabilities of phenotypes given choice or:

$$M=\left[\begin{array}{ccc}P(J=1\mid C=1)& \cdots P(J=j\mid C=1)\cdots & P(J=J\mid C=1)\\ ⋮& \ddots & ⋮\\ P(J=1\mid C=I)& \cdots P(J=j\mid C=I)\cdots & P(J=J\mid C=J)\end{array}\right]$$

The objective of a design that optimizes the discrimination between patient phenotypes for the two-alternative case would be to find profiles A and B that make the diagonal elements of $\text{Λ}$ as close as possible to 1, while making the off-diagonal elements as close as possible to zero. In other words, optimizing discrimination in a question requires that $\Lambda$ approach an identity matrix. When considering two alternatives, we can optimize $\text{Λ}$ by varying the attributes in A and B until the determinant of $\text{Λ}$ is as close as possible to 1. A transformation of the determinant, such as $-{(det(\text{Λ})-1)}^2$, can be used so the global maximum is achieved at the identity matrix.

It is unlikely that a single choice question would be sufficient for the $\Lambda$ matrix to become an identity matrix. Limitations on possible combinations of attribute levels and the degree of variability group preferences generally will make that infeasible. Thus, the optimization problem requires asking the fewest questions required to identify respondent classification with an acceptable level of accuracy. In the next section, we discuss how to link a series of $\Lambda$ matrices in a series of choice questions using a recursive Bayesian approach.

Learning across questions – Recursive Bayesian approach

While optimizing a single $\Lambda$ would provide the most information with a single choice question, building a Bayesian link between questions can take advantage of a sequence of choices to achieve acceptable discrimination levels among phenotypes. A recursive Bayesian approach offers a systematic way to update phenotype priors based on what we learn from answers to previous choice questions.²⁶ This approach accepts the posterior consistent with the choice elicited in a given question as the prior phenotype probability for $\Lambda$ in subsequent questions. Doing so results in progressively better priors for devising progressively more discriminating questions.

For example, a two-alternative two-phenotype scenario has the following initial discrimination score matrix ${\Lambda }^{\prime }$:

$${\Lambda }^{\prime }=\left[\begin{array}{cc}\sum _j{(P(J=j\mid C=A))}^2& \sum _{j}P(J=j\mid C=A)P(J=j\mid C=B)\\ \sum _{j}P(J=j\mid C=A)P(J=j\mid C=B)& \sum _j{(P(J=j\mid C=A))}^2\end{array}\right]$$

After solving for profiles A and B in ${\Lambda }^{\prime }$, we can use the elements in it to replace the priors for each class and solve a new version of lambda, ${\Lambda }^{″}$.

Note, however, that the specific prior will depend on the answer to the choice question developed through the optimization of ${\Lambda }^{\prime }$. Since we know the probability that choices are associated with each phenotype, recording a specific choice updates the prior. The new prior is the calculated probability of each phenotype given the observed first choice. Thus, ${\Lambda }^{″}$ can be defined as follows:

$${\text{Λ}}^{″}=\left[\begin{array}{cc}\sum _j{\left(\frac{{\rho }_j^{\prime }P(C=A\mid J=j)}{P(C=A)}\right)}^2& \sum _j\left(\frac{{\rho }_j^{\prime }P(C=A\mid J=j)}{P(C=A)}\right)\left(\frac{{\rho }_j^{\prime }P(C=B\mid J=j)}{P(C=B)}\right)\\ \sum _j\left(\frac{{\rho }_j^{\prime }P(C=A\mid J=j)}{P(C=A)}\right)\left(\frac{{\rho }_j^{\prime }P(C=B\mid J=j)}{P(C=B)}\right)& \sum _j{\left(\frac{{\rho }_j^{\prime }P(C=A\mid J=j)}{P(C=A)}\right)}^2\end{array}\right]$$

Where

$${\rho }_j^{\prime }=\frac{P(J=j)P\left(C=C^{\prime }\mid J=j\right)}{P\left(C=C^{\prime }\right)}$$

is the probability that the respondent has preferences consistent with phenotype $j$ given their choice in the previous question $\left(C^{\prime }\right)$, which would be A or B. Note that with this approach we can chain as many lambda matrices as we need to discriminate between phenotypes. Also note that, at a minimum, we need $q$ questions with $I$ alternatives to discriminate between $I^q$ phenotypes. Thus, with two-alternative questions, we would need a minimum of 1 question for 2 phenotypes, 2 questions for 4 phenotypes, 3 questions for 8 phenotypes, and so on.

While it is possible to achieve discrimination to a desired level by adding questions that evaluate the same preference phenotypes, it may be desirable to systematically focus on the classification of smaller groups of phenotypes across questions. For example, if there are 4 phenotypes of interest in the patient population, we could have an initial question that discriminates between two pairs of phenotypes, say group L1 and group L2. A second question could then be optimized to further distinguish between phenotypes within L1 and L2.

It is important to note that this approach would require using modal probabilities for the assignment of an individual to a phenotype group. It also assumes that only the phenotypes within a group are relevant for the optimization of subsequent questions. For a more detailed discussion on how this can be accomplished, please refer to Appendix B.

Case study - Treatment of first-time anterior shoulder dislocation (FTASD) BACKGROUND

To evaluate the approach described above, we relied on data from a recent study on treatment of FTASD that elicited patient preferences for various treatment features.¹³ FTASD management involves either nonoperative treatment with immobilization, operative treatment with arthroscopic stabilization or open repair.^{27, 28} In the previous study, individual-level preferences were estimated from an adaptive conjoint analysis (ACA) survey (N=200) with 8 graded-pair questions on four attributes: limitations in arm motion after surgery, need for avoidance of contact sports and lifting overhead, chance of another dislocation, and out-of-pocket cost of treatment. Table 1 presents the attribute table from the survey.

Table 1.

Attribute table for original preference study

Attribute	Attribute level
Limitation in motion	No limitations Not able to raise arm above shoulder (No lifting) Need to use sling for a month (Sling 1 month)
Time to avoid activities	1 month 3 months 1 year
Risk of second dislocation	5 out of 100 (5%) 20 out of 100 (20%) 80 out of 100 (80%)
Out-of-pocket cost	$0 $100 $200

Although all three interventions are considered effective, there is no superior treatment for most patients. Ultimately, the decision to choose one of these options depends on the patient’s perspectives on the relative importance of the risk of another injury, their ability to use their arm while recovering, how long patients need to avoid engaging in sports, and the cost of the procedure.^{13, 29–31} We identified groups with similar preferences using the original data to construct a preference-diagnostic tool. We then simulated the sensitivity and specificity of the tool based on various assumed levels of response errors.

METHODS

We used a hierarchical cluster analysis with individual-level preferences using Ward’s method, which minimizes within-cluster variation based on Eucledian utility-scaled distances between preference points, to identify a tree of four respondent types grouped into two groups of preference phenotypes within the study sample.³² The resulting tree structure and the average preferences for patients in each group were used to generate a diagnostic design that was optimized to discriminate between respondent phenotypes. We generated these questions by setting up the minimum number of questions needed to identify 4 patient phenotypes. While closeness of phenotype preferences can influence the number of questions needed to identify a phenotype reliably, we determined that two choice questions were sufficient given the hierarchical nature of our phenotype grouping. This, however, is a decision that could change with the application. Having two choice questions meant having an initial choice question presented to all respondents to establish the probability that a respondent was in one of two phenotype groups (each with two patient types), and a second choice question to identify the specific preference phenotype. Thus, we needed to design a total of 3 choice questions. We used an evolutionary algorithm in R (Global Optimization by Differential Evolution³³) to identify the treatment alternatives that optimized the discrimination score for the three choice questions. To avoid solutions that were highly discriminatory but had a very skewed choice distribution², we added a penalty for skewness to the determinant of the lambda matrix. This penalty took the form of $2\sigma$, where $\sigma$ was the variance of the diagonal elements in the lambda matrix, or the within phenotype discrimination score. A copy of the code generating the design is provided in Appendix C.

We used the links in the recursive Bayesian approach described before to generate choice questions to diagnose each phenotype. Because the design was prepared following a hierarchical structure, each question classifies the respondents in increasingly granular categories. Thus, even one choice is enough to start diagnosing respondents’ preference type.

To evaluate the performance of the diagnostic questions, we simulated responses from 10,000 hypothetical respondents randomly assigned to each of the 4 classes. We assumed different error variances in the responses, corresponding to how often an incorrect choice was elicited from the hypothetical respondents. We then evaluated the sensitivity of misclassification to the size of the error variance. We specified error variances by drawing errors from a uniform distribution between 0 and $X$ on the choice probabilities, where $X$ increased with the error variance. Details on the mechanics of the simulations are provided in Appendix C (as part of the program provided) and Appendix D.

RESULTS

We identified four patient types in the FTASD study data. The relative attribute importance for the four preference phenotypes is shown in Figure 3. The importance plot shows the relative influence that each attribute had on the treatment choice in the FTASD study using profile-based normalization.¹³ That is, the utility difference between the most and least desirable profiles that can be generated with the attribute levels is normalized to 1.

Figure 3.

Attribute importance by patient phenotype

Based on these preferences, a 3-question preference diagnostic with 2 alternatives per question was prepared. An initial question was used to classify respondents in one of two groups of two phenotypes each. The other two questions were used to classify respondents between the phenotypes within these groups.

Tables 2 to 4 summarize each of the final set of choice questions developed. The tables also include the posterior probabilities for each phenotype conditional on a respondent’s choice. As shown in Table 2, there is a 91% chance of presenting as phenotype 1 or 2 if the respondent chooses alternative A and a 92% chance of presenting as phenotype 3 or 4 if the respondent chooses alternative B. Assuming respondents see the second question (i.e., the respondent chooses alternative A in the first question), there is a 90% chance of presenting as phenotype 1 if they choose alternative B, and an 88% chance of presenting as phenotype 2 if they choose alternative A. Finally, posterior probabilities are 87% and 88% for phenotypes 3 and 4, respectively, depending on respondents’ choices to question 3.

Table 2.

First diagnostic question

	Alternative A	Alternative B
Limitation in motion	No limitations	Sling 1 month
Time to avoid activities	1 month	8 months
Risk of second dislocation	79%	26%
Out-of-pocket cost	$200	$0
Posterior probability of class membership by phenotype	Choosing A	Choosing B
Phenotype 1	4%	45%
Phenotype 2	4%	46%
Phenotype 3	42%	8%
Phenotype 4	50%	1%

Table 4.

Second diagnostic question – For patients who select B in the first question

	Alternative A	Alternative B
Limitation in motion	No lifting	Sling 1 month
Time to avoid activities	5 months	12 months
Risk of second dislocation	80%	5%
Out-of-pocket cost	$190	$148
Posterior probability of class membership by phenotype	Choosing A	Choosing B
Phenotype 3	12%	88%
Phenotype 4	89%	11%

The performance of the diagnostic classifications is shown in Figure 4. The vertical axis shows the proportion of respondents or instances correctly allocated to phenotypes (i.e., true positives). This figure was generated with results from 10,000 simulations of choices with varying levels of choice errors. For each simulation, respondents from each of the four phenotypes were drawn using a uniform distribution, leading to near equal representation of all phenotypes. Responses for the choice questions from each individual were calculated and the final classification was compared between the answers with and without choice errors.

Figure 4.

Percentage of true positives by phenotype and error variance

Note that the horizontal axis shows the error variance. Percentages were obtained out of the number of cases simulated for each phenotype. Thus, the percentages across classes can add to more than 100%. Assignment of classifications was robust for all four phenotypes with a 95% true positive rate up to error variances of 0.71 (phenotype 3) and 0.83 (phenotype 2). This is represented by the error variance at which each curve falls below the 95% green dashed line in Figure 4.

In terms of diagnostic specificity (i.e., true negatives), Figure 5 shows the percentage of the simulated patients correctly classified as being in other phenotypes. As before, these figures show the proportions by error variance. Classification was robust to the 95% level across phenotypes up to an error variance of 0.81 (phenotype 1) and 0.89 (phenotype 4). This is represented by the error variance at which each classification curve falls below the green dotted line in Figure 5.

Figure 5.

Percentage of true negatives by phenotype and error variance

Figure 6 presents the overall percentage of cases correctly classified by error variance. The curves show the sums of true positive and true negative cases divided by the overall number of cases in the simulations. The points at which classification was correct at the 95% confidence level varied among phenotypes. Phenotype 3 falls under the 95% confidence line at an error variance of 0.79. Phenotype 4 has the most robust classification with 95% correct classification up to an error variance of 0.87.

Figure 6.

Overall percentage of cases correctly classified by error variance

In the worst scenario considered, where the variance error is 1 and even dominated scenarios are as prone to errors as any other choice question, phenotype 1 is expected to be correctly classified about one-third of the time. For phenotype 4, this proportion is about 84%. The complement of these percentages indicates the misclassification likelihood with the proposed questions.

DISCUSSION

To enhance shared decision making by incorporating patient preferences in clinical decision making, we have demonstrated that it is possible to develop a preference-diagnostic tool using a small number of choice questions. The tool relies on constructing treatment alternatives to identify patient-preference type using a small number of choice questions. Such a preference-based diagnostic tool in clinical settings could help identify patients’ likely membership in clinically-relevant preference types for shared decision making. In addition, the tool could document that patients’ benefit-risk preferences align with particular treatment alternatives.

Our numerical example defines patient-preference phenotypes based on a data-driven process, but the proposed approach also could be used to identify patients with specific preferences such as individuals who would accept risks associated with a medication class for which benefits and risks are bounded between known values. The exponential improvement in pairwise discrimination as questions are added also could prove helpful in situations where a large number of classes are of interest. As an example, identifying patients from one of 36 phenotypes would require a minimum of 5 questions. This is well within the usual number of questions included in choice-based preference surveys, like DCEs.

Another important advantage of this type of design is that it is expected to minmize respondent burden as the choice questions are prepared to maximize the probability of choice for someone in any of the preference phenotypes considered. Thus, choices between these options would be expected to be fairly obvious to respondents. Nevertheless, further research would be needed to assess whether indeed there would be gains in measurement motivated by the presentation of questions that are potentially easier to answer by respondents.

Our simulations indicated that a 2-question diagnostic tool accurately predicted classification across 4 preference phenotypes over a range of assumed response-error rates. However, more research is needed to better understand the circumstances under which such a tool would perform poorly. Also, more research is needed to understand the performance characteristics of such a tool when used in clinical practice.

Finally, while we relied on patient preference data from a previous conjoint analysis exercise, this may not be necessary to take advantage of the approach proposed here. There could be situations where a clinically-relevant patient type is known based on previous literature, expert opinion, or the specifics of the clinical context. For example, we may be interested in knowing whether patients consider some benefits to be more important than specific risks. One could, in principle, generate preference priors that are consistent with the clinically-relevant patient type. The diagnostic tool then could provide a probabilistic assessment of the patient’s proximity to that preference phenotype.

Limitations

Our work has a number of important limitations. First, using HCA could lead to different numbers of preference groups depending on a number of assumptions, including distance measures between preferences. Different approaches to defining patient phenotypes also could lead to different groupings. In addition, it is possible that larger or smaller sets of subgroups could improve the classification of the diagnostic. We see this as another possible research avenue as future versions of the tool could jointly optimize class definitions and the design for discrimination between classes. Second, it is unclear whether the data collected from the original study provided a representative distribution of preferences among patients with FTASD. However, there is evidence of general consistency in preference results across studies and settings³⁴ and measures of preference heterogeneity are already in use to support policy decisions.³⁵ Our proposed approach provides a way to utilize that existing infrastructure to build tools that allow ongoing collection of preference evidence. A validation exercise would be critical to determine the reliability of the diagnostic questions in the patient population before supporting policy decisions. That validation process goes beyond the scope of this manuscript, but should be addressed in future work.

Finally, more powerful search algorithms could have improved our ability to diagnose preferences. While this does not negate the value of the proposed questions, better tools could be developed to enhance diagnostic sensitivity and specificity.

Conclusions

Preference elicitation, or preference clarification, frequently is evaluated as part of shared decision aids, and it is broadly agreed that patient preferences should play an important role in treatment decisions.³⁶ Informal diagnosis of preferences has been done across different disease areas, but to our knowledge no preference instrument has been designed as a diagnostic tool. That is, relying on benchmarks that allow evaluating Type I and Type II errors. Even advanced adaptive designs for estimating individual-level preferences are not prepared with the intent of evaluating proximity to some “truth” in the same way that other diagnostic tests are evaluated. We propose a novel approach to developing experimental designs for DCEs to be used for understanding whether an individual has preferences that align with specific treatment options. This perspective is different from the current paradigm for the creation of experimental designs in choice experiments. Instead of designing alternatives optimized for the collection of preference information, we optimize the experimental design to measure someone’s proximity to a patient phenotype who meet some predetermined clinically-relevant preferential relationship between decision aspects.

Our results suggest that this approach could help diagnose patient preferences for treatments for a condition such as FTASD with acceptable precision using as few as two choice questions. Using such an efficient preference-diagnostic tool could reduce misalignment of treatment choices and patient preferences. However, the preference-diagnostic tool is not a conventional decision aid. Rather, it is similar to vital-sign and other clinical measures used to diagnose a patients’ clinical condition. Effective patient-centered health care requires shared decision making that jointly considers an accurate preference diagnosis along with personalized predictions about clinical outcomes for individual patients. This process can help prioritize treatment alternatives that a clinician can discuss in more detail with a patient allowing them to focus on concerns that are of greater importance to each individual.

Table 3.

Second diagnostic question – For patients who select A in the first question

	Alternative A	Alternative B
Limitation in motion	No lifting	No limitations
Time to avoid activities	6 months	12 months
Risk of second dislocation	17%	80%
Out-of-pocket cost	$128	$0
Posterior probability of class membership by phenotype	Choosing A	Choosing B
Phenotype 1	87%	13%
Phenotype 2	11%	89%

Highlights.

• Approaches that combine patient preferences and clinical evidence can facilitate effective patient-provider communication and more patient-centric healthcare decisions. However, diagnosing individual-level preferences is challenging and no formal diagnostic tools exist.

• We propose a structured approach to efficiently diagnose patient preferences based on prior information on the distribution of patient preferences in a population.

• We generated a 2-question test of preferences for the outcomes associated with the treatment of first-time anterior shoulder dislocation (FTASD).

• The diagnosis of preferences can help physicians discuss relevant aspects of the treatment options and proactively address patient concerns during the clinical encounter.