Preference-Sensitive Management of Post-Mammography Decisions in Breast Cancer Diagnosis

Abstract

Decision models representing the clinical situations where treatment options entail a significant risk of morbidity or mortality should consider the variations in risk preferences of individuals. In this study, we develop a stochastic modeling framework that optimizes risk-sensitive diagnostic decisions after a mammography exam. For a given patient, our objective is to find the utility maximizing diagnostic decisions where we define the utility over quality-adjusted survival duration. We use real data from a private mammography database to numerically solve our model for various utility functions. Our choice of utility functions for the numerical analysis is driven by actual patient behavior encountered in clinical practice. We find that invasive diagnostic procedures such as biopsies are more aggressively used than what the optimal risk-neutral policy would suggest, implying a far-sighted (or equivalently risk-seeking) behavior. When risk preferences are incorporated into the clinical practice, policy makers should bear in mind that a welfare loss in terms of survival duration is inevitable as evidenced by our structural and empirical results.

1. Introduction

It is estimated that 249,260 women were diagnosed with invasive breast cancer and 40,450 died from breast cancer in 2016 in the United States alone (Siegel et al. 2016), which makes breast cancer the most common non-skin cancer and the second leading cause of cancer death among women. Approximately, one in eight American women is expected to develop breast cancer over her lifetime and one in thirty-six is expected to die from this disease. Mortality due to breast cancer can be greatly reduced when the disease is diagnosed early enough (Horner et al. 2009). The most common and economical method to detect breast cancer at its early stages is screening mammography, an X-ray examination of the breast. Many medical organizations in the United States, recommend annual mammography examinations for women over age 40. When a patient undergoes a screening mammography, a radiologist assesses the images and chooses one of the three options: (i) biopsy to confirm whether the specific lesions are cancerous or not; (ii) follow-up the patient in the short-term (6 months) with screening mammography; or (iii) routine mammography. Such decisions made following mammography are called the diagnostic decisions.

Inherent forms of uncertainty and complexity differentiate health-related problems such as post-mammography diagnostic decisions from other problem domains. In Kenneth Arrow’s classic essay on welfare economics of medical care, “existence of uncertainty in the incidence of disease and the efficacy of treatment” are the core differentiating characteristics of health problems (Arrow 1963). Medical screening, such as screening mammography, is a way of dealing with the uncertainty about a future illness, namely breast cancer, by attempting to detect the disease before it becomes life threatening. When breast cancer is detected at its early stages, the uncertainty around treatment effectiveness diminishes (American Cancer Society 2013). Confronting the uncertainty of a disease, but not the effectiveness of treatment as in the early stages of breast cancer, the patient’s choice will depend on his/her risk preferences (Picone et al. 1998, 2004). In particular, because mammography is an imperfect test, the diagnostic decisions based on it entail a trade-off between catching the cancer early and bearing the risk of performing invasive procedures on women who do not actually have cancer (Mandelblatt et al. 2009). The risk of false assessments based on mammography and the potential consequences along with possible differences in valuation of different outcomes require a preference-sensitive approach to diagnostic decisions based on mammography.

Achieving preference-sensitive care requires an approach to clinical decision-making called the “shared decision-making” which entails more patient involvement in clinical decisions and suggests physician-patient discussion of the treatment options, the associated uncertainties, and the possible outcomes (Charles et al. 1997). The recently enacted policies reemphasize the importance of shared decision-making and recommend for the incorporation of the patient preferences into the decision-making process (e.g., see ACA 2010). Although the level of involvement depends on the individual patient, an increasing number of patients are leaving the passive role and actively engaging in the decision-making process through discussions with the radiologist or seeking a second opinion. According to a survey on preferences for testing and treatment decision-making, most women preferred to have an active role in decision-making (Davey et al. 2002). Specifically, when considering the patients who had been sent for diagnostic mammography due to a suspicious finding, 49% shared the decision-making with their doctors, 36% delegated their decisions to their doctors, and 45.5% made their own decisions based on the information provided by their doctors.¹ Moreover, 23.8% and 41.2% of the patients who had undergone diagnostic mammography demanded more information on benefits and side effects, respectively.

Each individual is unique in her clinical characteristics and preferences in the context of tolerating potential consequences such as a false-positive assessment. Therefore, informing the patients of the benefits and shortcomings of each decision and reaching a consensus together with the referring physician, the radiologist, and the patient reflective of the patient’s valuations is critical. For example, a patient who has been asked about false-positive biopsies said, “False-positives? So what? Better to be safe than sorry, in my opinion. I’d rather have an unnecessary biopsy than die of undetected cancer” (Barker and Galardi 2011). In contrast, another patient who was biopsied before said, “The biopsy is painful and scary, and I wonder what damage they may be doing with all the poking” (Carollo 2010). These examples suggest that there is heterogeneity in patients’ preferences and that the patients may differ in their attitudes toward diagnostic decisions and the resulting life expectancy.

Non-inclusion of patient preferences in modeling post-mammography diagnostic decisions would imply a risk-neutral assumption which could make sense from a societal point of view when we consider the society as being composed of individuals with various risk preferences. However, when a physician makes a decision for an individual patient, the patient’s and/or physician’s risk preferences would matter; therefore, making choices not only based on objective measures of survival, but also on risk preferences would be important (McNeil et al. 1978, Pauker and McNeil 1981). For example, a substantial number of people would have clear preferences when given a choice between 5 years of life for certain and a gamble with equal likelihood of either immediate death or 10 years of life. Decision models representing clinical situations where treatment options entail a risk of morbidity or mortality should, therefore, consider the variations in risk preferences for individuals or the society as a whole (Cher et al. 1997, Pliskin et al. 1980).

1.1. Problem Definition

In this study, we formalize and empirically validate risk-sensitive Markov decision process (MDP) models as applied to medical decision analysis. Specifically, we have developed a methodology to consider the risk preferences of patients in the context of optimal diagnostic decisions following mammography. Our model selects from the three alternatives that a radiologist can recommend while considering the patient’s preferences: biopsy, short-term follow-up, and routine mammography. We model this problem as a finite-horizon discrete-time MDP model where the reward structure is constructed using invertible utility functions to make our MDP model risk-sensitive. The utility functions in this study represent simple characterizations of risk-postures that may be important in clinical decision-making and reflect actual patient behavior as discussed in section 2.2.

Although several application areas have been explored using risk-sensitive MDPs (Liu 2005), the use of risk-sensitive MDPs in the health context is rather recent. Early attempts in modeling risk behavior for liver transplant decisions called for further research into explaining observed patient behavior (Batun et al. 2010). Erkin et al. (2010) used inverse optimization to elicit patient valuations of various health outcomes for living-donor liver transplant decisions. Tilson and Tilson (2013) developed an MDP model to investigate the optimal decisions under various criteria including a risk-sensitive scenario with exponential utility. The authors illustrated the variation in treatment decisions following an incidental diagnosis of non-symptomatic intracranial aneurysm, using simulated data. Previous literature addressing post-mammography decisions considers patients or decision makers as risk-neutral. Chhatwal et al. (2010) studied optimal biopsy decisions following mammography. Ayvaci et al. (2012) explored optimal diagnostic decisions under resource-limited settings using a constrained-MDP framework. Ayvaci et al. (2017) focused on follow-up decisions and provided insights into their extensive use in clinical practice. Recent literature has also studied behavioral biases in post-mammography decisions. In particular, Ayvaci et al. (2017) examined the role of timing the use of risk profile information and the associated human biases in diagnostic performance. Ahsen et al. (2018) developed a classification algorithm to debias radiologists’ biased mammography assessment.

1.2. Operational Relevance of Preference-Sensitive Medical Decisions

In an opinion article, Green (2012) summarizes the role of operations management (OM) in healthcare as “how to better utilize resources to improve outcomes while reducing costs.” Healthcare industry has certain idiosyncrasies that differentiate it from other industries in terms of (operations) management in an organization, particularly in regards to (i) how resource utilization decisions are made, (ii) how outcomes/quality measures are defined, and (iii) how revenues are tied to outcomes. Next, we briefly explain the role of medical decisions from operations management lens with a focus on preference-sensitive care.

Regarding the resource utilization, how healthcare organizations and physicians co-produce care is an important factor in the connection between efficient utilization of resources and medical decisions. Medical decisions made by physicians determine the level of resource use, therefore directly impacting the operations of the hospitals (Mehrotra and Hussey 2015). While hospitals/health maintenance organizations (HMOs) traditionally manage the resources that physicians utilize and leave the physicians, for the most part, independent in their medical decisions, hospitals/HMOs can also manage the actual delivery by focusing on the management of medical decisions through establishing guidelines and protocols and implementing them via clinical decision support systems. Our work relates to the latter management approach where guideline development for risk-sensitive breast cancer diagnostic decisions is the context.

Regarding the outcome metrics used in healthcare for performance improvement, patient-centered care outcomes, therefore patient preferences, play a key role. Patient-centered medical decisions are associated with improved patient experience and has been shown to translate into better patient satisfaction in a variety of decision contexts (Weng 2008) including breast cancer (Lantz et al. 2005). Following Institute of Medicine’s seminal report on patient-centered care (Institute of Medicine (IOM) 2001), patient experience, and therefore patient satisfaction, has become increasingly important for hospital administrators and physicians. In 2015, 53% of the US hospitals reported that patient experience or satisfaction is one of their top three organizational priorities while 83% of the US hospitals have established formal structures (e.g., chief experience officer) for managing patient experience (Wolf 2015).

Regarding the tying of patient-centered outcomes to payments, recent healthcare reform efforts focusing on alternative payment and delivery models emphasize patient-centered care (Burwell 2015). In radiology practices, the need for demonstrating value-based care and engaging patients in diagnostic decisions is gaining paramount traction (Parikh and Yang 2016) where breast imaging has been at the forefront (Javitt 2016). The new value-based care environment increasingly links the payments to quality metrics in breast imaging (Parikh and Yang 2016) or other types of radiological imaging (Rawson 2015, Sarwar et al. 2015). As part of value-based payments, radiology practices could use clinical outcomes (e.g., the rate of false breast biopsy) or experience-related outcomes (e.g., the extent of communication with patients in imaging-based care) to define value (Abujudeh et al. 2010). Despite the challenges associated with defining the right set of metrics that would measure what matters the most for the patient (i.e., the patient’s preferences) and indicate higher quality care, the radiology community has already embraced the idea of putting the patient at the center (Carlos et al. 2012). In fact, in the last few years, there have been a large number of articles published in the radiology literature discussing the importance of incorporating patient preferences into diagnostic decision making (e.g., see Elmore and Kramer 2014, Itri 2015, Rosenkrantz and Rawson 2017). Surveys measuring patient-centered care outcomes that target radiology departments are also likely to take hold (Lang et al. 2013).

1.3. Contributions

We have several contributions in this study. First, we introduced a modeling scheme for tailoring sequential medical interventions toward risk-sensitive care. Extant OM literature on disease management addresses conditions including chronic diseases (Ibrahim et al. 2016, Skandari et al. 2015), transplantation (Alagoz et al. 2004, Su and Zenios 2004, 2006), cancer (Cevik et al. 2018, Erenay et al. 2014, Güneş et al. 2015, Zhang et al. 2012), and HIV (Zaric et al. 2000) where decision makers are risk-neutral. Although there exist few recent studies that investigated risk-sensitive MDP modeling for various medical decisions, our approach in modeling sequential decisions is unique because our model accounts for the expected future longevity as the reference point at which the risk-sensitive decision is made and also treats the expected intermediate longevity as the pay-off of the lottery. This approach better reflects the trade-off that a patient would face in a sequential medical decision-making encounter. Moreover, the existing work either lacks clinical data and medical/policy insights or do not focus on explaining the existing behavior.

Second, our prescriptive approach could inform both policy makers and the clinicians alike in an era where preference-sensitive care is highly advocated. To the best of our knowledge, there are no other studies that investigated the role of risk preferences in breast cancer diagnostic decisions. Considering the fact that 63 out of 10,000 women in the United States are biopsied in any given year (Ghosh et al. 2005) and approximately 4 out of 5 biopsies have benign outcomes (Poplack et al. 2005), our findings shed light on a highly utilized and uncertain procedure that should ideally account for patient preferences. We show that—with the shared decision-making becoming more of a part of the health-care—a welfare loss in terms of survival duration is inevitable. Our modeling approach can help in quantifying this welfare loss. Moreover, we seek to determine the tendency of current risk behavior as implied by the clinical practice by using an innovative approach to evaluate these decisions. In some sense, we reverse-engineer the already made diagnostic decisions to find out the risk behavior associated with those decisions as described in section 5. We are able to demonstrate the percentage discrepancy between the actual practice and the risk-sensitive optimal policies and find the best fitting risk behavior. Our findings provide a baseline understanding of current standards of practice (in a single institution) based on decision-making behavior and associated outcomes. The approach we employ is unique in terms of numerically exploring the best-fitting risk behavior, using a large mammography database and can also be employed for evaluating other sequential medical decisions.

Lastly, we introduce a novel empirical method in estimating the transition probability matrices of MDPs. Despite substantial advancements in methods for modeling health decisions, reliance on small data sets and occasional use of data from multiple sources in validating these models continues to prevail in disease modeling (Caro et al. 2012). The use of small data sets could make the estimation of transition probability matrices of an MDP a difficult task for which various approaches were proposed (Craig and Sendi 2002). Because of the curse of dimensionality, datasets easily become insufficient for estimating the transition probability matrices for MDPs. The non-standard empirical method we developed uses a parametric approach to overcome the curse of dimensionality and is consistent with the expected transition behavior as further described in section 4.

The rest of this study is organized as follows. In section 2, we give an overview of our modeling approach and describe our choice of specific utility functions. In section 3, we formulate the risk-sensitive MDP model for optimizing the diagnostic decisions. Our formulation can utilize any invertible utility function. We also provide the structural insights from the model in section 3. In section 4, we describe our data sources and detail the estimation of the parameters. In section 5, we present the numerical analysis using the utility functions we have identified. We discuss potential implementation in clinical practice and policy implications in section 6.

2. Our Approach for Modeling Preferences

This section provides the framework for our modeling of risk-sensitive diagnostic decisions following mammography. In particular, section 2.1 discusses the use of utility functions in modeling risk behavior in the context of this study, section 2.2 describes the three functional forms of utilities and the implied behavior of these functional forms in the context of medical decisions over survival, and section 2.3 formally defines the problem and presents the mechanics of evaluating lotteries at each decision point.

2.1. Modeling Preferences Over Survival

To better reflect the competing interests in health care decision-making, one can construct a value function over the attributes and then represent the risk preferences by assigning a utility function over the value measure which is consistent with value-focused thinking (Keefer et al. 2004). However, an explicit clarification of the values are essential for successful decision-making (Keeney 1994). In the context of diagnostic decisions based on mammography, a radiologist would make a decision by implicitly considering several factors including the likelihood of breast cancer, life expectancy of the patient, other co-morbid conditions, and the patient’s attitude toward biopsy/the possibility of cancer. A value measure that could account for such factors is expected survival duration with quality adjustment, which has been suggested as an objective value measure for normative decision analysis in healthcare (Cher et al. 1997, Miyamoto and Eraker 1989).

The primary method of choice in modeling individual behavior in the health context has been expected utility theory (Gold 1996). By imposing a utility function over the quality-adjusted survival duration, one can capture gains from the alternatives in both quantity and quality (Torrance and Feeny 1989). Risk attitude in such a setup would then refer to the curvature of the utility function over the quality-adjusted survival duration. When defined as such, the utility function measures the preference over marginal gains of life-years at different levels such as gains of life-years in the short term or in the long term. Namely, risk-averse (concave) utilities imply a tendency to prefer survival in near future over survival in the distant future, whereas risk-seeking (convex) utilities imply a tendency to prefer survival in the distant future as opposed to survival in the near future.

2.2. Utility Functions Explored

Various functional forms of utility were used in modeling the risk behavior in the medical context with respect to expected survival duration. We choose three classes of utility functions to represent patient behavior toward post-mammography diagnostic decisions: exponential, power/logarithmic, and logistics utility functions. We choose these classes of utility functions because they were extensively used in modeling risk behavior for medical decisions (Gafni and Torrance 1984, Miyamoto and Eraker 1989, Pliskin et al. 1980, Verhoef et al. 1994) and more importantly, these utility functions may reflect actual patient behavior as we describe next. In the following, we list examples of patient behavior (denoted by $\mathcal{B}$) reported on breast cancer detection and post-mammography diagnostic decisions. We use medical literature and news stories on the subject in finding our examples. Note that one of the authors of this manuscript is a radiologist and the behaviors listed below are consistent with patient behaviors she has encountered in clinical practice.

• (${\mathcal{B}}_1$) “False-positives? So what? Better to be safe than sorry, in my opinion. I’d rather have an unnecessary biopsy than die of undetected cancer” (Barker and Galardi 2011).
• (${\mathcal{B}}_2$)“The biopsy is painful and scary, and I wonder what damage they may be doing with all the poking” (Carollo 2010).
• (${\mathcal{B}}_3$)“Percentages mean more to me because you don’t know if you’ll live 15 years and 1 day but with percentages you can’t put a precise time limit on it, so that’s why I’d go for 0.1 %” (Duric et al. 2007).
• (${\mathcal{B}}_4$)A review study on age differences in decision-making scenarios found that “older adults were less willing to accept reduced quality of life in exchange for even minimal longer survival time” (Pinquart and Duberstein 2004).
• (${\mathcal{B}}_5$)“I weighed the options as my hospital date approached. Average risk, after all, is not 0. Could I live with that? Part of me still wanted to extinguish all threat. I have a 9-year-old daughter; I would do anything—I need to do everything—to keep from dying” (Orenstein 2013).

We connect these examples to our modeling and the choice of utility functions under the description of each utility function presented below.

Exponential Utility Function:

Exponential utility functions characterize a risk posture called “constant absolute risk aversion” (Pratt 1964). In the medical context, constant absolute risk aversion represents the risk behavior when a patient’s preference on absolute gains or losses with respect to survival duration solely depends on the absolute survival duration at stake. Exponential utility function characterizes zero-switch rule, which implies that if a patient accepts a risky treatment at a younger age, she would likely also accept a similar risky treatment at an older age. The functional form of the exponential utility is given by

$$u(y)=\left\{\begin{array}{ll}y\hfill & \text{if}\hspace{0.33em}\gamma =0,\hfill \\ \frac{1-e^{-\gamma \times y}}{1-e^{-\gamma }}\hfill & \text{otherwise},\hfill \end{array}\right.$$ (1)

where γ represents the level of risk-aversion (or seekingness). Behaviors ${\mathcal{B}}_1$ and ${\mathcal{B}}_2$ may represent such behavior since the patient has a preference toward the life-years in the long term (as in behavior ${\mathcal{B}}_1$) or in the short-term (as in behavior ${\mathcal{B}}_2$) independent of expected life. The long-term survival preference values the overall longevity more (as in behavior ${\mathcal{B}}_1$) whereas the short-term survival preference values the biopsy disutility in the near future more than the overall longevity (as in behavior ${\mathcal{B}}_2$). Note that when γ approaches 0 from above or below, Equation (1) becomes a linear function, implying risk-neutral behavior. Positive γ values imply risk-averse behavior whereas negative γ values imply risk-seeking behavior.

Power/Logarithmic Utility Function:

As opposed to the constant absolute risk aversion feature of exponential utility functions, power/logarithmic utility functions characterize a risk posture called “constant relative risk aversion” (Pratt 1964). The functional form of the power/logarithmic utility is given by

$$u(y)=\left\{\begin{array}{ll}-y^{1-c}\hfill & \text{if}\hspace{0.33em}c>1,\hfill \\ ln(y)\hfill & \text{if}\hspace{0.33em}c=1,\hfill \\ y^{1-c}\hfill & \text{if}\hspace{0.33em}c<1,\hfill \end{array}\right.$$ (2)

where the single parameter of power/logarithmic utility function, denoted by c, defines an individual’s willingness to exchange long-term and short-term survival (Miyamoto and Eraker 1989). In the medical context, constant relative risk aversion represents the risk behavior when a patient’s preference toward percentage gains or losses with respect to survival duration solely depends on the proportion of survival duration at stake but not on the absolute number of remaining life-years. In terms of the absolute number of life-years, the behavior translates to decreasing risk-aversion or increasing risk-seekingness for positive or negative c values, respectively. Behavior ${\mathcal{B}}_3$ may represent such behavior because the patient has a preference toward the percentage of remaining life-years at stake when making medical decisions. Behavior ${\mathcal{B}}_4$ may also be considered as reflecting the power/logarithmic utility due to the difference between the older and younger patients in terms of willingness to accept a trade-off between survival in the short-term vs. long-term.

Logistics Utility Function:

According to prospect theory, risk behavior depends on the perception of whether the outcome is considered a gain or loss (Kahneman and Tversky 1979). Although linear, power/logarithmic, and exponential utility functions are useful in modeling risk attitude toward survival duration (Miyamoto and Eraker 1989), these functional forms lack the ability to vary risk behavior with respect to a reference point. With regard to gambles for length of life, an association of risk-aversion with relative gains and risk-seekingness with relative losses is plausible (van Osch et al. 2006). Verhoef et al. (1994) use the “aspiration level” argument in explaining longevity-dependent risk behavior toward medical therapies. Namely, a person may accept a risky therapy with higher long-term survival as compared to a less risky alternative with better outcomes in the short-term because of the person’s willingness to achieve a certain aspiration level. The functional form of logistics utility function is given by

$$u(y)=\frac{\rho }{(1+\alpha /y^{\beta })}.$$ (3)

The function is s-shaped such that it is convex at smaller y values and it becomes concave at the inflection point y*, or the aspiration level. The implied aspiration level for a given set of parameters can be calculated by $y^{\ast }=\alpha \times {\left(\frac{\beta -1}{\beta +1}\right)}^{1/\beta }$. Behavior ${\mathcal{B}}_5$ may represent such behavior because the patient defines the preference with respect to a goal, seeing her 9-year-old daughter grow up.

2.2.1. Preference-Sensitive Decisions in Practice.

Effective risk-sensitive decisions in breast cancer diagnosis depend on eliciting patient preferences on survival. In particular, determining what utility function best reflects the patient preferences and estimating the parameters that capture the patient’s risk attitude is essential. Regarding the utility functions, our qualitative mapping of behaviors ${\mathcal{B}}_1-{\mathcal{B}}_5$ to functional forms used in the literature gives a practical example of identifying attitudes toward risk in breast cancer diagnostic decisions, namely preferences around false-positives or reassurance based on biopsy. A specific survey instrument can explicitly determine whether a patient exhibits constant or absolute risk aversion or whether an aspiration level of survival is meaningful to her (e.g., see Harrison et al. 2005, for a review of instruments). Systematically building on these (and other) behaviors, a carefully crafted set of stylistic questions as part of the instrument can reveal the form of the utility function that can model the patient perspective. Alternatively, a physician can implicitly assess the patient’s risk aversion based on interview with her (Mulley 1989). An editorial by Sadigh et al. (2011) recognizes the role of both decision aids and radiologists in eliciting patient values and preferences in breast cancer screening.

Regarding the estimation of parameters of the utility functions, the well-established theory and the extensive applications in medical decision making indicate the feasibility of estimation in breast cancer diagnosis context as well. A recent systematic review found that the most commonly used method for value clarification was utility theory (18%) and a significant proportion of studies used explicit (interactive) means to measure values (42%) (Fagerlin et al. 2013). A plethora of methods for elicitation ranging from conjoint analysis to rating and ranking and to standard gamble are available. Particularly, the standard gamble method based on utility theory is a widely used method for preference elicitation (Klose et al. 2016). In the context of breast cancer diagnosis, an interactive computer-based interview tool can first provide the necessary background for the patient to understand the survival, value of early diagnosis, and other relevant information. Based on the patient age and other characteristics, the tool can pose a standard gamble with a certain number of life years (e.g., the average expected survival for the patient) and iterate over uncertain prospects. The uncertain prospect can be breast-cancer free survival with probability p and survival with breast cancer with probability 1 — p. The tool then can present a series of gambles with changing p until the patient is indifferent between the certain number of life years and the uncertain prospects. Bala et al. (1999) provide an example of standard-gamble based preference elicitation using computer-based interview tool in the context of shingles. The existing value clarification methods in various breast cancer-related decisions (e.g., Moumjid et al. 2003), as well as our framework, can inform the design of preference elicitation in breast cancer diagnosis. In section 6.1, we provide further discussion of how to translate our findings into clinical practice, yet leave the design of a complete value clarification exercise in our context for future work.

2.3. Formalization of Diagnostic Decisions as Sequential Lotteries

Given the preferences of a patient, the task is to find the diagnostic action maximizing the total expected utility. We refer to each diagnostic action as a lottery because each diagnostic action can be considered as an uncertain proposition that a radiologist is facing with. Following a mammogram, there will be three lotteries to choose from: lotteries for biopsy, short-term follow-up, and routine mammography (Figure 1a). The value of each lottery can be determined as described in section 2.3.1 below. The pay-offs of each lottery are the intermediate or the lump-sum rewards (only when the biopsy reveals cancer). The total expected maximal certainty equivalent of longevity in the consecutive decision epoch, called the “expected future longevity,” is the level at which the value of a lottery is evaluated. We assume that the decision maker is the radiologist who chooses one of the lotteries on behalf of the patient while considering the patient’s preferences, and the patient would adhere to the radiologist’s recommendation.

Figure 1.

Lottery Representation of Diagnostic Actions

Notes: Part (a) depicts all of the lotteries that the patient chooses from at any decision epoch. The squares represent decisions, the triangle represents the terminal node, the circles represent uncertainties, the bold lines represent connection to the sequential decisions, and the dashed lines represent the information set such that the decision maker is not aware of the existence or absence of cancer. Part (b) depicts the generic lottery corresponding to a patient in state X under action a at time t.

For a meaningful comparison of decision outcomes due to various behaviors, we need to distinguish between the certainty equivalent of longevity and the actual survival duration. In the rest of this study, we use the term “quality-adjusted life-years” (QALYs) to refer to expected quality-adjusted survival duration and interchangeably use the term “longevity” and “quality-adjusted longevity” to refer to the certainty equivalent of expected quality-adjusted survival duration that each decision alternative entails for a given utility function. Longevity, as defined in this paper, takes into account the risk attitude toward survival duration with quality adjustment.

2.3.1. Mechanics of Valuation of Each Lottery.

We decompose the uncertainty the patient is facing in terms of expected longevity into two periods since the uncertainty that the radiologist faces at each decision epoch has two components: (i) the uncertain longevity in between the current decision epoch and the next decision epoch (including the quality adjustment due to the diagnostic action) and (ii) the uncertain longevity from the next decision epoch onwards. The decomposition of the payoffs as either intermediate or lump-sum expected longevity along with the uncertainty as to whether the patient has cancer or not forms a lottery.

We let x be a random variable (patient’s health state), a be the action, and t be the time. Figure 1b depicts the lottery representing any diagnostic decision, where ϒ^{(x, a)}(t) represents the maximum expected future longevity starting in the next decision epoch onwards, y^{(c, a)}(t) represents the intermediate longevity payoff when the patient has cancer (denoted by c), and y^{(b, a)}(t) represents the intermediate longevity payoff when the patient has no cancer (denoted by b) given that action a is chosen at time t. Then, ${\tilde{y}}^{(x,a)}(t)$ represents the lottery, and the $\text{C}\left({\Upsilon }^{(x,a)}(t),\hspace{0.33em}{\tilde{y}}^{(x,a)}(t)\right)$ represents the corresponding certainty equivalent in terms of longevity evaluated at ϒ^{(x, a)}(t) when the patient’s state is x and action a is chosen at time t. The lottery is between y^{(c, a)}(t) and y^{(b, a)}(t). We then write the utility over certainty equivalent as

$$\begin{gathered} u\left({\Upsilon }^{(x,a)}\left(t\right)+\mathbf{\text{C}}\left({\Upsilon }^{(x,a)}\left(t\right),{\tilde{y}}^{(x,a)}(t)\right)\right) \\ =E\left[u\left({\Upsilon }^{(x,a)}\left(t\right),{\tilde{y}}^{(x,a)}(t)\right)\right]. \end{gathered}$$

The risk behavior can be characterized by comparing $\left({\Upsilon }^{(x,a)}\left(t\right)+\mathbf{\text{C}}\left({\Upsilon }^{(x,a)}\left(t\right),{\tilde{y}}^{(x,a)}(t)\right)\right)$ and $\left({\Upsilon }^{(x,a)},\left(t\right),+,E,\left[{\tilde{y}}^{(x,a)}(t)\right]\right)$ such that the equality between the two implies risk neutrality, inequality implies risk aversion if the former has a lower value, and inequality implies risk-seeking behavior if the former has a higher value. When the inverse of the utility function has a closed-form expression, certainty equivalent can easily be calculated using:

$$\begin{gathered} \mathbf{\text{C}}\left({\Upsilon }^{(x,a)}(t),{\tilde{y}}^{(x,a)}(t)\right)=u^{-1}\left(E\left[u\left({\Upsilon }^{(x,a)}(t)+{\tilde{y}}^{(x,a)}(t)\right)\right]\right) \\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}-{\Upsilon }^{(x,a)}(t). \end{gathered}$$ (4)

3. Problem Formulation and Structural Analysis

Following the modeling framework proposed in section 2, we will represent breast cancer diagnostic decisions as a discrete-time finite-horizon risk-sensitive MDP model with n + 1 time points (t) where t ∈ T={0, 1, 2, …, n}. Next, we detail the risk-sensitive MDP model.

The state space χ indicates the cancer risk status of the patient where we represent the state of the patient at time t using x_t. More formally, the state space χ consists of the set of risk scores S_t = {0, 1, 2, …, S} and two absorbing states, De for death, and Ca for cancer. Each risk score represents a discreticized probability of cancer where we divide a unit interval into S + 1 intervals with S — 1 intervals of size 1/S and two states of size 1/(2 × S) (namely x_t = 0 and x_t = S). For example, when the probability of cancer is 0.0132 and S = 1000, we can obtain the risk score x_t by multiplication, 0.0132 × 1000 = 13.2, and rounding to the nearest integer to find x_t = 13.

The action space $\mathcal{D}$ indicates the decision made based on the risk score x_t at time t where we represent the decisions using $d_t^{x_t}\in \mathcal{D}$. More formally, $\mathcal{D}$ consists of three decisions: biopsy (B), short-term follow-up (F), and routine mammography (R).

The transition probabilities depend on the time, state, and the chosen decision. The patient comes back for regular screening mammography at time t + 2 when the decision taken at time t is R, she comes back for a follow-up visit at time t + 1 when the decision taken at time t is F or when the decision taken at time t is B and the outcome of biopsy is benign. Let q_t (De, Ca) represent the probability of death during time t when the patient has cancer but has not started treatment yet, q_t (De) represent the probability of death when the patient has no cancer, and w (x_t) represent the numeric probability of cancer as indicated by x_t where x_t ∈ S_t holds. For example, when S = 1000 and x_t = 13, then w(x_t) = 0.013 holds.

We model the probability of transition to the death state under decision F as a function of death probabilities weighted by the probability of cancer or no cancer during the current decision epoch. We scale the probability of transition to any state other than death by the probability of staying alive during that decision epoch to make probabilities sum to 1. Under decision F, the transition probabilities are summarized as follows:

• $$\begin{gathered} p_t\left(De|x_t,F\right)=w(x_t)\times q_t(De,Ca) \\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}+(1-w(x_t))\times q_t(De)\hspace{0.33em}\text{for}\hspace{0.33em}{x}_t\in S_t, \end{gathered}$$

• $p_t(Ca|x_t,F)=0\hspace{0.33em}\text{for}\hspace{0.33em}{x}_t\in S_t,$

• $p_t(x_{t+1}|x_t,F)$ for x_t, x_t+1 ∈ S_t is numerically estimated and scaled by $(1-p_t(De|x_t,F))$.

Modeling of transition probabilities under decision R is similar to that of decision F except that we use the transition probabilities of two consecutive decision epochs under decision F to compute the transition probabilities under decision R.

Under decision B, the patient immediately transitions to either the cancer state Ca if the biopsy reveals cancer or no-risk state, x_t = 0, if the biopsy outcome is benign. We assume that biopsy is perfect which is a reasonable assumption because it is a highly accurate method for diagnosing breast cancer (Riedl et al. 2005). The transition probabilities therefore are

• $$\begin{gathered} p_t(x_{t+1}|x_t,B)=(1-w(x_t))\times p_t(x_{t+1}|0,F) \\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\text{for}\hspace{0.33em}{x}_t\in \chi \backslash \left\{Ca\right\}, \end{gathered}$$

• $p_t(Ca|x_t,B)=w(x_t)\hspace{0.33em}\text{for}\hspace{0.33em}{x}_t\in S_t.$

The rewards associated with each decision are calculated to be either the total expected longevity until the next decision epoch or to be a lump-sum expected longevity when a biopsy reveals cancer, both with the quality adjustments. The modeling of rewards takes into account age-based mortality rates in the presence or absence of cancer and age-based treatment-effectiveness in the presence of cancer.

We let ${\tilde{l}}_t(x_t,F)$ and ${\tilde{l}}_t(x_t,R)$ represent the lotteries on intermediate expected longevity at time t ∈ T \ {n} under short-term follow-up and routine screening mammography, respectively. We model the expected longevity until the next decision epoch as probability weighted survival. Namely, we assume a full-cycle length longevity under no death and half-cycle length longevity under death, which accounts for the possibility that death can occur anytime during the discrete time interval. We incorporate the quality of life adjustment by subtracting longevity values that represent the “disutility” of the patient for different diagnostic actions from the expected longevity. This is an important part of modeling the rewards since it takes into account the adverse effects that can occur as a result of the selected diagnostic actions. The disutilities are denoted by d(F) and d(R) for the respective actions. Under decision F, we denote the corresponding expected longevity values for the lotteries by $\widehat{l_t^c(F)}$ when the patient has cancer at time t and by $\widehat{l_t^b(F)}$ when the patient has no cancer at time t. We obtain the expected quality-adjusted longevity payoffs using $l_t^c(F)=\widehat{l_t^c(F)}-d(F)$ and $l_t^b(F)=\widehat{l_t^b(F)}-d(F)$. Under decision R, we can sum the expected longevity values for two consecutive decision epochs where the expected longevity in the second decision epoch is conditioned on surviving the first one. The corresponding expected quality-adjusted longevity payoffs are $l_t^c(R)=\widehat{l_t^c(R)}-d(R)$ and $l_t^b(R)=\widehat{l_t^b(R)}-d(R)$. In our formulation, we can use w(x_t) as the weight for the cancer arm of the lottery when the patient has cancer under decisions F, B and R.

We let ${\tilde{l}}_t(x_t,B)$ represent the lottery on the longevity payoffs for the biopsy decision at time t ∈ T \ {n}. The payoffs of the biopsy lottery depend on the outcome of the biopsy. When the biopsy reveals cancer, a lump-sum payoff, $l_t^c(B)$, defined as the expected remaining longevity, $\widehat{l_t^c(B)}$, minus the time-dependent disutility of biopsy, d_t(B), is accrued based on the assumption of immediate start of treatment. When a biopsy reveals a benign outcome, however, the patient moves to the free-of-cancer state, state 0, accruing $l_t^b(B)=\widehat{l_t^b(B)}-d_t(B)$ where $\widehat{l_t^b(B)}$ is the expected longevity when biopsy reveals no cancer at time t.

We define L_t(x_t) as the maximum total expected certainty equivalent of longevity that the patient can attain when her current state is x_t at time t. Following our modeling setting described in section 2.3, one can use Equation (4) to find the certainty equivalent for an invertible utility function. Hence, we can find the optimal policy by solving the following optimality equations:

$$\begin{gathered} L_t(x_t)=\max \left\{u^{-1}\right.\left(w(x_t)\times u\left(l_t^c(B)\right)+(1-w(x_t))\right. \\ \hspace{1em}\hspace{1em}\hspace{1em}\times u\left.\left(l_t^b(B)+\sum _{x\prime \in \chi }{p}_t(x\prime |0,F)L_{t+1}(x\prime )\right)\right), \\ \hspace{1em}\hspace{1em}\hspace{1em}{u}^{-1}\left(w(x_t)\times u\left(l_t^c(F)+\sum _{x\prime \in \chi }{p}_t(x\prime |x_t,F)L_{t+1}(x\prime )\right)\right. \\ \hspace{1em}\hspace{1em}\hspace{1em}+(1-w(x_t))\times u\left.\left(l_t^b(F)+\sum _{x\prime \in \chi }{p}_t(x\prime |x_t,F)L_{t+1}(x\prime )\right)\right), \\ \hspace{1em}\hspace{1em}\hspace{1em}{u}^{-1}\left(w_t(x_t)\times u\left(l_t^c(R)+\sum _{x\prime \in \chi }{p}_t(x\prime |x_t,R)L_{t+2}(x\prime )\right)\right. \\ \hspace{1em}\hspace{1em}\hspace{1em}+(1-w_t(x_t))\times u\left.\left.\left(l_t^b(R)+\sum _{x\prime \in \chi }{p}_t(x\prime |x_t,R)L_{t+2}(x\prime )\right)\right)\right\} \\ \hspace{1em}\hspace{1em}\hspace{1em}t\in \{0,1,\dots ,n-1\},x_t\in S_t, \end{gathered}$$ (5)

with the boundary condition,

$$L_n(x_n)=l_n\hspace{1em}\forall x_t\in \chi \backslash \left\{De\right\},$$ (6)

where l_n represents the terminal longevity for all states.

3.1. Structural Insights

In this subsection, we present only the main analytical results without technical details and refer the reader to the Appendix S1 for the details as well as for other results that were instrumental in proving the main results. In this section, we consider only the exponential utility function which satisfies u(a + b) = u(a)u(b) and u⁻¹(ab) = u⁻¹(a) + u⁻¹(b) for any positive a and b, since it simplifies structural analysis and more importantly, better approximates radiologist’s actual test behavior as shown in section 5. The exponential utility function, when used in a risk-sensitive MDP model, allows for the separability of the reward functions due to the above condition and we can relate the two Markov reward lotteries at two successive stages. We define the following modified reward functions for state and action pairs, $r_t(x_t,d_t^{x_t})$, to facilitate the structural analysis:

$$\begin{gathered} r_t(x_t,B)={\widehat{r}}_t(x_t,B)-d_t(B) \\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{0.33em}=u^{-1}\left(w(x_t)u\left(\widehat{l_t^c(B)}-d_t(B)\right)+(1-w(x_t)\right) \\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{0.33em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}u\left(\widehat{l_t^c(B)}-d_t(B)+\sum _{x\prime \in \chi }{p}_t(x\prime |0,F)L_{t+1}(x\prime )\right. \\ {r}_t(x_t,F)={\widehat{r}}_t(x_t,F)-d(F)=u^{-1}\left(w(x_t)u\left(\widehat{l_t^c(F)}-d(F)\right)\right. \\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{0.33em}\hspace{0.17em}+(1-w(x_t))u\left.\left(\widehat{l_t^c(F)}-dF\right)\right), \\ {r}_t(x_t,R)={\widehat{r}}_t(x_t,R)-d(R)=\widehat{r}t(x_t,F) \\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{0.33em}\hspace{0.17em}+\sum _i{p}_t(x\prime |x_t,F){\widehat{r}}_{t+1}(x\prime ,F)-d(R) \\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{0.33em}\hspace{0.17em}=u^{-1}\left(w(x_t)u\left(\widehat{l_t^c(F)}+\sum _i{p}_t(x\prime |x_t,F){\widehat{r}}_{t+1}\right.\right. \\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{0.33em}\hspace{0.17em}\left.(x\prime ,F)+d(R)\right)+(1-w(x_t))u\left(\widehat{l_t^b(F)}\right. \\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{0.33em}\hspace{0.17em}+\left.\left.\sum _i{p}_t(x\prime |x_t,F){\widehat{r}}_{t+1}(x\prime ,F)-d(R)\right)\right), \end{gathered}$$ (7)

Using the above definitions, we rewrite the optimization problem given in Equation (5) as

$$L_t(x_t)=\max \{L_t(x_t,B),L_t(x_t,F),L_t(x_t,R)\},$$ (8)

where

$$\begin{gathered} L_t(x_t,B)=u^{-1}\left(w(x_t)u\left(l_t^c(B)\right)+(1-w(x_t))u\left(l_t^b(B)\right.\right. \\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{0.33em}\hspace{0.33em}+\left.\left.\sum _{x\prime \in \chi }{p}_t(x\prime |,F)L^{t+1}(x\prime )\right)\right)=r_t(x_t,B). \\ {L}_t(x_t,F)=u^{-1}\left(w(x_t)u\left(l_t^c(F)+\sum _{x\prime \in \chi }{p}_t(x\prime y|x_t,F)L_{t+1}(x\prime )\right)\right. \\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{0.33em}\hspace{0.33em}+(1-w(x_t))u\left(l_t^b(F)+\left.\sum _{x\prime \in \chi }{p}_t(x\prime |x_t,F)L_{t+1}(x\prime )\right)\right) \\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{0.33em}\hspace{0.33em}=u^{-1}(w(x_t)u\left(l_t^c(F)\right)+(1-w(x_t))u\left.\left(l_t^b(F)\right)\right) \\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{0.33em}\hspace{0.33em}+\sum _{x\prime \in \chi }{p}_t(x\prime |x_t,F)L_{t+1}(x\prime ) \\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{0.33em}\hspace{0.33em}=r_t(x_t,F)+\sum _{x\prime \in \chi }{p}_t(x\prime |x_t,F)L_{t+1}(x\prime ). \\ {L}_t(x_t,R)=u^{-1}\left(w(x_t)u\left(l_t^c(R)+\sum _{x\prime \in \chi }{p}_t(x\prime |x_t,R)L^{t+2}(x\prime )\right)\right. \\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{0.33em}\hspace{0.33em}+(1-w(x_t))u\left(l_t^b(r)+\left.\sum _{x\prime \in \chi }{p}_t(x\prime |x_t,R)L^{t+2}(x\prime )\right)\right) \\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{0.33em}\hspace{0.33em}=u^{-1}(w(x_t)u\left(l_t^c(R)\right)+(1-w(x_t))u\left.\left(l_t^b(R)\right)\right) \\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{0.33em}\hspace{0.33em}+\sum _{x\prime \in \chi }{p}_t(x\prime |x_t,R)L_{t+2}(x\prime ) \\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{0.33em}\hspace{0.33em}=r_t(x_t,R)+\sum _{x\prime \in \chi }{p}_t(x\prime |x_t,R)L_{t+2}(x\prime ). \end{gathered}$$ (9)

In our analysis, we focus on three main aspects related to the optimal longevity and the resulting policy: (i) the change in the optimal longevity with respect to breast cancer risk and age, (ii) characterization of the risk-sensitive optimal policy as a threshold policy, and (iii) the comparison of optimal longevity and the corresponding QALYs. We state the assumptions (A1)–(A7) needed for the analysis and provide the justification for them in Appendix S1. The following theorem characterizes the change in the optimal longevity with respect to breast cancer risk and age.

Lemma 1. Under respective assumptions, the following hold independent of risk preferences

1. When (Al), (A2), (A3), and (A4) hold, L_t(s) is non-increasing in s,

2. When (A4), (A5), (A6) and (A7) hold, L_t(s) is non-increasing in t for all s.

Lemma 1 implies that a patient’s valuation of propositions (the maximal certainty equivalent) (i) never increases by the risk of breast cancer assessed from a mammogram, (ii) never increases by age for a given risk of breast cancer under any risk behavior. This is in some sense the consistency result such that the patients will never value an elevated risk of breast cancer or an older age under the same health condition more, no matter what the preferences are. We next define the double control-limit policy (DCL) for each t as follows.

Definition 1. A policy is a DCL if there exists θ1(t) ≤ θ2(t)

$$\begin{gathered} L_t(x_t)=L_t(x_t,R),\forall x_t\le {\theta }_1(t) \\ {L}_t(x_t)=L_t(x_t,F),\forall {\theta }_2(t)<x_t\le {\theta }_2(t) \\ {L}_t(x_t)=L_t(x_t,B),\forall {\theta }_2(t)<x_t\le N \end{gathered}$$ (10)

Our second main theorem proves the existence of a DCL policy under certain conditions.

Theorem 1. Suppose the transitions functions pt(xt+1|xt, F) and pt(xt+2|xt, R) satisfy (Al). If conditions given in Equations (37)–(41) in Appendix S1 hold, then, there exists an optimal DCL policy.

Theorem 1 extends the optimal DCL policy result for risk-neutral patients (Alagoz et al. 2013) to the case where patients are risk-sensitive. The clinical intuition behind the conditions stated in Equations (37)–(41) are as follows. Inequality (37) suggests that the minimum expected loss in longevity due to waiting two periods (decision R) in a higher risk score is no less than the maximal expected gain from biopsying in a lower risk score. In some sense, the condition indicates that the discrepancy between the expected cost of waiting two periods and the expected benefit from biopsying is increasing as the risk score increases. Inequality (38) suggests that the higher the cancer probability is, the more likely the patient is to move to a higher risk score in the future. Inequality (39) suggests that as the risk score increases, moving from the current risk score state to a lower one is less likely, while Equation (40) suggests that as the risk score increases, moving from the current risk score state to a higher one is more likely. Finally, Inequality (41) represents the cost-benefit trade-off between routine mammogram and biopsy relative to a follow-up action. The clinical intuition is that, as the risk score increases, expected net cost or benefit of biopsy in comparison to a follow-up decision is higher than the expected net cost or benefit of follow-up action as compared to a routine mammogram. We also numerically calculate the violations of these assumptions using real clinical data in Appendix S2.3 and find the violations are very small.

For any risk preference, let δ be the corresponding risk-sensitive optimal policy when the optimality equations as defined in Equations (5)–(6) are solved. For a reasonable comparison of outcomes due to various utility functions or parameter values of the same utility function, we need an alternative measure to the total expected certainty equivalent. An important outcome measure is total expected QALYs. One can obtain the total expected QALYs by reevaluating the optimality equations of (5)–(6) for policy δ under a risk-neutral assumption. We define V^δ as the total expected QALYs corresponding to the optimal policy δ under risk-sensitivity, that is,

$$V^{\delta }:=\left[\sum _{t=0}^{n-1}{\tilde{l}}_t\left(x_t,d_t^{x_t}\right)+l_n\right]$$ (11)

holds where the utility function is dropped from the expectation because of the risk-neutrality.

Lemma 2. Let δrn he the optimal policy under risk-neutral preferences and δrs be the optimal policy under risk-sensitive preferences. Optimal total expected QALYs corresponding to δrn, namely $V^{{\delta }_{rn}}$, is an upper bound for that of δrs such that $V^{{\delta }_{rn}}\ge V^{{\delta }_{rs}}$ holds for any risk-sensitive preferences.

Lemma 2 is a straightforward but important result from policy-making standpoint. It shows that the risk-sensitive preferences never produce better health outcomes than risk-neutral preferences when the measure of comparison for health outcomes is quality-adjusted survival duration. Moreover, the result re-emphasizes the difference between the policy planners and the individual patients in terms of preferences. A social planner may focus on maximizing the survival by having an objective valuation over the life expectancy, whereas, an individual patient may well prefer a lower overall life-expectancy because of her risk-sensitive attitude toward life-years.

4. Parameter Estimation

We use a large mammography database and medical literature in estimating the parameters for our model. We summarize our data sources in Table 1. The clinical data consisted of 65,892 consecutive mammography records collected from 18,270 patients between April 5, 1999 and February 9, 2004 at an academic, tertiary-care medical center (Chhatwal et al. 2009). The mammography records in our database were matched to cancer registry which provided the knowledge on whether the patient actually had cancer or not independent of the radiologist’s assessment. We used the clinical data in estimating the parameters related to the probability of cancer. There are two estimation tasks in relation to the use of probability of cancer in our model. The first is the estimation of the probability of cancer, and the second is the estimation of die transition probability matrices.

Table 1.

Data Sources

Relevant parameter	Source	Summary
${\tilde{l}}_t(x_t,B)$	Jemal et al. (2007), Arias (2006)	Age-dependent death probabilities for treated cancer
qt(De, Ca)	Haybittle (1998)	Age-dependent death probabilities for untreated cancer
qt(De)	Arias (2006)	Life tables from the National Center for Health Statistics of Center for Disease Control and Prevention
pt(Xt+1|Xt, F)	Clinical database with consecutive mammograms (Medical College of Wisconsin)	Probability of cancer and the transition probability matrices
d(F) = 0.1 days	Gram et al. (1990)	Disutility values for diagnostic decisions
d(R) = 0.1days
d(B) = 7 days*

Notes:

^*We assume that the disutility of biopsy linearly increases by age because of increased risk of complication and quality of life effect when performing biopsies at older ages.

We use the Bayesian Network (BN) of Chhatwal et al. (2010), an artificial intelligence model, to estimate the probability of cancer based on the features observed on a mammogram and the demographic factors of the patient. The probability of cancer, also referred to as the risk scores, is used as the state of our model; therefore, a more granular probability of cancer would make our results more insensitive to the potential biases in calculation. However, the granularity of the probability of cancer estimates is limited due to the unavailability of data as we explain next.

We were limited by the size of data in choosing the size of state space despite the fact that our clinical data is one of the few large databases with consecutive mammograms assessed at the finding level with both a biopsy and a registry match. There are several reasons for this limitation. First, our data comes from an actual clinical practice where the prevalence of breast cancer in general is very low. Therefore, most probability of cancer estimates would lie in the lower values that are very close to probability 0. This is exacerbated by the fact that if a patient was diagnosed with cancer at the time of the mammogram, the likelihood of cancer is high for a given mammography record. Therefore, such patients with high probability of cancer would be biopsied and subsequently treated leading to a low frequency of mammography records with higher probability of cancer in our database. Second, the estimation of the transition probabilities requires highly rich longitudinal data of mammography records and the need for data exponentially increases when more granular risk scores are used.

4.1. Estimation of the Transition Probability Matrices

Initially, we experimented with discrete probability distributions of various state sizes for estimating the transition probability matrices. The method of discrete distributions effectively worked for the choice of 101 states. When we expanded the state size, the method quickly failed because of the limited data. As an alternative to a discrete distribution for state transitions, we explored fitting of continuous distributions. Before fitting a distribution, we have characterized the empirical distributions, using boxplots and histograms. We used R software (R 2009) for distribution fitting and relevant statistical analysis.

Figure 2 depicts the boxplots of transitions for the initial six states when the state size is 501 with 0.2% probability intervals for x_t ∈ S_t \ {0, 500}. The depicted boxplots and the rest (not shown) suggested a tendency of right skewness for most of the states, especially when the sample size was large enough. Next, we plotted the empirical distributions using the histograms for the initial three states. Figure 3a and the rest of the histograms (not shown) suggested that the distribution mostly peaked at the value where the transition is starting. For example, if the patient’s risk score is 13 (i.e., S = 1000 and probability of cancer is 0.013), then she is more likely to stay in risk score 13 in the next decision epoch. Our empirical finding is also sensible from a clinical standpoint because no change or slow changes typically occur in consecutive mammography exams; therefore, remaining in the same risk of breast cancer from one mammography to another is a likely result. Note that such occurrence is common to most medical problems.

Figure 2.

Boxplot of Progression of Probability of Cancer between Stages

Notes: For the sake of clarity, we do not expose the outliers in the figure.

Figure 3.

Histograms of Progression of Probability of Cancer between Stages When S = 500

Notes: Dotted lines represent the starting point for each transition distribution and N stand for the sample size.

To investigate further, we drew the skewness-kurtosis plot as suggested by Cullen and Frey (1999) (Figure 4). The skewness-kurtosis plots indicate that three common right skewed distributions, gamma, Weibull, and lognormal could be considered as the candidate distributions. Among the three, we obtained the best statistics from fitting the gamma distribution. However, the fitted distributions did not necessarily have their mode at the value where the transition started. In order to enforce the location of each distribution, we used the Pearson type III distribution, also called the “three-parameter gamma distribution” with location (µ), scale (σ), and shape (ξ) parameters.

Figure 4.

Skewness-Kurtosis Graph When S = 500 [Color figure can be viewed at wileyonlinelibrary.com]

Three-parameter gamma distribution is, in essence, a type of gamma distribution where the location can be controlled with the parameter µ and is positively skewed. The probability density function is given by

$$f(x)=\frac{{\left|x-\mu \right|}^{\xi -1}\times exp\left(-\left|x-\mu \right|/\sigma \right)}{{\sigma }^{\xi }\times \Gamma (\xi )}$$ (12)

and has a finite lower bound. The only complication remaining was the estimation of transition probability distributions for higher risk scores, namely, the states corresponding to higher cancer probabilities. We handled this by fitting separate regression models into the shape and scale parameters over the state space and using the predicted values and the location parameters for assigning distributions. Upon experimenting the calculation of transition probabilities for different state sizes at a reasonable range, we chose the state size of 501 for presenting the results of numerical experiments in the rest of this study. We made this choice based on the data limitations described earlier and the statistical significance of the estimated parameters.

5. Numerical Experiments

We first solved our MDP model with risk-neutral assumption which helped us to specify the upper bound (as proven in Lemma 2) for our risk-sensitive MDP model. We achieved risk neutrality by simply assigning c = 0 for the power utility function as defined in Equation (2). Figure 5 shows the optimal policy as a function of age and the risk of breast cancer under risk-neutrality where we observe two threshold levels. The higher is the threshold to biopsy above which the optimal decision is to biopsy. The lower is the threshold to follow-up, where the optimal decision is to follow-up the patient in the short-term for the values that are above this level and below the biopsy threshold level. Next, we compare the optimal policies under risk-sensitive assumption to that of the risk-neutral assumption. For the clarity of presentation, we present only the biopsy thresholds in the rest of the numerical results.

Figure 5.

Risk-Neutral Optimal Breast Cancer Diagnostic Decisions

We performed sensitivity analyses on the parameters of the utility functions identified in section 2. We present the change in the optimal policy when the parameters are modified to represent various risk behaviors. Previous work has attempted to estimate the parameters of utility functions over longevity (e.g., see Pliskin et al. 1980, for an early work in the management literature). Most studies relied on 50–50 gamble and certainty equivalents in eliciting longevity preferences of patients. We briefly summarize the findings of selected work related to our parameter choices. For power utility, Stiggelbout et al. (1994) assessed the utilities of disease-free cancer patients and found the parameters to range from 0.3 to 1.78 (mean 0.74), while Miyamoto and Eraker (1984) reported the greatest density of estimated parameters to range from 0.4 to 2 among 46 coronary artery disease patients. Wakker (2008) argues that negative parameters may have been overlooked in estimation methods and uses a range from −1 to 2 for determining the best fit. Following Wakker (2008), we use c = −1 and c = 2 to represent moderate risk-seeking and risk-averse behaviors in our numerical analysis, respectively. For exponential utility, Abéllan-Perpiñán et al. (2006) studied 296 subjects and found the best fitting utility parameter as 0.48 while (Wakker 2008) reported 0.064 for the same. The assessed certainty equivalents of Verhoef et al. (1994) correspond to a range from −1.3 to 6.9 with a mean of 0.65. We use γ = −2 and γ = 1 to represent moderate risk-seeking and risk-averse behaviors in our numerical analysis, respectively. Finally, several studies successfully fitted logistics utility functions (e.g., see van Osch et al. 2006). We benchmark on previous work in determining a reasonable range of aspiration levels for the numerical analysis, which mostly lies within 5 to 30 years (Verhoef et al. 1994).

For the rest of this manuscript, we will avoid using the terms “risk-averse” or “risk-seeking” for the following reasons: First, our discussion in this chapter revolves specifically around the risk posture with respect to survival duration, which in essence is characterized by the curvature of the utility function, namely the convexity or concavity. The degree to which a utility function is convex or concave roughly represents the propensity toward the corresponding risk posture. Furthermore, since we focus on the analysis of decisions in the selection of medical therapies involving longevity trade-offs, the terms risk-averse and risk-seeking may be confusing. For example, risk-aversion may normally be interpreted as the aversion or proneness toward cancer or death. However, in the context of this study, because risk-averse utilities suggest preference of survival in the near future as opposed to survival in the distant future, we will call risk-averse behavior as “short-sighted” behavior. Similarly, because risk-seeking utilities imply a tendency to prefer survival in the distant future as opposed to survival in the near future, we will call risk-seeking behavior as “far-sighted” behavior.

5.1. Optimal Policy under Exponential Utility

Figure 6 shows the change in the optimal policy when the parameter γ for the exponential utility function of Equation (1) is varied. For clarity of presentation, we depict the biopsy thresholds in Figure 6a and the follow-up thresholds in 6b. There are several insights one can get from the risk-sensitive optimal policies. First, contrary to our initial expectation, we observed that the short-sighted behavior (as in behavior ${\mathcal{B}}_2$) is associated with less biopsies and the far-sighted behavior (as in behavior ${\mathcal{B}}_1$) is associated with more biopsies as compared to the risk-neutral behavior, that is, when γ = 0. Similar arguments hold for the follow-up decisions. In our experiments, we observed that the increasing non-negative γ values, or increasingly short-sighted preferences, may raise the threshold to biopsy and the threshold to follow-up suggesting a more conservative approach to diagnosing the disease. For a short-sighted person at a low risk score, opting out from the biopsy is preferred mainly because they will experience an immediate loss of utility due to anxiety, the possibility of complications, bleeding etc. (valued more under shortsighted preferences), which may outweigh the expected benefits in the future (valued less under short-sighted preferences). Second, we observed that the changes in the biopsy and follow-up thresholds due to changing risk parameters were more apparent at younger ages. At age 40, the effect of risk preference was highest and then it slowly diminishes approximating to the risk-neutral counterpart at older ages. This could be explained by the fact that at older ages, the lotteries had smaller payoff values because of the increased probability of death due to non-cancer causes and therefore the impact of preferences would be rather limited. Third, the optimal policy under short-sighted behavior exhibits first decreasing and then increasing biopsy and follow-up thresholds by age. Such observation deviates from the risk-neutral counterpart’s increasing thresholds by age. Increasing thresholds is clinically reasonable because treatment and its associated reduction in quality of life, may not be justified for an older patient with low risk scores because the patient may die of other causes before cancer causes mortality. When short-sighted behavior is introduced, more emphasis is placed on avoiding the disutility in the short-term. Because the exponential utility involves an attitude considering the absolute number of life-years at stake, younger patients will observe a more pronounced effect of the disutility as opposed to older patients and the disutility effect will reduce by age. According to Figure 6, the trade-off between avoiding the disutility and avoiding the mortality switches at around age 70 in favor of avoiding mortality when the patient is short-sighted with a γ value of 1.

Figure 6.

Risk-Sensitive Optimal Breast Cancer Diagnostic Decisions When Exponential Utility Function With Parameter γ is Used Notes: Positive parameter values characterize short-sighted behavior with increasing γ values implying increasing propensity to prefer short-term survival, negative parameter values characterize far-sighted behavior with decreasing γ values implying increasing propensity to prefer long-term survival, and a parameter value of 0 imply risk-neutrality.

Although counter-intuitive at first sight, the findings are in agreement with the existing literature such that when risk behavior is modeled over survival duration to determine the best choice, short-sighted behavior (risk-aversion) may favor the option with higher chances of survival/less morbidity in the short-term (Miyamoto and Eraker 1989). This was first illustrated by McNeil et al. (1978) in which two potential treatments for lung cancer were examined from the risk preference perspective: radiation therapy which had superior short-term survival vs. surgical treatment which had superior long-term survival. The findings suggest higher utility for radiation therapy when the risk posture is short-sighted (risk-averse) and higher utility for surgical treatment when the risk posture is far-sighted (risk-seeking). McNeil et al. (1978) and several subsequent studies (McNeil and Pauker 1979, Pauker and McNeil 1981) demonstrate the notion that when medical treatment alternatives differ in their survival outcomes in the near future vs. life in the distant future, short-sighted (risk-averse) behavior would favor the treatment with higher survival probability in the near future, whereas the far-sighted (risk-seeking) behavior would favor the alternative with higher survival probability in the distant future (Miyamoto and Eraker 1989).

5.2. Optimal Policy under Power Utility

We present the change in the optimal policy when the parameter c of the power/logarithmic utility function of Equation (2) is varied in Figure 7. We present only the biopsy thresholds because of the similarity in the trends between the biopsy and follow-up thresholds as in Figure 6a and b. Analogous to the results for the exponential utility functions, the threshold to biopsy and the threshold to follow-up were non-decreasing as the preferences of the patient became increasingly short-sighted and non-increasing as the preferences of the patient became increasingly far-sighted. However, there is a deviation from the exponential counterpart such that the magnitude of impact due to risk behavior does not vary by age. Although these results can be initially confusing, this behavior might be explained by constant relative risk-aversion as explained in the following.

Figure 7.

Risk-Sensitive Optimal Breast Cancer Biopsy Decisions When Power/Logarithmic Utility Function With Parameter c is Used

Notes: Positive c values characterize short-sighted behavior, negative c values characterize far-sighted behavior, and a c value of 0 imply risk-neutral behavior.

A measure for the curvature of the utility function, called the “Arrow-Pratt Concavity Index” (Pratt 1964), remains constant over the argument values for the exponential family. For the power family however, the level of concavity (or convexity) is relatively constant with respect to the argument values. Namely, the valuation of the lotteries depends on the relative value of life-years at stake which is consistent with a patient’s expression of her preferences as in behavior ${\mathcal{B}}_3$. In the context of breast cancer diagnostic decisions, because the payoff values and the expected future survival diminish approximately proportionally with increasing age, the valuation of the corresponding lotteries under power family would remain relatively constant. In other words, we may explain the proportional effect of risk-sensitivity on thresholds at different ages by proportionally diminishing payoffs and constant relative risk-aversion argument. Behavior ${\mathcal{B}}_4$ is also consistent with the power utility and our findings because older patients mostly have higher biopsy thresholds than younger patients independent of the tendency toward short-sighted or far-sighted behavior.

5.3. Optimal Policy under Logistics Utility

Next, we present the change in the optimal policy when one of the parameters, α or β of the logistics utility function, of Equation (3) is varied while the other is held fixed. Without loss of generality, we assume a ρ value of 1 which implies that the utility values would be bounded by 1 for our experiments. The chosen parameters imply a series of aspiration levels that are scattered between 10 and 40 life-years. To put the aspiration level in context, for the patient with behavior ${\mathcal{B}}_5$ who has a 9 year-old daughter, an aspiration level of 10 relates to the patient’s goal of seeing her daughter grow up to age 19. By contrast, an aspiration level of 40 for a middle-aged patient may correspond to a goal of reaching a very old age.

In Figure 8, we present the change in the optimal policies for various α values when β is held fixed at 3. As a is increased, the aspiration level increases and ranges approximately between 12 and 28 QALYs. Although subtle, we observe non-increasing thresholds for all age groups with increasing aspiration level. The changes with respect to the risk-neutral thresholds were more apparent at all but the oldest age group. The age range where the change is minimal compared to the risk-neutral case drifted toward the younger age group as the aspiration level is increased. This could be explained by the location of the inflection point at which the behavior would be close to risk-neutral. For example, consider a 40-year-old patient with behavior ${\mathcal{B}}_5$ and that her aspiration level is 11.91 as in Figure 8. Because this patient is relatively young and her expected life is much longer than her aspiration level, she will be less likely to prefer a biopsy as compared to a risk-neutral stand. As her aspiration level increases, we would expect that her tendency to prefer biopsy would increase because she has more at stake if she decides not to get a biopsy.

Figure 8.

Risk-Sensitive Optimal Breast Cancer Biopsy Decisions When Logistics Utility Function With Increasing α Values Are Used

Figure 9 shows a series of optimal policies for various β values when α is held fixed at 40. As β is increased, the aspiration level increases and converges to 40 QALYs. The aspiration levels varied approximately between 23 and 39 QALYs. When the aspiration level is increased, we observe an increasing far-sighted behavior or equivalently decreasing thresholds. At younger ages where the expected survival is close to 40, we observe the effect of short-sighted behavior such that the biopsy or follow-up thresholds were higher than the risk-neutral case. At the middle-age and the older age groups, the expected future longevity falls under the convex part of the logistics function; therefore, the thresholds are lower than the risk-neutral case.

Figure 9.

Risk-Sensitive Optimal Breast Cancer Biopsy Decisions When Logistics Utility Function With Increasing β Values Are Used

5.4. QALYs as a Function of Utility Parameters

We empirically explore the change in the total expected QALYs of the risk-sensitive optimal policies as compared to that of the risk-neutral policy (calculated using Equation (11)). Figure 10 shows the change in the population-averaged QALYs obtained from optimally solving the risk-sensitive MDP model with the pre-identified utility parameters. All experiments appearing in Figure 10 are consistent with Lemma 2 such that the optimal total expected QALYs corresponding to the risk-neutral preferences is an upper bound for the optimal total expected QALYs implied by any of the risk-sensitive preferences. Our numerical results add to the insights of Lemma 2 by quantifying the gap in terms of the expected QALYs between the risk-neutral and risk-sensitive preferences.²

Figure 10.

Sensitivity of Total Expected QALYs per Person Corresponding to the Optimal Policy Obtained From the Risk-Sensitive MOP Model over the Parameters of the Utility Functions

Notes: The marked points in (a)–(b) represent the parameter values where risk-neutral behavior holds. In (d), β values are changed for fixed α values and in (c), α values are changed for fixed β values.

The figures for both the exponential and power family of risk-sensitivity suggest a unimodal pattern for increasing far-sighted parameter values with a peak at the risk-neutral parameter value and a faster decreasing pattern for increasingly short-sighted parameter values. This suggests that a higher biopsy rate has a minimal effect on QALYs. The steeper decrease for short-sighted parameter values occurs because with increasingly short-sighted parameter values, more cancers will be missed, and therefore the benefits of early detection will not be realized. The QALY reduction in the far-sighted behavior for both the exponential and power families could be explained by the increased biopsy rate. As the preferences become increasingly far-sighted, more biopsies would be recommended, and therefore higher disutilities would be incurred.

For the logistics utility function, we conduct a one-way sensitivity analysis by varying one parameter and fixing the other. Although it is difficult to discern a clear trend, the results suggest that a higher aspiration level as governed by α when β is fixed reduces the gap between the optimal total expected QALYs of the risk-neutral preferences and that of the risk-sensitive preferences (Figure 10c). However, we observed a reversed trend for increasing β values when α is fixed (Figure 10d). Recall that increasing β values for fixed α would induce an increasing tendency to prefer a certain aspiration level as implied by the fixed α. In some sense, stronger preferences for a certain aspiration creates worse outcomes because it is not aligned with optimal long-term survival therefore explaining the reduction.

An observation based on Figure 10 is the relatively small QALY impact of risk-sensitive policies as compared with a risk-neutral one. Although small, QALY impact is significant in our context due to following. First, health screening policies are often built around small changes. For example, Pace and Keating (2014) report the benefit of screening mammography as “among 10,000 women aged 50 years undergoing annual screening for 10 years, approximately 302 would be diagnosed with invasive breast cancer or DCIS, between 56 and 64 women would die of breast cancer despite screening, and between 3 and 32 breast cancer deaths would be averted through screening,” which corresponds to a small life years gain per patient, however accumulates into deaths averted in a screening population. As Wright and Weinstein (1998) highlight, small gains in QALYs for screening population are equivalent to gains of years from breast cancer treatment with more than 73 million women over age 40 (in the US) being affected. Second, for patients with far-sighted behavior, the benefits from more aggressive biopsy decisions will be limited due to (i) the small incidence of cancer and (ii) the smaller utility loss with biopsy relative to the substantial loss from missed cancer. Hence, the impact for patients with far-sighted behavior will be limited. In contrast, the loss due to missed cancer as a result of short-sighted behavior outweighs the biopsy disutility, hence the QALY impact is more prominent. Finally, as opposed to small QALY impact, the cost impact will be substantial. We estimated the total expected lifetime costs for a woman with 0% risk of breast cancer at age 40 by assuming expected costs of $362 for follow-up imaging, $1258 for biopsy, no cost for a routine mammogram, and a 3% inflation rate (Poplack et al. 2005). Under the risk-neutral optimal policy, the total expected lifetime cost was $26,708.13. When the patient is short-sighted with γ = 1 under exponential utility, the total expected lifetime cost was $9799.47, while it was $56,482.04 when the patient is far-sighted with γ = −2.

5.5. Reverse-Engineering the Implied Risk Behavior in Clinical Practice

We also investigate the risk behavior implied by the current clinical practice using our mammography database. For this purpose, we identified two performance metrics to evaluate various risk-sensitive behaviors where we define a set of parameter values over the utility set. The first metric, called the percentage of decisions in agreement, measures the percentage of mammograms for which the action taken by the radiologist and the action recommended by the risk-sensitive optimal policy concurs. The second metric, called the percentage of more aggressive diagnostic decisions by the radiologist measures the percentage of mammograms among all mammograms with no cancer association where the radiologists recommended a more aggressive diagnostic action than the risk-sensitive optimal policy. More specifically, the numerator is the sum of the (i) number of mammograms where a biopsy or follow-up is recommended by the radiologist while the optimal policy is regular screening and (ii) number of mammograms where a biopsy is recommended by the radiologist while the optimal policy is follow-up. We divide the sum by the total number of mammograms without a cancer association to produce such measure.

We find that the percentage of decisions in agreement ranged between 65% and 75%. Although performing the estimation on the whole database is reasonable, we did not include the mammograms for which the risk of breast cancer is very close to 0% risk (or no finding). This is because clinical practice encounters a significant number of mammograms with probability almost 0 and the associated decision is clearly to do nothing, so these cases may dominate our results. Risk preferences would be more relevant when the risk of breast cancer is not very low where the preference of the patient may actually sway the decision. Therefore, we limit our analysis to the mammograms where the probability of cancer is estimated to be above the overall cancer detection rate of 0.6% (Yasmeen et al. 2003). Note that the sensitivity analysis over the choice of this interval or an analysis at the level of individual radiologists reveal similar insights, suggesting the robustness of our findings.

In Figure 11, we graph the two metrics we have identified for a series of parameter values over the utility set. We observe that the percentage of radiologists’ decisions in agreement with the optimal policy is higher for more far-sighted parameter values with a peak of 38.4% at the most far-sighted preferences for the exponential utility (Figure 11a), 34.3% at the most far-sighted preferences for the power utility (Figure 11b), and 36.4% at the highest aspiration level for the logistics utility (Figure 11c). Figures 11a, b, and d also suggest that radiologists deviated from the optimal actions most of the time in favor of more aggressive diagnostic actions as compared to the optimal policy for various parameters and utilities. The smallest deviation again has occurred for the most far-sighted parameter values. Although the differences in percentage values are small for various parameter values and utilities, exponential family of utilities has the best performance in terms of capturing the risk behavior implied by the clinical practice from which our data is derived. This finding reinforces previous research which suggests linear/exponential utility functions are better approximation for preferences over survival duration when averaged over populations (Miyamoto and Eraker 1989). Moreover, we find that current clinical practice exhibits a tendency toward far-sighted behavior, suggesting more aggressive use of diagnostic actions than a purely risk-neutral policy.

Figure 11.

Comparison of Risk-Sensitive Optimal Policies Under Various Parameter Values to the Actual Clinical Practice

Notes: The marked points in (a)–(d) represent the parameter values where risk-neutral behavior holds.

Our approach to revealing preferences from actual decisions is similar to that of Abellán-Perpiñán et al. (2006), where authors minimize the deviation between predicted and observed life-year related choices to find the exponential and power utility parameter values for an experimental population. We emphasize that our elicitation intends only to explain the tendency of the observed behavior in the context of its resemblance of risk-sensitive decisions. Although our approach uses the radiologists’ recommendations in determining the preferences, the observed behavior may also reflect patients’ preferences if shared decision making took place. Aside from an explanation of medical decisions due to preference-related factors, there may be variety of factors on the demand side (e.g., patient’s out-of-pocket payments) or on the supply side (e.g., fear of litigation).

6. Discussion

In this study, we investigate the role of risk preferences in the context of post-mammography diagnostic decisions through developing a risk-sensitive optimization model, which maximizes the total expected utility of a patient defined over longevity. We account for the morbidity and mortality associated with diagnostic actions by subtracting disutility values from the expected quality-adjusted survival duration. We explore various utility functions which represent simple characterizations of risk-postures. We use a large mammography database to estimate the parameters of the risk-sensitive MDP model and conduct extensive sensitivity analyses for various utility functions to gain insights into how risk-sensitive decisions would optimally be made.

Several aspects of our results deserve attention. First, in compliance with the supporting evidence in the medical/psychology literature as described in section 5, our analysis suggests more conservative use of diagnostic actions with increasing short-sighted behavior and more aggressive use of diagnostic actions with increasing far-sighted behavior. From a clinical practice standpoint, our insights into the trade-off between the life-years in the short-terms vs. long-term in terms of patient preferences could help physicians better communicate with their patients various options that are available after mammography. Second, we find that short-sighted behavior under exponential utility is associated with first decreasing and then increasing thresholds to biopsy with increasing age, which contradicts the clinically intuitive non-decreasing thresholds to biopsy under risk-neutrality. Although contradicting clinical intuition at first sight, less aggressive biopsying of younger patients as compared to older patients is reasonable under short-sighted behavior because younger patients’ valuation of disutility in the short-term may be higher than their valuation of life-years at stake in the long-term. As the patients get older, the discrepancy between the valuation of life-years in the short-term vs. long-term reduces and when the two coincide, we start observing non-decreasing thresholds similar to the risk-neutral preferences. Third, a patient’s choice of expressing their valuation of uncertain outcomes in terms of absolute vs. relative number of life-years at stake (as in the exponential vs. power utility, respectively) determine how thresholds under risk-sensitive preferences compare to the thresholds under risk-neutral preferences. We find that the choice of absolute valuation leads to a higher threshold discrepancy in younger vs. older patients whereas relative valuation leads to a similar effect on thresholds over different age groups. Lastly, evaluating our model using the mammography data from an actual clinical practice reveals an interesting insight. We find evidence on more aggressive use of follow-ups and biopsies than the purely risk-neutral policy would suggest. In the context of our modeling, more aggressive diagnostic decisions are consistent with far-sighted (risk-seeking) behavior for medical decisions.

6.1. Potential Translation into Clinical Practice

Implementing our findings in an actual clinical decision context depends on (i) targeted use of information or information technologies by patients, and (ii) successful elicitation of patient preferences as was also discussed in section 2.2.1. The evidence on how to achieve the two objectives is abundant. We discuss how patient-oriented decision aids enable preference-sensitive decisions in actual clinical practice (O’Connor et al. 1999).

The interest in measuring and including patient preferences in the delivery of healthcare via patient-oriented decision aids has risen in the last two decades (e.g., see Brennan 1998, Ruland 1998, for early discussions). Based on International Patient Decision Aid Standards (IPDAS), the Cochrane Collaboration³ reviewed the randomized controlled trials on preference-sensitive treatment and screening decisions. The review identified 115 studies involving 34,444 participants and concluded that the use of decision aids helped patients clarify their values and make more informed decisions (Stacey et al. 2014). There is also a vast literature on the successful implementation of research similar to ours.⁴

Various approaches to patient-oriented decision aids are available. The decision aids may be in the form of a web tool available to the public, an application in the personal health record (PHR) for use by the patient, a clinical decision support (CDS) available to the physician to discuss with the patient, a pamphlet, or a video. We refer the reader to Sepucha and Mulley (2009) for an extensive discussion of the role of decision aids and various approaches. However, we note that the increasing use of the internet and the widespread diffusion of electronic health records (EHR) or PHRs into healthcare present a great opportunity for the involvement of patients in preference-sensitive decisions. Specifically, successful dissemination and use of patient decision aids for breast cancer across 70 cancer centers Silvia et al. 2008), EHR initiatives for decision aid implementations led by large medical centers (Sepucha and Mulley 2009), and successful commercial (e.g., Health Dialog, Health-wise) and internet-based models (e.g., decision aids developed by Mayo Clinic for use in chronic care) are some indicators that preference-sensitive decisions can take hold.

As we have previously discussed, an important problem to address for integrating risk-sensitive decisions into CDS is value clarification. The labor-intensiveness of elicitation may present as a challenge in value clarification (Charles et al. 1997). However, increasing consumerism in healthcare, push for shared-decision making initiatives, regulatory efforts, spreading health information technologies, and patients’ increased access to health information could make elicitation of preferences and their incorporation into EHRs or PHRs a viable option (Cok and Correa-de Araujo 2011). For example, a physician may use a CDS designed for elicitation that is available in the EHR during care encounter. The EHR and the CDS may also carry the information for physicians to make the linkage between patient preferences, guidelines, and the existing scientific evidence. Alternatively, a patient may access to her PHR and interact with the elicitation tool to reveal her preferences to be stored in her records. The revealed preferences may be updated as the patient desires. Moreover, technology-based assessment via a PHR may provide the desired privacy and the time flexibility for successful elicitation (Brennan 1998).

6.2. Policy Implications

An alternative to integrating patient preferences at the point of decision is their inclusion in the clinical practice guidelines (CPG) (Krahn and Naglie 2008). The CPGs are systematically developed recommendations about the care of patients in specie clinical circumstances (Lohr and Field 1992). In general, CPGs are created considering an average patient and a typical decision-making context; therefore, incorporating preferences into CPGs may not be a simple task. However, a systematic approach to guideline making can assess the role of patient preferences and develop preference-based CPGs (Boivin et al. 2010). To that end, several studies such as Dirksen et al. (2013) present a protocol for integrating patient preferences into the health policy decisions and CPGs.

The issue of translating quantitative rules derived from mathematical models of systems into qualitative guidelines useful for practitioners is complex and challenging in any context, and healthcare is not an exception. However, our findings can still be a useful starting point for policy making in preference-sensitive breast cancer diagnosis. In particular, our findings could inform the policy making around the creation of preference-sensitive CPGs in two levels. First, our study provides a high level guidance on optimal policies when patient preferences are considered. The CPGs can clearly state the prominence of preferences in the breast cancer diagnosis and recommend options for preference assessment ranging from an informal discussion with patient to computer-based assessment of patient’s valuation of alternatives. Bridging the gap between patient-oriented decision aids and CPGs could support practices that are both evidence-based and are also cognizant of patient preferences (van der Weijden et al. 2012). Second, at the individual decision-making level, physicians and patients could have an informed discussion on the extent and the direction of impact of behavior on short- and long-term outcomes. Assuming the patient preferences are assessed and is known for the physician, in an individual care encounter, a CPG can include a discussion of the range of possibilities that can occur as a result of preference-sensitive decisions. For example, our numerical analysis reveal the differences in managing older vs. younger patients when the patient is concerned about a relative life years at stake as opposed to being concerned about the absolute number of life years at stake. In the former case, deviation from risk-neutral policy is similar for the older vs. younger patients under risk-sensitivity whereas, in the latter case, the deviation is minimal for older patients while it is large for younger patients. Overall, our study, in some sense, provides preference-based, evidence for guideline developers through the use of a decision model that explicitly accounted for utilities—an advocated improvement over evidence-based medicine (Krahn and Naglie 2008).

6.3. Conclusion

In conclusion, patients may have varying preferences with regard to health related decisions (Arora and McHorney 2000) and our results show that the optimal policies are sensitive to the risk preferences to some extent. Although creating a clear guideline for respective risk behavior may be difficult, shared decision-making that is increasingly advocated (Buist et al. 2013, van Ravesteyn et al. 2012) necessitates a better understanding of optimal policies under various risk behaviors. Note that while this study focuses on post-mammography diagnostic decision-making, the same framework can easily be applied to other problems in medical decisions. Once incorporated into practice, policy makers should also bear in mind that a welfare loss in terms of survival duration is inevitable as evidenced by our structural and empirical results. As the shared decision-making becomes more of a part of the healthcare, loss in total expected QALYs at the population level will ensue. Because it is important to recognize the fact that people may have varying preferences for gambles involving length of life (McNeil and Pauker 1979, McNeil et al. 1978), identifying the utility functions that represent the risk behavior is also important (Nease 1994). We find that the exponential utility better represents observed behavior in post-mammography diagnostic decisions as compared to log/power or the logistics utility functions. Although the exponential family of utility functions are not always valid in describing individual behavior, they may be used as an approximate representation of risk posture in applied decision-making (Miyamoto and Eraker 1989).

Given the importance of optimal medical decisions from a policy and management perspectives (e.g., for guideline making both perspectives matter) as well as the above-mentioned implications in management of a healthcare organization, we study the role of patient preferences in optimal sequential decisions. Incorporating patient preferences in health-related decisions can make the care delivery effective (Kaplan et al. 1989, Stewart 1995) and possibly more efficient (Arterburn et al. 2012, Dentzer 2013). In addition, literature reports that patients who perceive that their preferences are not adequately incorporated into the decision making are more likely to build negative attitude for the provided care and become dissatisfied in many decision contexts (Williams and Fleming 2011) including breast cancer-related decisions (Lantz et al. 2005). The increasing emphasis of payments based on patient satisfaction (Ramhmdani and Bydon 2016) makes patient preferences even more important for healthcare providers. Evaluating optimal policies under risk-sensitivity and quantifying the impact in a sequential decisionmaking context could provide the policy-makers, providers, patients, and the researchers the valuable information about the implications of incorporating preferences into the medical decisions.