Deliberation before determination: the definition and evaluation of good decision making

Abstract

Objectives  In this article, we examine definitions of suggested approaches to measure the concept of good decisions, highlight the ways in which they converge, and explain why we have concerns about their emphasis on post‐hoc estimations and post‐decisional outcomes, their prescriptive concept of knowledge, and their lack of distinction between the process of deliberation, and the act of decision determination.

Background  There has been a steady trend to involve patients in decision making tasks in clinical practice, part of a shift away from paternalism towards the concept of informed choice. An increased understanding of the uncertainties that exist in medicine, arising from a weak evidence base and, in addition, the stochastic nature of outcomes at the individual level, have contributed to shifting the responsibility for decision making from physicians to patients. This led to increasing use of decision support and communication methods, with the ultimate aim of improving decision making by patients. Interest has therefore developed in attempting to define good decision making and in the development of measurement approaches.

Method  We pose and reflect whether decisions can be judged good or not, and, if so, how this goodness might be evaluated.

Results  We hypothesize that decisions cannot be measured by reference to their outcomes and offer an alternative means of assessment, which emphasizes the deliberation process rather than the decision’s end results. We propose decision making comprises a pre‐decisional process and an act of decision determination and consider how this model of decision making serves to develop a new approach to evaluating what constitutes a good decision making process. We proceed to offer an alternative, which parses decisions into the pre‐decisional deliberation process, the act of determination and post‐decisional outcomes.

Discussion  Evaluating the deliberation process, we propose, should comprise of a subjective sufficiency of knowledge, as well as emotional processing and affective forecasting of the alternatives. This should form the basis for a good act of determination.

Introduction

Two ideas have influenced the practice of medicine over the last few decades. The first is evidence‐based medicine ¹ , the attempt to use the results of empirical science as an overt basis for making judgments about the provision of clinical care. The second has been shared decision making ² , the attempt to involve patients in decision‐making tasks, especially where decisions, in the face of uncertain or equivocal evidence of benefit, are sensitive to personal preferences. The result has been unprecedented interest in how to involve patients in decision making and how to measure success. It has become clear that it is difficult to know what would constitute success in terms of process and outcome. In summary, there remains a debate about the characteristics of a ‘good’ decision.

How to measure this construct would depend on definition. We note of course that many measurements exist which aim to address important elements that surround the task of decision making, such as conflict ³ , satisfaction ⁴ and regret. ⁵ In addition, we are aware of more recent efforts to describe and measure a concept which has been called ‘decision quality’, where the effort is to pinpoint the nature and characteristics of a good decision. ⁶ At first, this seems an eminently feasible task but as we elaborate here, this first impression obscures an inherently complex and difficult task and one which hampers progress in the development, evaluation and implementation of decision support interventions. We begin by providing a context and an explanation for interest in defining the characteristics of a good decision. We proceed by commenting on existing definitions of a good decision, critiquing those which fail to address a concern about post hoc evaluations, comment on their definition of knowledge and their lack of distinction between the deliberation (pre‐decisional) phase, and the act of decision determination – of choosing an option. We end by proposing a framework for evaluating the deliberation phase which we hypothesize, if done well, may lead to a good decision determinations.

The emphasis on involving patients in decisions is a relatively recent development in medicine. The new emphasis on participation in decisions is partly driven by the realization that in many clinical scenarios, scientific evidence for the effectiveness, safety and cost‐effectiveness of interventions is often insufficient to determine a clear superiority between alternative approaches. Evidence‐based medicine has emphasized the need to integrate empirical research data into clinical practice. Paradoxically, this empirical focus has drawn attention to the considerable uncertainty that surrounds the majority of interventions in health care and therefore, that individual preferences should be given voice in decision‐making processes. ⁷

As well as the scientific and economic imperative to involve patients in decisions, there exists the additional ethical imperative of ensuring that patient views are given a central place in clinical encounters. ⁸ This marks a definite shift away from paternalism to a process where patients are offered support to become actively engaged in decision‐making tasks. ⁹ Terms such as ‘shared decision making’, ¹⁰ ‘evidence‐based patient choice’ ¹¹ and ‘informed choice’ ¹² have become common parlance. ¹³ Several approaches have emerged to promote shared decision making, including professional skill development ¹⁴ and the design of developed decision support interventions, often called decision aids. ¹⁵ These interventions are information resources that are designed to help patients understand that options are available and to become involved in decision‐making interactions.

Given the prominence of this trend to involve patients in decision making and the continuing lack of consensus over measurement, ¹⁶ , ¹⁷ we wish to influence the debate on how to measure the construct of a good decision: how can such a concept be defined and measured? A better answer to this conundrum would help make progress in designing and evaluating decision support and doctor patient communication strategies, as well as improve decisions and decision‐making processes. ¹⁸ , ¹⁹

Our intention is to examine proposals to measure the concept of good decisions and to explain our concerns. We examine a number of different approaches to evaluating decisions. We pose the questions of whether a decisions can be judged good or not, and if so how this goodness might be measured. We hypothesize that decisions cannot be measured by reference to their outcomes and offer an alternative means of assessment, which emphasizes the process of deliberation, rather than the decision’s end results. We propose decision making comprises a pre‐decisional process prior to arrival at post‐decisional outcomes and consider the implications for evaluation. We consider how this model of decision making may serve to develop a new approach to evaluating what constitutes a good decision. We start by considering a range of definitions before offering our perspective on some of their converging aspects.

Existing proposals to measure the concept of a good decision

In a series of essays, several experts considered the conceptual question ‘what is a good decision’. ¹⁸ Although they offered neither formal definitions nor suggestions as to how to measure ‘good decisions’, a notable consistency was the view that it was important to avoid dependence on the outcomes when evaluating the quality of decisions. By outcomes, they referred to either physical outcomes (e.g. a leg has been amputated and gangrene prevented) or emotional outcomes (e.g. the patient feels anxious). Good (or bad) decisions may have good or bad outcomes by virtue of chance, at the level of individual consequences. It is important to stress that this refers to specific decisions and where stochasticity has relevance. At population levels, either a collection of individuals or multiple repeated decisions by the same individuals, probability exerts an influence and outcomes become highly relevant measures of success, as is the case in experimental trials. However, we are interested here in working at the level of a specific decision taken by one individual, where probabilities cannot be patterned by examining large numbers.

This view that outcomes cannot be relied on as measures of good decisions is not novel. ²⁰ , ²¹ but it is remarkable how little attention has been given to this view. Fisher and Fisher ¹⁸ state that ‘because a good or bad outcome may powerfully influence perceptions (of decisions)…such a judgement is best made before the outcome is known’ (p. 190). This conclusion is especially true in situations where the evidence of harm versus benefit is finely balanced: such situations of equipoise are precisely those where shared decision making is most relevant and where, by definition, the decision process, rather than the probability of a beneficial outcome, as determined by clinical trial evidence, predicts choice. To foster informed decision making on behalf of patients, a desirable process has to be delineated, measured and evaluated. The outcome, while important in many ways, is less imminent in this respect. Furthermore, due to the stochastic nature of medical outcomes, adding the outcome to the measure may well taint it in a way that is uninformative about the process of making a good decision.

A reliance on outcomes as the measure of a good decision is rendered even weaker when temporal issues are also considered. Let us consider a patient who has breast cancer, facing a choice between mastectomy and breast conservation surgery. Although there are many differences between the surgical approaches, long term survival after both procedures is equivalent and this is therefore a difficult decision. We may suggest then that a good decision should be measured by assessing outcomes. Should we ask the patient how she feels the day after the operation, or might the assessment be better done the following week when they are partly recovered, or when a year has passed and she has adapted and remains well? What if the patient is well for 6 months but then suffers a local recurrence of cancer (more likely after breast conservation)? Judgement about the outcome would change dramatically, and so, by association, would judgement about whether the initial decision was good or not. Perceptions about the wisdom, or otherwise of a decision to have surgery, will be influenced heavily by outcomes as they unfold in time. In summary, post hoc assessments, based on the outcomes of decisions, are unsafe measures of good decisions. This much is agreed, but we are left asking what remains: what can be used to describe the characteristics of good decisions, at the time they have to be made, i.e. by real patients in real clinical situations.

Ratliff et al. ¹⁸ , in describing a good decision, argued a definite need for knowledge – that the person is informed – and that, in addition, decisions should reflect personal preferences. While these elements are present in most definitions of a good decision the notion that outcome should be removed from the definition, which Ratliff advocated, is most often not pursued. Marteau, in 2001, proposed the construct of ‘informed choice’, and wrote that: ‘an informed choice is one that is based on relevant knowledge, consistent with a decision‐maker’s values and behaviourally implemented’. ²² These views were echoed in more recent work, as O’Connor introduced the term ‘decision quality’ in 2003 and proposed that: ‘decision quality can be measured by…knowledge about the options and outcomes, realistic perceptions of outcome probabilities, and agreement between patients’ values and choices’. ²³ Sepucha et al. ⁶ , developing this idea has proposed that ‘…the quality of a preference‐sensitive clinical decision can be defined as the extent to which the implemented decision reflects the considered preferences of a well‐informed patient’ (p. 262).

Inherent in all three of these definitions, to lesser or greater extents, are three similarities. The first is the lack of distinction drawn between the process of arriving at a decision, we use the term deliberation, and the decision taken, we use the term determination for this act. Second, that the measurement should be one based on a standard of adequate knowledge and, third, that it should be consistent with the individual’s views, attitudes, values or preferences (the terms used are often interchanged). These definitions are remarkably consistent but we contend that they are unsatisfactory.

Critique of existing approaches to measuring the concept of a good decision

In this section, we offer a criticism of the proposed approaches of measuring the construct of a good decision. First, we propose that existing approaches have not adequately distinguished deliberation about decision making from the determination of the decision. Second, we consider difficulties with the measurement of both knowledge and preferences and, thirdly, we outline the potential development of a measure to evaluate the deliberation phase of decision making, in a fashion that overcomes the above mentioned issues.

Distinguishing deliberation from determination

Insights into the difference between the processes of deliberating about decisions and the determination of decisions was provided by interviewing patients as they spoke of their efforts to make sense of being involved or not in efforts at shared decision making. ²⁴ Existing measurement proposals fail to clearly distinguish between the process of arriving at a decision, deliberation, and the fulcrum of the event, that is, the determination of a decision and the consequential events. These steps are distinguishable and, moreover, need different evaluation approaches. Efforts so far to define the measurement of a good decision have not clearly separated these steps. The table parses these elements into two phases: deliberation and determination (see Table 1).

Table 1.

 Distinguishing decision‐making phases

Decision‐making phases	Description	Measurement elements
Deliberation	1. Information search 2. Knowledge gain 3. Appraisal of knowledge sufficiency 4. Imagining counterfactuals 5. Affective forecasting 6. Preference construction	Perceived sufficiency of information gain, knowledge gain, perceived clarity about the nature, loss and gain of counterfactuals based on preference construction. Measure this phase as the construct of ‘deliberation’, perceived ease or difficulty, and assess
Determination	Integrating deliberation inputs and making a choice (i.e. the determination), prior to enacting the decision.	Record the option chosen; evaluate rationale. Measure by assessing evaluations of the enacted decision. For example, is there regret? Or rejoice?

Having distinguished these phases, we examine again the measurement proposals of informed choice ²² and decision quality ⁶ , and notice that they do not explicitly distinguish decision deliberation and decision determination. Marteau used the term ‘behaviourally implemented’, suggesting that it was the determined decision, or an action (based on a decision taken) performed prior to the assessment of ‘informed choice’ that the focus of her proposed measure. O’Connor’s definition of ‘decision quality’ contains the phrase ‘agreement between patients’ values and choices made’, again implying that the assessment is focused on the determined decision and not on the deliberation. ²³ Sepucha used the term ‘implemented decision’ as the focus of ‘decision quality’, again implying the focus to be on the determined decision and not on the deliberation phase of decision making.

However, attempts to improve decision making typically include efforts to increase the amount and salience of information and attempts to help individuals explore what is important to them as individuals, in other words, to examine personal values and preferences. Decision support interventions are designed to help the process of deliberation, to help people facing difficult decision gain insights that are based on increased knowledge, as well as forecasts of how they might feel about imagined futures, and their considered preferences about a range of imagined counterfactuals (what if situations). It is possible to intervene in these processes, by introducing high quality information, improving the discourse with health professionals and by introducing deliberation tools that help rank and weigh the relevant attributes In other words, we can improve deliberation processes in the hope that this leads to an improvement in the decision determined. Here then is the nub: decision making cannot in our view be evaluated by a method that does not distinguish between the deliberation and the determination phase. We need to evaluate both the decision making (the perceived or observed process) and the determination (whether or not the decision itself is considered ‘good’) by whatever parameters we choose to use.

The problem with knowledge

Most attempts at defining good decision making propose that knowledge about options, their attributes and consequences is necessary. This seems logical: choices made in the absence of any knowledge are mere guesses. Yet, when we attempt to evaluate this construct, problems appear. What is meant by knowledge, knowledge about what for instance? Knowledge about the nature of outcomes, what it might be to experience them, or do we mean the probabilities of those outcomes? Or knowledge about the features (attributes) of short, medium and long term future states, given possible pathways? Or knowledge about perceived forecasts of different counterfactual states? Knowledge about how adaptation might change our future judgements about the utility of those states? Indeed, these are only a selection of knowledge areas we could specify. And the questions do not end there. Can an optimal degree of search for information – a proxy for knowledge – be set: for when to stop the search? Can we specify the level of knowledge that is necessary and sufficient (different for every decision scenario), or is this a quantum that we allow the decision maker to self‐determine (e.g. I now know enough)? Do we assume that being informed equates to understanding or does this not worry us? Sepucha et al. have made progress by developing a ‘decision quality’ measure, which relies on an assessment of selected key knowledge items, typically about the nature and outcomes of options. ²⁵ It will be important to see how these perform across different patient characteristics. An additional endeavour, given the decay of knowledge over time, is to determine the correct assessment point ²⁴ , and to state what time lag between decision and knowledge assessment is reasonable. To summarise, although appearing attractive to measure, knowledge as a necessary component of a good decision – both as an evaluation of the effectiveness of deliberation and as an evaluation of the determined decision – may be difficult to operationalize and, at a deeper level, may reflect an untested assumption – that a standardized, pre‐specified level of knowledge is necessary in order to achieve a high level of deliberation.

The fluid nature of preferences

Personal preferences are also proposed as important determinants of good decision making but a lack of terminological precision bedevils this area. We note that many terms are used, terms such as values, preferences, attitudes, beliefs, opinions and reasons. ²² , ²³ , ²⁶ , ²⁷ Often the term is used to describe high level constructs or attitudes – such as truth‐seeking or risk aversion or a general personal stance on, say, the avoidance of medication. The term value has also been used to describe the concept of utility or benefit assessment ²⁸ and part of expected utility theory. ²⁹ The term preferences is often used interchangeably with values but seems to refer most often to ranking one or more attributes of options or that some options have a more convincing or more worrying range of disadvantages, e.g. drug side‐effects that a patient prefers to avoid, and hence avoids the associated option. Such preferences are malleable to preference reversal due to variations in the ways they are presented (e.g. a how many lives will be saved by a treatment option versus. how many lives will be lost, ³⁰ , and therefore it is doubtful that they constitute a valid and consistent measure of a good decision. ³¹

Another widely recognized difficulty is the view that preferences are constructed rather than revealed. Lichtenstein and Slovic ³² present evidence showing preferences are constructed as individuals gain information, obtain representations and consider alternatives against each. Furthermore, people are often inconsistent. Work on transitivity, one of the axioms of expected utility theory, has demonstrated that people who prefer A to B and B to C often do not follow the mathematical logic of preferring A to C, because alternatives do not share the same features. The process of considering preferences affects their construction and manifestation (the underlying assumption of advertising) and so preferences change as contexts and circumstances change, which is almost bound to happen in health‐care contexts. The fluidity, malleability and their constructed nature make it difficult to put preferences themselves forward as a measurement criterion for a good decision. Nevertheless, we do support the notion that individuals need to differentiate between poorly considered preferences and preferences formed after more rounded considerations, in other words, after sufficient deliberation. ³³

Evaluations of the deliberation phase

It follows from our arguments that evaluations of decision making need to consider and define both the deliberation and determination phases of decision making, and further consider whether a relationship exists between them. For the purposes of this article, we focus on the deliberation phase. Although we pointed out the difficulties of setting out that the acquisition of a pre‐defined standardized amount of knowledge is necessary in order to achieve good deliberation, we also accept that decisions made without any information whatsoever are mere guesses. It follows therefore that an understanding that options are available is the first and fundamental issue, data about the nature of the decision, the relevant option set, the advantages and disadvantages of options and their associated outcomes, and the probabilities of such outcomes occurring would confer benefit. Given mental processing limitations, and the fact that reasoning is typically based on scarce inputs ³⁴ it might suffice to allow the decision maker to articulate what it is she seeks in an alternative and what she wishes to avoid. An objective measure of knowledge may not be instrumental in capturing idiosyncrasies in the desired amount of information. We therefore arrive at the view that the minimum achievement for deliberation is the awareness that options exist and that their attributes are relevant. Further deliberative work would lead to greater differentiation between options.

Second, we note that most choices, especially difficult choices, involve integrating both cognitive and emotional contributions. The work of imagining how different futures could play out, the attempt to envisage and in so doing experience counterfactual scenarios is a key step in active deliberation. ³⁵ In addition, it may be helpful to examine the patients’ current and forecasted feelings towards differing counterfactual futures ³⁶ , and explore to what extent known biases have been moderated.

We propose that a good decision process would therefore provide opportunities to undertake these steps, prior to a decision‐making task. It seems obvious but nevertheless important to state, that these processes, of providing information, of working to imagine differing counterfactual futures, of constructing preferences, as well as eliciting current and forecasting future affective responses are co‐dependent on each other. A decision maker cannot form a view about the attractiveness of alternatives unless he or she has data about their attributes and probabilities; similarly, cognitively processed information alone does not enable a decision maker to form a view and how he or she feels about the imagined possibilities are relevant and important inputs into the determination of a decision.

We therefore propose that these are prerequisites for an effective deliberation process. We thus define deliberation as requiring and accepting a state of equipoise, that relevant options exist, and that these options need to be understood and considered. In addition, to be able to differentiate these options prior to constructing preferences, the following steps would confer advantage: gain information about the attributes of options in formats which are accessible (characteristics, process and outcome probabilities); imagine counterfactuals (what if scenarios) which provide insights into the possible consequences of options and the impacts of current emotional reactions and affective forecasts. It is important to note here that significant scientific advances have been made over the last decade in how to portray risk ³⁷ , ³⁸ and to use language so as to maximize understanding. ³⁹ Effective deliberation methods would help individuals address these issues and allow them to understand when they are ready to proceed to a decision determination phase.

Measurement could be based on a number of data collection methods. The deliberation process could be observed and analysed, although as most deliberative processes occur over time, the research burden is considerable. Another method is to assess the patient’s perception about and recall of the deliberation steps. Have they received information and was it sufficient for them to be able to conceptualize the choice, differentiate the options and their attributes? Have they been able to imagine the relevant counterfactuals and consider their consequences, and assess their emotional reactions, sufficient for them to judge themselves to be in a position to determine a decision? Note that we do not propose a prescriptive threshold: we do not decree how much information or how much deliberation is required but we do suggest that it is possible to collect data on the levels of deliberation achieved. Our proposal will likely lead to a measurement that while it may will be inevitably subjective, having no pre‐set benchmarks or gold standards, it will have the merit of being a gauge of an individual’s journey from a position of not understanding that options exist, that they need to be considered, that each option has attributes, and so on, to a position of having deliberated to a level where they feel confident in having arrived at a point where they feel able to determine a decision.

Conclusion

Existing measurement proposals fail in our view to make a clear distinction between the work of deliberation and the step of determining a decision, between a decision‐making process and the decision itself. We feel that clarifying this distinction allows us to make progress towards a measurement that specifically addresses the work required in active deliberation. Note however that attempts to improve decision making typically address the process aiming to influence the decision determination and so being able to define and evaluate a good process will help inform whether we can examine relationships between these two phases. We intend in the next phase of this work to undertake two related tasks: to delineate a measurement for the deliberation phase and to examine existing approaches to measuring the decision determination phase, and their fit to a measure of deliberation.