Exploring Decisional Conflict With Measures of Numeracy and Optimism in a Stated Preference Survey

Abstract

Objectives

Low optimism and low numeracy are associated with difficulty or lack of participation in making treatment-related health care decisions. We investigated whether low optimism and low self-reported numeracy scores could help uncover evidence of decisional conflict in a discrete-choice experiment (DCE).

Methods

Preferences for a treatment to delay type 1 diabetes were elicited using a DCE among 1501 parents in the United States. Respondents chose between two hypothetical treatments or they could choose no treatment (opt out) in a series of choice questions. The survey included a measure of optimism and a measure of subjective numeracy. We used latent class analyses where membership probability was predicted by optimism and numeracy scores.

Results

Respondents with lower optimism scores had a higher probability of membership in a class with disordered preferences (P value for optimism coefficient = 0.032). Those with lower self-reported numeracy scores were more likely to be in a class with a strong preference for opting out and disordered preferences (P = 0.000) or a class with a preference for opting out and avoiding serious treatment-related risks (P = 0.015).

Conclusions

If respondents with lower optimism and numeracy scores are more likely to choose to opt out or have disordered preferences in a DCE, it may indicate that they have difficulty completing choice tasks.

Introduction

Stated preference studies are increasingly used to assess patient preferences for treatment attributes in health care. ¹ These studies can examine which attributes of a treatment matter to patients, the relative importance of these attributes, and the benefit-risk tradeoffs patients are willing to accept in making treatment decisions. Discrete-choice experiments are a common method for eliciting patient preferences by offering survey participants a series of hypothetical choices between medical treatments or medicines with varying attributes. In real life, health-related treatment decisions can be confusing and stressful, and there is evidence that patients and people making decisions for others, especially parents, experience difficulty making treatment-related medical decisions.^2–8 This decisional conflict is defined as the state of uncertainty people experience when deliberating which course of action to take when making treatment-related medical decisions. ⁹

Choice of Psychological Constructs

In this study, we used two common psychological constructs to explore evidence of potential decisional conflict among parents making hypothetical treatment-related decisions in a stated preference survey. There are numerous measures designed to capture constructs known to affect health behaviors, including attitudes and perceptions about health, illness, and medicine; health literacy; and numeracy, among others. Russo and colleagues ¹⁰ systematically reviewed psychological instruments and constructs used by preference researchers and identified 18 types of constructs used in 33 preference studies or health-related decision-making studies, some of which were associated with patients’ preferences or decisions in health-related contexts.

When considering which constructs to include in the stated preference survey, we considered whether they could be administered in an online format, were free to use, the increase in overall survey length, and the decision context presented to respondents in the survey instrument. Respondents were asked to suppose that their child would develop a lifelong illness in the future requiring daily management. Then, they were asked to choose between treatment alternatives to delay disease onset. These alternatives were defined by probabilistic outcomes. Thus, we hypothesized that a respondent’s expectations for a good outcome (optimism) and their perceived quantitative abilities (numeracy) would be relevant in this context. Based on this, we selected two psychological instruments that measure constructs associated with difficulty in making health care–related decisions: the Life-Orientation Test–Revised (LOT-R) and the brief three-item Subjective Numeracy Scale (SNS).^11,12

The LOT-R provides a score of dispositional optimism. ¹¹ Dispositional optimism is an individual’s generalized tendency to expect either positive (optimistic) or negative (pessimistic) outcomes across different domains of their life.^13–15 Optimism was chosen because we hypothesized that people with higher optimism would respond to treatment choices systematically differently than respondents with lower optimism since optimism has been shown to be negatively correlated with being bothered by symptoms of disease and has been shown to be associated with decision-related distress.^4,13 The SNS provides a measure of an individual’s perceived quantitative abilities and their preference for information presented numerically. ¹⁶ Numeracy was chosen because respondents in the survey would be asked to evaluate interventions defined, in part, by attributes with probabilistic levels, and it seems reasonable to expect that survey respondents with lower numeracy would have greater difficulty completing the choice tasks.

Optimism and Decisional Conflict

Those who generally expect negative health outcomes likely experience greater uncertainty when deliberating a course of action when making treatment-related decisions. Previous research using the LOT-R has indicated that low optimism is associated with difficulty in making treatment-related decisions. Steginga and Occhipinti ⁴ examined the association between decision-related distress and dispositional optimism among patients with prostate cancer. They found that greater optimism was associated with less distress related to making a treatment decision. ¹⁷ Similarly, Orom and colleagues ³ found that lower optimism was associated with greater difficulty in the decision-making process in patients with prostate cancer.

These studies focused on patients with a cancer diagnosis in the context of understanding how the emotional toll of a cancer diagnosis affects a patient’s ability to make treatment-related decisions. In the context of a preference survey, respondents are presented with hypothetical treatment-related decisions, but they are asked to assume the hypothetical scenario is true for them and asked to decide as though it were a real choice. If a person’s lack of optimism is a sign of potential difficulty in making actual health-related decisions, it may be that they also have difficulty choosing between hypothetical treatments in stated preference surveys.

Numeracy and Decisional Conflict

Low numeracy may also be associated with a lack of desire to participate in making treatment-related decisions. de Bruin and colleagues have shown that lower numeracy in older adults may be due to a lack of motivation. ¹⁸ Goggins and colleagues ¹⁹ examined the relationship between numeracy (using the brief three-item SNS) and involvement in the decision-making process among patients hospitalized with cardiovascular disease. ²⁰ They found that patients with higher numeracy desired more participation in the problem-solving and decision-making process.

If individuals who prefer not to receive information numerically and who have lower self-perceived numerical abilities also prefer to be less involved in health-related treatment decisions, it may be that this lack of desire for participation hinders their ability or interest in participating in the decision-making scenario in a preference survey.

Decisional Conflict in a Discrete-Choice Experiment

We conducted a discrete-choice experiment (DCE) survey to investigate parents’ preferences for treatments to delay the onset of insulin dependence in children who will develop stage III type 1 diabetes (T1D). We investigated whether low optimism and low self-reported numeracy among parents were associated with evidence of decision-making difficulty in a DCE. If respondents with lower optimism and/or lower numeracy scores are experiencing decisional conflict when choosing among the treatment options in the hypothetical choice questions, we may expect to see evidence of this decisional conflict in the data in two ways. First, if a higher rate of respondents with low optimism or low numeracy, compared with those with higher optimism and numeracy scores, choose to opt out, it may be an indication of difficulty in decision making. Similarly, if respondents with low optimism or lower numeracy appear to prefer worse outcomes to better outcomes when those with higher optimism and numeracy scores do not, it may be an indication of decisional conflict. It is not choosing opt out alone, or the appearance of disordered preferences alone that indicates decisional conflict, but that they happen more frequently among those with low optimism and numeracy scores and not among those with higher optimism and numeracy that indicates respondents may have experienced decisional conflict.

Methods

Survey Design and Administration

A DCE was administered online to 1501 adult parents in the United States with children currently under the age of 18 years. The DCE aimed to elicit the parents’ preferences for treatments that delayed the onset of T1D in children who had tested positive for the autoantibodies associated with T1D (see DiSantostefano et al. ²¹ for study details). Respondents chose between two hypothetical interception treatments or an opt out (monitoring only, no treatment) in a series of experimentally designed choice questions. The hypothetical treatments in each choice question were defined by six attributes (two efficacy and four risk attributes), each with two to four levels (see Supplemental Table S1 for attributes and Supplemental Figure S1 for an example DCE question). Two of the risk attributes were presented numerically. The survey included two comprehension questions—a question designed to evaluate comprehension of risks presented graphically in an icon array with humanoid figures and a DCE question that presented a dominated treatment option (a treatment option that was worse in all respects than the other treatment option). In addition to the DCE, the survey included demographic questions.

Qualitative interviews (n = 10) and a focus group (n = 5) with parents were conducted to explore parents’ concerns and perceived benefits of the treatment and gauge parents’ understanding of the hypothetical scenario presented in the survey. The interviews and focus groups were also used to refine survey content. Prior to fielding, the survey was pretested with an additional 15 parents. The study survey was conducted in accordance with good research practices,^22,23 complied with the Declaration of Helsinki, and received an exemption from full review by an institutional review board.

Measuring Dispositional Optimism

To measure dispositional optimism, approximately half of respondents were randomly administered a slightly modified version of the LOT-R (Figure 1A). We omitted four filler statements that are not included in the LOT-R dispositional optimism score. A subset of respondents rated their agreement with the remaining six statements (see Figure 1A) on a 5-point scale (I agree a lot = 4; I agree a little = 3; I neither agree nor disagree = 2; I disagree a little = 1; I disagree a lot = 0).

Figure 1.

Optimism and numeracy questions in the survey. (A) Optimism question. The measure of dispositional optimism, which was based on the LOT-R (Life-Orientation Test–Revised). ¹¹ The instrument was slightly modified for administration. Four filler statements were omitted, and the response categories were labeled differently than in the LOT-R. “Strongly agree” and “strongly disagree” were presented as “I agree a lot” and “I disagree a lot”; “agree” and “disagree” were presented as “I agree a little” and “I disagree a little”; and “neutral” was presented as “I neither agree nor disagree.” (B) Numeracy question. The measure of subjective numeracy; the brief three-item Subjective Numeracy Scale. ¹²

The scores were summed to derive an overall dispositional optimism score for each respondent. Thus, the scores could range from 0 to 24. Three statements were positive (e.g., “In uncertain times, I usually expect the best”) and three were negative (e.g., “If something can go wrong for me, it will”). Negatively worded statements were reverse-coded before being scored. A higher score indicated greater optimism, while a lower score indicated less optimism.

Measuring Subjective Numeracy

To measure subjective numeracy, the survey included the brief three-item SNS (Figure 1B). ¹² The first two questions (“How good are you at working with fractions?” and “How good are you at figuring out how much a shirt will cost if it is 25% off?”) focus on a respondent’s self-assessment of their ability to work with numbers. The third question (“How often do you find numerical information to be useful?”) captures a respondent’s preference for numerical information.

Responses to the numeracy questions were scored on a 6-point scale and summed across the three questions; thus, the total subjective numeracy score could range from 3 to 18. A binary variable indicated that a respondent’s total numeracy score was higher than the median score for the sample.

Data Analysis

Correlation Between Optimism and Numeracy Scores and Respondent Demographics and Comprehension Questions

Summary statistics on the optimism and numeracy scores for the sample were calculated. Spearman correlation coefficients were calculated between total optimism, numeracy, and respondent demographics.

Spearman correlation coefficients were also calculated between total optimism and total numeracy scores and whether respondents correctly answered a question designed to evaluate their comprehension of risks presented graphically in an icon array with humanoid figures and whether they chose a treatment option that was worse in all respects in a dominated choice question.

Latent Class Analysis of Discrete-Choice Experiment Data

To identify potential variation in preferences for the treatment attributes by optimism and numeracy, we analyzed the DCE data using latent class analysis (LCA), which identifies distinct classes of respondents based on unobserved heterogeneity in preferences. ²⁴ Separate LCA models were run: one to explore the relationship between of optimism and preference and a second to explore the relationship between numeracy and preference. We evaluated models for two to eight classes, and the optimal number was determined by Akaike and Bayesian information criteria to be a five-class model. With LCA, respondent characteristics can be used to predict the likelihood that respondents are in one of the identified classes. In our analysis, this membership probability was predicted by a respondent’s total optimism score or total numeracy score. As some demographic and socioeconomic characteristics were expected to be correlated with the optimism and numeracy scores, these were not included in the membership probability function to avoid confounding. Four-class models were also conducted to evaluate the sensitivity of the results to the number of classes.

To investigate whether the results are robust to different scoring methods, we created additional variables for use in sensitivity analyses. For optimism, a binary variable indicated that a respondent’s total optimism score was lower than the median score of 15 for the sample. The scores for the first two numeracy questions were summed separately from the third question to create a unique numeracy ability score and a numeracy preference score for each respondent. A mean numeracy score across the three questions was also calculated for each respondent. Separate latent class models were estimated with each of these new variables one at a time.

Evidence of Decisional Conflict

We used the results of the analysis to identify evidence of decisional conflict in the survey responses. Spearman correlation coefficients (5% level) were used to investigate correlations between total optimism or total numeracy scores and the number of times a respondent chose to opt out in the DCE. A negative correlation between higher optimism or numeracy scores and frequency of choosing to opt out may be evidence of potential decisional conflict.

Decisional conflict was also identified by looking at the results of an LCA of the DCE data. The estimated preference weights for the opt-out parameter in each of the classes in the latent class models is a measure of respondents’ preference for opting out. A high preference weight for opting out among classes associated with low optimism and/or low numeracy scores may be an indication of difficulty in decision making.

In addition, disordering of preference weights for levels of an attribute in which a worse outcome appears to be preferred to a better outcome among classes associated with low optimism and/or low numeracy scores may also be an indication of difficulty in decision making. Specifically, if the parameter on the total optimism score or total numeracy score is negative and statistically significant at the 5% level for a class demonstrating a preference for opting out or with disordered preferences, it may be evidence of decisional conflict in the DCE.

Results

The study sample included 1501 parents: 600 who had a child with T1D and 901 who did not. The mean age was 40 years, 64% were female, and approximately 55% had a 4-year college degree or more (see DiSantostefano et al. ²¹ for more details on the sample).

A random portion of respondents (n = 635), both those with and without a child with T1D, were administered the optimism questions. Those with higher optimism scores were more likely to have a higher total numeracy score, be older, retired, male, and a high school graduate. The optimism score was not correlated with employment, disability, or having a college education. A total of 1501 respondents were administered the numeracy questions. Those with higher numeracy scores were more likely to be male, employed, or a college graduate and less likely to be female, unemployed, a homemaker, or disabled.

Respondents with lower optimism were more likely to choose the treatment profile that was worse in all respects than another profile in a dominant choice question. However, the total numeracy score was not correlated with choosing the worst treatment profile in a dominated choice question. Neither the optimism score nor the numeracy scores were correlated with an incorrect answer on a question designed to evaluate their comprehension of risks presented graphically (see Table 1).

Table 1.

Optimism and Numeracy Score Summary Statistics and Correlations

	Value
Total optimism score, n = 635
Mean (SD)	15.1 (4.7)
Median	15
Min, max	1, 24
Respondents whose total optimism score was less than median, n (%)	291 (46%)
Total subjective numeracy score, N = 1501
Mean (SD)	14.0 (3.5)
Median	15.0
Min, max	3, 18
Respondents whose total numeracy score was greater than median, n (%)	628 (42%)
Total subjective numeracy—ability score, N = 1501
Mean (SD)	9.2 (2.6)
Median	10
Min, max	2, 12
Total subjective numeracy—preference score, N = 1501
Mean (SD)	4.8 (1.2)
Median	5
Min, max	1, 6
Mean subjective numeracy score across 3 questions, N = 1501
Mean (SD)	4.7 (1.2)
Median	5
Min, max	1, 6
Spearman’s rank correlation coefficients with total optimism score, n = 635
Female a	−0.0861*
Age	0.0913*
High school diploma	0.0873*
4-Year college degree or more	0.0624
Graduate degree	−0.0453
Retired	0.0787*
Student	0.0228
Employed	0.0558
Unemployed	−0.0451
Disabled	−0.0284
Homemaker	−0.0659
Failed dominated choice question	−0.1070*
Number of times respondent chose to opt out (monitoring only)	−0.0114
Incorrect answer to question evaluating risk comprehension	−0.0429
Total subjective numeracy score	0.2503*
Spearman’s rank correlation coefficients with total subjective numeracy score, N = 1501
Female b	−0.2428*
Age	0.0393
High school diploma	0.0894*
4-Year college degree or more	0.2983*
Graduate degree	0.1941*
Retired	−0.0102
Student	−0.0283
Employed	0.1938*
Unemployed	−0.0964*
Disabled	−0.0544*
Homemaker	−0.1499*
Failed dominated choice question	0.0137
Number of times respondent chose to opt out (monitoring only)	−0.1213*
Incorrect answer to question evaluating risk comprehension	−0.0060

SD, standard deviation.

^aFor this statistic, n = 633, as two respondents responded “Prefer not to answer” when asked their gender.

^bFor this statistic, n = 1499, as two respondents responded “Prefer not to answer” when asked their gender.

^*Indicates statistically significantly different from zero at the 5% level (P < 0.05).

Evidence of Decisional Conflict in the Discrete-Choice Experiment Data

The total numeracy score was correlated with how often a respondent chose to opt out across the set of DCE questions they answered. Those with higher numeracy scores were less likely to choose to opt out (and more likely to choose a treatment), and those with lower numeracy scores were more likely to choose to opt out. The optimism score was not correlated with the number of times a respondent chose to opt out.

The LCA results for the model that included the total optimism score as a predictor of class membership are shown in Figure 2 (and Supplemental Figure S2). The estimated preference weights are shown for each of the five classes. The percentages shown in the legend for each class are the probability that an individual’s preferences in the sample could be characterized by that class. There was a 49% probability that an individual would be in one of two classes that displayed a strong preference for treatment over monitoring only: Class 1 (22% probability) displayed a strong preference for delaying the need for daily insulin, and Class 5 (27% probability) displayed a strong preference for reducing the chance of long-term complications. Conversely, Class 4 demonstrated a slightly higher preference for monitoring only and preferred to avoid all risks (16% probability). The remaining two classes had disordered preferences. Class 2 (11% probability) clearly preferred monitoring only (the opt out) over any treatment, and their preferences for the treatment were disordered, while Class 3 simply had disordered preferences (24% probability).

Figure 2.

Total optimism latent class preference results (n = 635). DKA, diabetic ketoacidosis.

Preference estimates from the LCA with membership probability predicted by total optimism are shown in Table 2. All results are interpreted in relation to the reference class, the class for whom reducing the chance of long-term complications was most important (Class 5). The coefficient for total optimism score was negative and statistically significant for the class with disordered preferences (Class 3). Respondents with lower optimism scores had a higher probability of membership in a class that had disordered preferences (P value for optimism coefficient, P = 0.032).

Table 2.

Preference Estimates From the Latent Class Analysis With Membership Probability Predicted by Total Optimism (n = 635) ^a

Attribute	Level	Class 1: Treatment and Short-Term Efficacy (22.3%)		Class 2: Opt Out and Disordered (10.8%)		Class 3: Disordered (23.9%)		Class 4: Opt Out and Avoiding Risk (16.4%)		Class 5: Treatment and Long-Term Efficacy (26.6%)
		Coeff.	P	Coeff.	P	Coeff.	P	Coeff.	P	Coeff.	P
Time until insulin dependence	4 additional years	1.376	0.000	0.094	0.543	−0.211	0.001	0.172	0.126	0.260	0.009
	2.5 additional years	0.209	0.013	−0.030	0.842	0.109	0.060	0.060	0.567	0.031	0.746
Reduces the chance of long-term health complications by 50%	Yes	0.354	0.000	0.076	0.478	0.007	0.877	0.123	0.101	1.066	0.000
Chance of hospitalization due to DKA when become insulin dependent	None	0.435	0.001	0.321	0.057	−0.108	0.152	0.812	0.000	0.318	0.009
	1%	0.297	0.008	0.315	0.091	0.005	0.947	0.572	0.000	0.198	0.079
	4%	−0.268	0.031	−0.291	0.162	0.053	0.468	−0.474	0.001	−0.070	0.536
Chance of serious infection from the treatment	2%	0.137	0.130	−0.408	0.021	0.038	0.505	0.486	0.000	0.049	0.577
	6%	−0.492	0.000	−0.083	0.636	0.029	0.653	−1.772	0.000	−0.432	0.000
Skin reaction from the treatment for several days each month	Yes	−0.114	0.099	−0.544	0.000	0.089	0.018	−0.212	0.003	−0.092	0.149
3 days of nausea a month for first 3 months	Mild	0.195	0.038	−0.292	0.101	0.013	0.830	−0.186	0.097	0.049	0.630
	Moderate	−0.379	0.000	−0.394	0.029	0.072	0.223	−0.514	0.000	−0.228	0.036
Monitoring only (opt out)		−3.953	0.000	0.695	0.042	−0.729	0.000	−0.909	0.000	−2.093	0.000
Total optimism score		0.006	0.859	−0.027	0.450	−0.063	0.032	0.025	0.453	Ref
Constant		−0.281	0.635	−0.512	0.359	0.806	0.088	−0.881	0.128
Log-likelihood	−4049.8324
BIC	8679.900
K	68

BIC, Bayesian information criterion; coeff., coefficient; DKA, diabetic ketoacidosis; K, number of parameters; Ref, reference class.

^aBold indicates a statistically significant parameter in the membership probability model.

Figure 3 (and Supplemental Figure S3) shows the preference weights for the LCA model that included the total numeracy score as a predictor of class membership. The characterization of the five classes is the same: two classes preferred treatment, one risk-averse class preferred monitoring only (the opt out), one preferred opting out with some disordering, and one had disordered preferences.

Figure 3.

Total numeracy latent class preference results (N = 1501). DKA, diabetic ketoacidosis.

Preference estimates from the LCA with membership probability predicted by total numeracy are shown in Table 3. The coefficient for total numeracy was negative and statistically significant in the membership model for the class that preferred opting out with disordering (P = 0.000; Class 2) and the class that preferred opting out and avoiding serious treatment-related risks (P = 0.015; Class 4). Respondents with lower self-reported numeracy scores were more likely to be in a class with a strong preference for opting out and with disordering among treatment attributes or a class with a preference for opting out and avoiding serious treatment-related risks.

Table 3.

Preference Estimates From the Latent Class Analysis With Membership Probability Predicted by Total Numeracy (N = 1501) ^a

Attribute	Level	Class 1: Treatment and Short-Term Efficacy (26.7%)		Class 2: Opt Out and Disordered (8.2%)		Class 3: Disordered (19.6%)		Class 4: Opt Out and Avoiding Risk (21.5%)		Class 5: Treatment and Long-Term Efficacy (24.0%)
		Coeff.	P	Coeff.	P	Coeff.	P	Coeff.	P	Coeff.	P
Time until insulin dependence	4 additional years	1.257	0.000	−0.110	0.466	−0.155	0.001	0.157	0.008	0.277	0.000
	2.5 additional years	0.183	0.000	−0.005	0.972	0.082	0.049	0.100	0.059	0.122	0.065
Reduces the chance of long-term health complications by 50%	Yes	0.301	0.000	0.016	0.864	0.036	0.240	0.184	0.000	1.223	0.000
Chance of hospitalization due to DKA when become insulin dependent	None	0.360	0.000	0.250	0.140	−0.011	0.849	0.504	0.000	0.340	0.001
	1%	0.296	0.000	0.412	0.012	−0.040	0.463	0.519	0.000	0.244	0.006
	4%	−0.276	0.000	−0.236	0.205	0.070	0.203	−0.309	0.000	−0.120	0.182
Chance of serious infection from the treatment	2%	0.167	0.001	−0.290	0.044	0.041	0.329	0.221	0.000	0.048	0.468
	6%	−0.507	0.000	−0.166	0.288	0.006	0.895	−1.270	0.000	−0.464	0.000
Skin reaction from the treatment for several days each month	Yes	−0.160	0.000	−0.461	0.000	0.065	0.022	−0.316	0.000	−0.052	0.253
3 days of nausea a month for first 3 months	Mild	0.124	0.013	−0.393	0.021	0.043	0.309	0.044	0.427	0.018	0.792
	Moderate	−0.435	0.000	−0.248	0.101	0.106	0.020	−0.608	0.000	−0.262	0.001
Monitoring only (opt out)		−4.099	0.000	1.199	0.000	−0.846	0.000	−0.734	0.000	−2.798	0.000
Total numeracy score		−0.004	0.880	−0.131	0.000	−0.016	0.575	−0.063	0.015	ref
Constant		0.172	0.678	0.692	0.113	0.029	0.946	0.775	0.044
Log-likelihood	−9253.8737
BIC	19146.488
K	68

BIC, Bayesian information criterion; coeff., coefficient; DKA, diabetic ketoacidosis; K, number of parameters; Ref, reference class.

^aBold indicates a statistically significant parameter in the membership probability model.

The results of the LCA were robust to changes in the optimism variable construction. When we replaced the total optimism score in the membership probability model with a binary variable indicating that an individual’s score was below the median score for the sample, the coefficient was positive and statistically significant for the class with disordered preferences (P = 0.0000), indicating that less optimistic individuals were more likely to be in the disordered class.

The results of the LCA were similar in four-class models (see Supplemental Tables S2 and S3 and Supplemental Figures S4 through S7). Respondents with lower optimism scores had a higher probability of membership in a class that had disordered preferences (P = 0.044). Respondents with lower self-reported numeracy scores were more likely to be in a class with a strong preference for opting out and avoiding serious treatment-related risks (P = 0.000). In the five-class model, the coefficient for total numeracy was also negative and statistically significant in the membership model for the class that preferred opting out with disordering, but this class is not present in the four-class model.

The results of the LCA were also robust to changes in the numeracy variable construction. The total numeracy preference score was negative and statistically significant in the membership model for the class that preferred opting out with disordering (P = 0.000), but it was not so for the class that preferred opting out and avoiding risks. All other models with variables capturing numeracy were negative and statistically significant for both the opt-out class with disordering and the class preferring opting out and avoiding risks. The total numeracy ability score was negative and statistically significant in the membership model for the class that preferred opting out with disordering (P = 0.000) and for the class that preferred opting out and avoiding serious treatment-related risks (P = 0.013). Similarly, the mean score across the three SNS questions was also negative and statistically significant for the two classes (P = 0.000, P = 0.015), as was the binary variable indicating an individual’s score was greater than the median for the sample (P = 0.029, P = 0.006).

Discussion

We used a DCE survey on parents’ perspectives on the benefit-risk tradeoffs for treating versus not treating an early stage of T1D to investigate whether decisional conflict could be uncovered with the use of psychological measures. Two psychological measures helped identify systematic differences in parents’ stated preferences for an interception treatment for T1D. These differences may indicate difficulty with decision-making in a survey presenting hypothetical treatment choices.

Optimism and Decisional Conflict

A latent class model indicated that a lower optimism score among parents was associated with a higher probability of membership in a class that had disordered preference estimates (i.e., apparent preference for worse outcomes over better outcomes). Disordered preferences may occur if respondents are answering randomly or are unengaged and not giving their full attention to the choice questions. Disordered preferences may also occur because respondents are considering only a subset of the attributes or only considering one of the attributes. Dominance on a single or a few attributes could be a rational choice and indicate that not all attributes matter. However, it could also be that respondents are simplifying the decision by focusing on only a few attributes, which could be an indicator of decisional conflict. In our study, in addition to disordered preferences, those with lower optimism were also more likely to choose the treatment profile that was worse in all respects in a dominated choice question. The results revealed that lower optimism was associated with choosing worse outcomes, potentially indicating a lack of understanding or difficulty completing the choice tasks among less optimistic individuals in the sample.

Numeracy and Decisional Conflict

A latent class model indicated that lower numeracy among parents was associated with a higher probability of membership in a class with a strong preference for opting out with disordering and a class with a preference for opting out and avoiding serious treatment-related risks. If respondents with lower numeracy are more likely to choose to opt out or have disordered preferences in a DCE, it could be an indication that they have difficulty completing DCE choice tasks. If respondents with lower numeracy are more likely to be in a class of risk avoiders when risks were presented numerically, it could be an indication of difficulty understanding information presented numerically.

Choosing the opt out in a DCE can be a rational choice. A respondent may feel that the benefit of treatment offered may not be of sufficient magnitude to offset the risks presented, may prefer to avoid all risks, or may have a genuine preference for the status quo. Although there are many plausible reasons why any one respondent might choose to opt out or demonstrate a reluctance to accept some risks, it is less clear why a lower numeracy score would be a defining characteristic of those who prefer to opt out, unless a reluctance or inability to choose is an indication of decisional struggle or lack of desire to participate in decision making. In real life, those who experience decisional conflict due to the complexity of the decision context or due to challenges in understanding probabilities may seek and rely heavily on expert opinion. If so, then opting out among these individuals could be a signal for seeking expert guidance in decision making rather than a true preference for avoiding treatment. It is also possible that opting out is an indication of lack of motivation for engaging in the type of decision skills needed to evaluate the tradeoffs presented in a DCE of moderate complexity.

Including comprehension questions and other validation-type questions, as we did in this study, are common methods for attempts at quantifying respondent comprehension and understanding and assessing internal validity after data have been collected. ²⁵ In this study, lower optimism was correlated with choosing the worst option in a dominated DCE choice question, but the optimism and, interestingly, numeracy scores were not correlated with performance on the risk comprehension question. However, all respondents—those who answered the risk grid comprehension questions correctly and those who did so incorrectly—were presented with the correct answer to the risk grid comprehension question, and those who initially gave an incorrect answer to this particular comprehension question may not necessarily have had trouble choosing between treatments in the DCE, as they may have learned how to interpret the risk grids correctly after being presented with the correct answer to the comprehension question.

Sensitivity Analyses

Reducing the number of levels to four in the latent class analysis yielded largely the same results. Those with lower optimism scores were more likely to display disordered preferences, and those with lower numeracy scores were more likely to prefer opting out and avoiding serious treatment-related risks.

The results were also robust to different ways of translating optimism and numeracy scores into variables that can be used in the analysis of the DCE data to characterize preferences. In separate latent class models, a variable capturing a respondent’s total optimism score and a variable indicating that an individual’s optimism score was below the sample median were both statistically significant and revealed that less optimistic individuals in the sample had a higher likelihood of having disordered preferences. Similarly, four additional latent class models, including a respondent’s numeracy score, confirmed an association between lower numeracy and choosing opting out with disordering. In addition, three of the four additional models confirmed an association between lower numeracy and being in a class that prefers opting out and avoiding serious treatment-related risks.

Study Limitations and Areas for Future Research

Some respondents in any stated preference survey will have difficulty completing choice tasks. This study provides insight into why some respondents may have more difficulty than others completing choice tasks and reveals two measurable psychological characteristics that could help researchers identify those in the sample who may have more difficulty than others. Although it is not reasonable nor advisable to simply identify these individuals after data collection and remove them from the analysis, knowing the proportion of the sample composed of these individuals could provide context for evaluating and understanding preference results. However, an important implication of this study is the need for surveys that are designed to improve respondent engagement and understanding and reduce decisional conflict where possible, especially among those with psychological characteristics known to be associated with difficulty in health-related decision making.

Although best practices and guidelines for conducting preference studies exist,^22,26 preference researchers continue to investigate ways to improve and measure respondent engagement and understanding, acknowledging room for improvement in how preference studies are conducted and our understanding of how well these instruments capture preferences. In a 2017 survey of authors publishing DCEs in health, questions about respondent understanding and difficulty completing choice tasks were the most common type of debriefing questions authors reported, including after a DCE. ²⁷ We did not include debriefing questions in our survey, but future research could test whether there is a correlation between acknowledging difficulty completing choice tasks in a debriefing question, having a lower optimism or numeracy score, and responses to DCE questions. Debriefing questions might also help investigate whether respondents with lower numeracy chose to opt-out because the choice questions were too difficult, because they needed more information, because they lacked motivation to engage in the decision-making process, because they would in real life seek an expert opinion, or because they genuinely preferred the opt-out.

Others have investigated whether utilizing videos or interactive survey elements may improve respondent understanding or engagement or whether the presentation format of the choice question (i.e., graphic, text, color coding) matters for respondent comprehension.^28–32 Future studies might investigate whether it is possible to simplify the decision-making context for those who are less optimistic or with lower self-perceived numerical ability with different interactive survey elements or different presentation formats.

This study sample was a general population sample of parents in the United States. A parent may experience greater decisional conflict when making health-related treatment choices affecting their child’s welfare, than when faced with similar choices for themselves. Therefore, it is possible that the degree of decisional conflict seen in our results is greater than if we had elicited preferences for a treatment for the parents themselves. Other studies in the future might investigate whether similar results might be found in patient populations making treatment-related decisions for themselves, in different disease areas, with smaller samples, or in other locations. It may be that other measures of optimism or objective numeracy are also associated with decisional conflict. There may also be other measurable psychological characteristics associated with decisional conflict not addressed in this study beyond dispositional optimism or self-perceived numeracy. The hypothetical scenario presented to parents in the survey was also quite complicated, with a hypothetical future diagnosis of a chronic disease and a hypothetical treatment that would only delay disease onset. They were asked to assume that their child had been tested for T1D antibodies and that, based on the test results, their child would develop T1D in the future. Parents were then asked to choose in a series of DCE questions between hypothetical treatments that would delay their child’s need for daily insulin or they could opt out. Additional research is needed to know if the results of this study generalize to preferences elicited under different hypothetical scenarios.