Validity of a Low Literacy Version of the Decisional Conflict Scale

Abstract

Objective

To evaluate the psychometric properties of the 4-factor low literacy Decisional Conflict Scale (DCS-LL) with men eligible for prostate cancer screening (PCS).

Methods

We used baseline (T0; n = 149) and post-intervention (T2; n = 89) data from a randomized, controlled trial of a PCS decision aid to assess internal consistency reliability and construct, discriminant, and factor validity.

Results

There was evidence of excellent internal consistency reliability (α’s ≥ .80) and fair construct validity (most r’s ≥ .40) for the DCS-LL except for the Supported subscale. The DCS-LL was able to discriminate between men who had decided and those who had not. There was evidence for the original 4-factor model at T0 but exploratory analysis suggested a 3-factor solution at T0 and T2 with Informed and Value Clarity as one factor.

Conclusion

For men eligible for PCS, feeling informed and feeling clear about values may not reflect distinct cognitive processes. Feeling supported may not be a factor contributing to uncertainty. Research should address whether current DCS subscales best represent the factors that contribute to uncertainty for PCS and for other screening decisions. Research should also explore the influence of health literacy on the factor structure of the DCS-LL.

1. Introduction

Patient decision aids reduce decisional conflict, the perceived uncertainty about which course of action should be taken, by providing information about options and by clarifying personal values [1–5]. For prostate cancer screening (PCS), there is uncertainty about the benefits of early screening in reducing prostate cancer mortality [6] and the decision requires trade-off of the potential physical and psychological harm from treatment options [6–10]. A recent systematic review of PCS decision aids studies indicated that these aids helped resolve decisional conflict [11].

The original, 16-item version of the Decisional Conflict Scale (DCS) is widely used for decision aid evaluations [12–14]. O’Connor and colleagues offer an alternative, 10-item version of the DCS, appropriate for individuals with low literacy skills (DCS-LL). While the DCS uses item statements and a 5-point agreement scale, the DCS-LL expresses items as questions with ordered category responses (yes, no, unsure) [1]. Additionally, the DCS-LL does not include the Effective Decision subscale and has 1 item less for the Uncertainty and Values Clarity subscales. Published reports of the use and psychometric properties of the DCS-LL are lacking [1]. The overall purpose of this analysis was to provide evidence for the reliability and validity of the DCS-LL before and after the use of a PCS decision aid.

2. Method

2.1. DCS-LL Adaptation, Sample, and Procedures

The items of the DCS-LL were adapted for PCS, as recommended by O’Connor [1] and has been done in other studies [15, 16]. Cognitive interviews with ten men were conducted to ensure comprehension of the items. Following the DCS user manual [1], we calculated total and subscale scores by summing the number of items, dividing by the number of items, and multiplying by 25 (range, 0 – no decisional conflict to 100 – extremely high decisional conflict).

The sample came from a randomized, controlled trial that evaluated a computerized PCS decision aid and an audio information booklet in two different patient populations, one a general medicine clinic in a publicly-funded hospital [17]. Men at the general medicine clinic who were found eligible for PCS completed the DCS-LL before viewing the aid (T0, n = 149) and two weeks after their visit (T2, n = 89). The average age for the sample was 54 years old (see Table 1 for other characteristics). This study was approved by the University of Texas, School of Public Health Research Service Center.

Table 1.

Sample characteristics (n = 149)

Characteristic	% (n)
Education
Did not graduate from high school	28.2 (42)
High school graduate/GEDa	43.0 (64)
Some college/vocational training	20.1 (30)
College graduate	4.7 (7)
Postgraduate degree	3.4 (5)
No response	.7 (1)
Ethnicity/Race
Black or African American	73.6 (109)
White	16.2 (25)
Mexican-American, Hispanic/Latino	8.8 (13)
Other	1.4 (2)
Health Status
Excellent	6.1 (9)
Very Good	14.1 (21)
Good	27.5 (41)
Fair	34.2 (51)
Poor	17.0 (26)
No response	.7 (1)
Insurance Status
No Insurance	67.8 (101)
Medicaid only	10.7 (16)
Medicare only	13.4 (20)
Medicaid + Medicare	0 (0)
Private	8.1 (12)
No response	0 (0)
Family History of Prostate Cancer
Yes	10.7 (16)
No	79.7 (119)
I’m not sure	9.5 (14)
Previous Prostate-Specific Antigen Test
Yes	30.9 (46)
No	51.7 (77)
Not sure	17.4 (26)
Missing	0 (0)

^aGED = general educational development

2.3. Analysis Strategy

Internal consistency reliability was assessed by examining the intraclass correlation coefficients (ICCs) from the factor analysis. Cronbach’s α was also calculated to for the scale and subscales. We expected α values and ICCs of ≥ .70, which is acceptable for research or group comparisons [18]; the DCS user manual reports an α = .86 for the DCS-LL using 63 women considering breast cancer treatment options [1].

Construct validity was assessed using an approach used by O’Connor and others [4, 15] and hypothesizes that the three subscales that contribute to uncertainty (Informed, Values Clarity, and Supported) are strongly and positively associated with the Uncertainty subscale [Pearson correlation coefficients (r) ≥ .40, p < .01].

To determine whether the DCS-LL could differentiate between men based on their intention to undergo PCS (discriminant validity), mean scores at T0 were compared using two-way analysis of variance with an a priori contrast. The contrast was created using men’s responses to, “Given what you know about prostate cancer and PSA testing, do you plan to have a PSA test?” We hypothesized that men who had made a decision (responded “yes” or “no”) would have higher decisional conflict scores than men who were not sure about screening (responded “not sure”).

To test factor validity, confirmatory factor analyses (CFAs) were conducted to assess model fit (Figure 1). We used a mean- and variance-adjusted weighted least-squares estimator (WLSMV) in Mplus because the data were ordered categorically and the scale items were non-normally distributed [19]. Model fit was assessed by examining the χ² test of model fit, the root mean square error of approximation (RMSEA), the comparative fit index (CFI), the Tucker–Lewis Index (TLI), and the weighted root mean square residual (WRMR). A combination of a statistically non-significant χ² value (p > .05), RMSEA < (.05 – .08), CFI > .95, TLI > .95, and WRMR < .90 indicates adequate model fit [20, 21]. Additional exploratory factor analyses were conducted using WLSMV with an oblique rotation (GEOMIN) [19].

Figure 1.

Four–Factor Model of the Low Literacy Decisional Conflict Scale (DCS-LL)

3. Results

Cronbach’s α for to the total scale and subscales indicated good internal consistency reliability (α’s ≥ .80), except for the Supported subscale (Table 2). This pattern was similar for the intraclass correlation coefficients.

Table 2.

Baseline (T0) and two–week follow–up (T2) internal consistency reliability for the low literacy Decisional Conflict Scale

Subscale	Cronbach’s α	Intraclass Correlation Coefficients (ICCs)
		Informed	Values Clarity	Supported
Baseline (T0)
DCS total	.834	---	---	---
Uncertainty	.817	.748	.637	.655
Informed	.838	---	.868	.429
Values Clarity	.820	---	---	.246
Supported	.468	---	---	---
Two-week Follow-up (T2)a
DCS total	.859	---	---	---
Uncertainty	.835	---	---	---
Informed	.818	---	---	---
Values Clarity	.917	---	---	---
Supported	.596	---	---	---

^aThe ICCs from the confirmatory factor analysis are presented for at T0 only because the T2 model was not identified.

There was some evidence for good construct validity for two of three subscales that contribute to uncertainty (Table 3). The Informed subscale was strongly correlated with the Uncertainty subscale at both time points and the Values Clarity subscale was strongly correlated (T0) and weakly correlated (T2) with Uncertainty. Poor construct validity was evident for the Supported subscale (T0 and T2).

Table 3.

Means, standard deviations (SD), and Pearson correlations coefficients for the low-literacy Decisional Conflict Scale and subscales

	Means (SD)	Correlation Coefficients
		Informed	Values Clarity	Supported
Baseline (T0)
Total DCS	33.66 (25.94)
Uncertainty	24.65 (37.04)	.492*	.441*	.354*
Informed	53.60 (40.71)	---	.629*	.241*
Values Clarity	39.80 (41.35)	---	---	.122
Supported	13.51 (21.42)	---	---	---
Follow-up (T2)
Total DCS	17.41 (22.48)
Uncertainty	12.36 (26.92)	.419*	.339*	.098
Informed	24.14 (34.27)	---	.878*	.290*
Values Clarity	25.57 (37.93)	---	---	.312*
Supported	10.35 (19.56)	---	---	---

^*Pearson correlation coefficient was significant at the 0.01 level (1–tailed).

The two-way analysis of variance results suggested that the DCS-LL could discriminate between groups of men based on whether they had made a decision to undergo PCS [F(1, 423) = 12.52, p < .001]. Men who were undecided about screening had significantly higher total DCS scores than those who had made a decision.

Evidence for good factor validity was indicated by adequate model fit for the DCS-LL at T0 according to all indices except χ² (p = .022) (Table 4). However, the 4-factor model at T2 could not be identified. We then conducted exploratory analyses with a WLSMV with oblique rotation (GEOMIN) estimator to explore the differences in the underlying structure of the DCS-LL at the two time points (Table 5). We determined the number of factors using scree plots and eigenvalues close to 1. A 3-factor model was indicated as the best solution at both time points. Additionally, factor 1 loaded all Uncertainty items, and factor 2 loaded all Informed and Values Clarity items. While all Supported items loaded on factor 3 at T2, one item (enough support) did not load on any factor and one time (feeling pressure) had a high cross-loading with factor 1.

Table 4.

Fit indices^a from the confirmatory factor analysis for the low-literacy Decisional Conflict Scale

Model	χ2 (df)b	P–value	CFI	TLI	RMSEA	WRMR
Baseline (T0) 4–Factor Model	25.216(13)	.022	.990	.989	.079	.659
Follow-up (T2) 4–Factor Modelc	---	---	---	---	---	---

^aCFI = comparative fit index; TLI = Tucker–Lewis index; RMSEA = root mean square error of approximation; WRMR = weighted root mean square residual

^bχ²(df) = Chi-square(degrees of freedom)

^cAn item in the uncertainty subscale had a negative residual variance; fixing the variance to 0 made the model un–identifiable.

Table 5.

Factor loadings^a from the exploratory factor analysis of the low-literacy Decisional Conflict Scale

Original Subscale	Items	Baseline (T0)			Two week follow–up (T2)
		Factor 1	Factor 2	Factor 3	Factor 1	Factor 2	Factor 3
Uncertainty	D1: sure what to do	.693			1.080
	D2: clear what choice is best	.839			0.761
Informed	D3: aware of options		.644			0.706
	D4: know advantages		.937			0.877
	D5: know disadvantages		.974			0.922
Values	D6: advantages importance		.861			0.923
Clarity	D7: disadvantages importance		.945			0.911
	D8: without pressure			.601			0.686
Supported	D9: enough support						0.693
	D10: enough advice	.537		.402			0.926

^aOnly factor loadings ≥ .400 are presented

Discussion and conclusion

4.1. Discussion

We found strong evidence for internal consistency reliability, construct and discriminant validity, and fair evidence factor validity and invariance before and after the use of a PCS decision for the DCS-LL overall and for most subscales. The poor performance of the Support subscale in most tests suggests that feelings of being supported may not be a contributing factor to the uncertainty whether or not to undergo PCS. This may be unique to screening decisions that may lead to other decisions and consequences if abnormal results are found. These future decisions may be where support is more relevant.

Although there was evidence for a 4-factor model at T0, exploratory analyses suggested a 3-factor model. Perceptions of feeling informed about options and the values a patient associates with those options appear to be measuring the same underlying construct. The low degree of decisional conflict reported by our sample may have made it difficult to discriminate between latent DCS factors [15]. A French study of cancer patients deciding whether to undergo BRCA genetic testing found it difficult to distinguish DCS latent factors, and the authors suggested that the amount of conflict generated by BRCA genetic testing is overestimated [22]. Other authors have suggested that patients experience decisional conflict only when they perceive that they have choice [15]. For PCS, some men may not view refusing screening as a real option because they may believe that “prevention is always beneficial” [23].

The results of the Informed and Values Clarity loading as one factor may have broader implications for evaluating decision aids. The International Patient Decision Aids Standards Collaboration has identified that two of the six decision process criteria for evaluating decision aids can be measured using these two subscales.[24, 25] Our results suggest that it may not be appropriate to use the two subscales as separate criteria.

4.2. Conclusion

Our study provides support for use of the DCS-LL in PCS decision aid studies, but suggests caution about using the individual subscales as evaluative criteria. In particular, the support subscale does not perform well. This study’s conclusions are limited to the small sample size at follow-up, which limited confirmatory factor analysis at T2. Additionally, the small sample size did not allow for us to test discriminant validity of the DCS-LL or explore decision aid groups at T2. Another limitation was that individual health literacy was not assessed, and it is unknown if the low literacy version is influenced by health literacy. Future research may want to explore the influence of health literacy on the factor structure of the DCS-LL.

4.3. Practice implications

Factors regarding uncertainty about a course of action may be unique to the type of healthcare decision. Future research should explore what factors men may find conflictual for PCS and other screening decisions. Future studies using the DCS should test psychometric properties, including factor comparisons to the original subscales.