Predictors of patient’s intentions to participate in pragmatic clinical trials: An initial exploration

Abstract

The Veterans Health Administration is implementing a pragmatic trial research program, called Point of Care Research (POC-R). The purpose of this telephone survey in which respondents were randomized to different framing conditions of the purpose of POC-R was to determine the impact of differing frames of the purpose of POC-R on attitudes towards the program and intentions to participate; and the relative importance of different beliefs and attitudes in discriminating low vs. high intenders to participate in POC-R. The survey addressed veterans’ perceptions and attitudes towards POC-R, and their willingness to participate in a pragmatic trial. Overall, respondents felt positively towards POC-R and intended to participate. Differing frames of the purpose of POC-R were not associated with either attitudes (towards the program) or intentions to participate. However, specific beliefs and attitudes toward POC-R program were predictive of intentions to participate.

Introduction

The Institute of Medicine (IOM) defines a Learning Health System as “one in which progress in science, informatics, and care culture align to generate new knowledge as an ongoing, natural by-product of the care experience, and seamlessly refine and deliver best practices for continuous improvement in health and health care.” (IOM, 2012). Pragmatic randomized clinical trials have been proposed as one mechanism to support a Learning Health System and improve the evidence base of clinical practice.

Pragmatic clinical trials are research studies that are conducted during the process of care under situations of clinical equipoise (in which the evidence regarding the risk/benefits of competing treatments is approximately equal) (Elwyn et al., 2000, Little et al., 2001) The goal of pragmatic trials is to ameliorate limitations to the generalizability of research findings by using: (1) typical clinical settings, (2) clinical populations that are representative of the targeted population (as opposed to the highly selected populations commonly enrolled in clinical trials), and (3) clinicians who practice in the situations where the intervention would be implemented (Chalkidou, Tunis, Whicher, Fowler, & Zwarenstein, 2012; Thorpe et al., 2009; Zwarenstein et al., 2008).

The Veterans Healthcare Administration (VA) is considering a new research program, called Point of Care Research (POC-R), which is based on the concept of pragmatic trials. The POC-R program would support research conducted during the process of care; randomization would be a part of regular clinical care decisions. To the degree that the treatment arms of a study are judged to have equipoise, specific POC-R trials may not require consent or additional oversight.

A pragmatic trial program highlights the tension between two perspectives of clinical research. In the traditional view, all clinical research puts patients at risk and therefore they must be protected through informed consent. In the pragmatic triallist’s view, the fact that both treatment arms represent accepted practice suggests that no differential harm is expected. In this latter view, it is thought to be both practical and ethical to allow lower levels of oversight and monitoring and to allow modification to the usual informed consent process (Vickers, & Scardino, 2009). This difference reflects the purpose of these different forms of research: traditional research is intended to develop new knowledge, while pragmatic trials are intended to compare efficacious interventions to identify the one that is most efficient and effective across a range of outcomes.

The frame that is used to explain the purpose of pragmatic trials to potential participants may be an important predictor of individuals’ willingness to participate, however this question has not yet been addressed. In this study, the question we sought to address is not whether one particular explanation of the purpose of point of care research would be better or worse than others but the more general question of whether framing effects would impact people׳s attitudes and intentions towards POC research. In summary of the goals of this study were to determine: (1) whether differing frames regarding the purpose of POC-R were associated with attitudes towards the program and intentions to participate; and (2) the relative importance of different beliefs and attitudes in discriminating low vs. high intenders to participate in POC-R. Finally, since consent models are an important implementation issue in POC-R, our third exploratory aim was to assess the relationship between individual’s beliefs about POC-R and their willingness to engage in yearly consent for the program.

Prior work

As part of an internal evaluation of the POC-R program, a series of focus groups were conducted across 7 VA medical centers with 48 patients (Weir, Butler, Barrus, & Lewis, 2013). Qualitative analysis of the transcripts from these focus groups found 6 different thematic areas: (1) concern over the potential burden of participating, (2) concern over the impact of the program on the provider–patient relationship, (3) the value of the research to the VA; (4) belief that it would improve care, (5) belief that, as veterans, they had a personal responsibility or duty to participate in research that might help other veterans, and (6) concern regarding differing models of consent. These themes were used for item construction in the development the survey.

In these focus groups, we found that nearly all patients had difficulty understanding the purpose of POC-R. We explored different explanations and purposes with the focus group participants in order to maximize their understanding of the program. Explanations that emphasized how POC-R might improve the quality of care, reduce costs and improve efficiency, and/or lead to improved scientific knowledge were all tried with these groups. However, we were not sure how these different explanations might impact attitudes and intentions to participate. Given that prior research has demonstrated that small differences in how programs or interventions are described may lead to significant impacts on perceptions of value, and on behavioral choice (Kühberger, 1998), we felt it was necessary to directly test these differing explanations of the purpose behind POC-R on attitudes or intentions to participate.

Methods

Design

The study was a randomized experiment embedded in a telephone survey. Participants were randomized to one of three frames describing the purpose of POC-R:

• 1)Improving clinical quality (“The goal of POC-R studies is to improve the quality of care by determining which options for care are safe, appropriate and meet performance standards”);
• 2)Reduce costs (“The goal of POC-R studies is to improve the efficiency of healthcare delivery by determining which options for care are the least costly and the most efficient. In POC-R studies, the treatment options being compared are both effective, but vary in terms of the ease and feasibility of implementation”);
• 3)Improve science (“The goal of POC-R studies is to enhance scientific knowledge by comparing treatments using large numbers of patients across diverse geographical areas and in a variety of natural care settings”).

Construction of the survey

The survey items were based on the themes identified in the focus groups and were constructed using an adapted Likert format, scored on a 1–7 scale. Survey items were created to assess individuals’ general attitude toward POC-R as well as to reflect the specific attitudes expressed in the focus groups. Finally, items to capture intentions to participate were created. Piloting of sample items was conducted with 5 veterans. The telephone survey used in this study is listed in Appendix 1.

Participants/setting

The study was conducted at HOSPITAL, this center includes a 113-bed hospital and 5 Community based outpatient clinics. The study was given IRB approval by both the University and VA Boards.

A sample of 496 English-speaking veterans without dementia who had been seen in a primary care clinic in the previous 3 years were selected at random from the VA’s clinical data warehouse and contacted by phone. Of this initial sample 333 did not respond to phone messages left asking them to call back regarding a survey, and 13 refused participation. 150 individuals agreed to participate and 141 provided complete answers to all survey questions.

Creation of outcome scales

We created and tested two scales from the raw survey data to measure our two outcomes of interest: Attitudes towards POC-R and Intentions to participate in POC-R. Finally, since consent models are an important aspect of pragmatic trials, we created a Consent scale to reflect participants’ willingness to engage in blanket consent.

The Attitude scale combined individual’s responses to 3 questions: (1) “Should POC-R be implemented in the VA” (question 7); (2) “You think POC-R is – not important/very important” (question 9) and (3) “You believe that the POC-R research program will improve the quality of care in the VA” – not at all/a great deal (question 10).

The Intention scale combined individual’s responses to 3 questions. The first question asked the likelihood they would agree to participate (question 4). The second question asked for the probability they would participate (question 8); this response was originally scored as 0–100% and was normalized to 0–7 prior to calculation of this scale. The third question asked for their willingness to participate in POC-R (question 11).

The Consent scale was the mean of the individuals response to two questions: “Based on what you know today, would you be willing to give a blanket consent, (covers all studies) yearly to be part of any local ongoing POC-R studies?” – very unlikely/very likely (Question 12), and “If you could consent only once for all ongoing POC-R studies in your VA for a year, you would be?” – Not at all willing/willing, (question 18).

Analysis

Several analyses were conducted, using R statistical computing software for all analyses (The R Project for Statistical Computing, 2012). First, we tested the internal reliability (Cronbach’s alpha) of the scales for attitudes toward the program, intention to participate in POC-R in general, and willingness to participate in blanket consent described above.

Second, to determine whether the differing frames of the purpose of POC-R affected attitudes towards the program or veterans’ intentions to participate, we created two linear models for each of the attitudes and intentions scales. In each model we regressed the scale on framing conditions (modeled as a factor) while correcting for baseline demographic differences between groups.

Third, to determine which beliefs would best discriminate individuals who were high vs. low intenders to participate in POC-R, we used a cross-validated LASS0 procedure on a logistic model. In this analysis we first created a median split on the intention scale, this was our dependent variable. We then ran a leave one out cross validation of a Least Absolute Selection Shrinkage Operator (LASSO) on the logistic model to select those beliefs that discriminated between high vs. low intenders. The LASSO uses a shrinkage parameter, lambda, to constrain the values of potential covariates in a model such that some coefficients are constrained to be zero and are therefore left out of the model. Model accuracy was assessed by using the cross validated predictions from the model in which lambda was within one standard error of the value of lambda which gave the least squared error. To calculate the relative importance of beliefs selected, we created a second model that included only covariates selected by the LASSO and calculated the 95% confidence interval for the odds ratios for each belief. Finally we plotted the relationship between individuals’ beliefs (using only those beliefs significant in the final model) and their intention class.

To inform future work we calculated the correlation between the consent scale and the other beliefs measured. These results are included in Appendix 2.

Results

Description of participants

Table 1 presents the age, number of self-reported clinic visits to the VA and number of self-reported VA hospitalizations for participants in each condition, and the counts and proportion of individuals in each condition who had either personally consented to VA research in the past or knew someone who had. Despite randomization to condition, Individuals who were randomized to the “improving the Quality of care” frame were significantly older than individuals in the other two conditions. There were no other significant differences between groups in these demographic measures.

Table 1.

Demographics of survey respondents.

	Science (n=47)	Cost (n=49)	Quality (n=45)	Test statistic	P-valuea
Age	62.0 (16.5)	60.9 (16.2)	67.9 (13.9)	4.6a	0.03
# Clinic visits in the VA in past year	15.4 (27.0)	9.6 (12.0)	18.4 (59.8)	1.3a	0.26
# Hospitalizations in the VA in past year	1.6 (7.9)	0.53 (1.0)	0.75 (3.0)	1.1a	0.29
Consented to VA research before?	15 (31.9)	10 (20.4)	8 (17.8)	2.9b	0.23
Know Others who have consented?	11 (23.4)	19 (38.7)	14 (31.1)	2.6b	0.27

^aF statistic.

^bChi-squared.

Internal reliability of scales

The internal reliability of the Attitude, Intention and Consent scales were excellent, with Cronbach’s alphas of 0.77 (95% CI 0.63–0.92), 0.89 (95% CI 0.77–1.01) and 0.88 (95% CI 0.69–1.08) respectively.

Impact of framing

On average, respondents expressed positive attitudes toward the POC-R program and moderate intentions to participate: mean Attitudes scale=5.4, SD=1.44, range: 1.0–7.0 and mean Intentions scale=5.0, SD=1.94, range: 0.66–7.0.

After accounting for baseline difference in age between groups, differing explanations of the purpose of POC-R were not associated with Attitudes towards the program. Table 2 presents the results of the two linear models testing this hypothesis.

Table 2.

Effect of framing on attitudes and intentions towards POC-R.

Outcome	Covariates	Coefficient	Std. error	T-value	P-Value
Attitudes	Cost frame	0.053	0.29	0.18	0.86
	Quality frame	−0.001	0.31	−0.006	0.99
	Age	0.009	0.007	1.1	0.27
Intentions	Cost frame	0.25	0.40	0.6	0.53
	Quality frame	0.53	0.41	1.3	0.20
	Age	−0.0004	0.01	−0.03	0.97

Logistic regression analysis

Table 3 presents the results of the Least Absolute Square Selection Operator (LASSO) of the logistic regression. Beliefs that characterized individuals who are high intenders to participate in POC-R were: (1) the belief that clinical research is valuable, (2) willingness to engage in either blanket or yearly consent for the program, (3) the belief that POC-R will improve the quality of care, and (4) a belief that participating in a research is a duty. The cross-validated model accurately classified 108/141 participants, an overall accuracy of 76.5%.

Table 3.

Results of least absolute squares selection operator analysis of beliefs predictive of low vs. high intentions to participate in POC-R.

Belief	Odds ratio
I am willing to engage in a blanket consent for POC-R	1.27
POC-R will improve the quality of care	1.24
Clinical research is valuable	1.14
I would be willing to consent once a year to participate in POC-R	1.08
Participating in a research is a duty	1.02
It will take effort to participate in POCR	1.00
It will take time to participate in POCR	1.00
Doctor’s control over care will change	1.00
If I participate in POC-R my relationship with my doctor will change	1.00
Quality of care will change	1.00
POC-R should be in the VA	1.00
Clinical research is important	1.00
It is important to me to be informed of research results	1.00
It is important to me to consent to every study	1.00
My relationship with my doctor is important to me	1.00
It is Important to me to be compensated for participating in research	1.00
It is important to me to keep my health record private	1.00

Table 4 presents the results of the logistic model with only the covariates from the LASSO included.

Table 4.

Logistic regression model using beliefs selected by LASSO of low vs. high intenders to participate in POC-R.

Belief	Odds ratio (95% CI)	Std. error	Z value	P-value
Clinical research is valuable	2.46 (1.18–5.95)	0.40	2.23	0.02⁎
I am willing to engage in a blanket consent for POC-R	1.39 (1.07–1.85)	0.14	2.44	0.01⁎
POC-R will improve the quality of care	1.43 (0.99–2.14)	0.19	1.85	0.06
I would be willing to consent once a year to participate in POC-R	1.16 (0.86–1.56)	0.15	0.96	0.33
Participating in a research is a duty	1.12 (0.90–1.39)	0.11	1.04	0.29

^⁎P<0.05.

Discussion

In this study we examined whether differing frames of the purpose of the VA’s Point of Care research (POC-R) program were associated with differences in attitudes toward the program and willingness to participate. In addition, we examined whether different beliefs about the program would be associated with willingness to participate. We found that framing was not associated with differences in attitudes or intentions. In contrast, we found that individuals’ beliefs about POC-R did discriminate individuals who were low vs. high in intentions to participate. More specifically, beliefs that clinical research is valuable and that POC-R would improve the quality of care were positively associated with intentions to participate. In addition, individuals who were willing to participate in a yearly or blanket consent for POC-R studies were also more likely to be high intenders to participate in POC-R, thus addressing an important implementation issue.

The results of this study are important for two reasons. First, since the goal of POC-R is to recruit large, representative samples of patient populations, knowledge of beliefs that influence patients’ willingness to participate should help with developing recruitment and strategies and materials for the program. These correlations would be likely to replicate in the non-VA population as well. Second, our finding that there was no significant differences in attitudes towards the program or intentions to participate across the three framing conditions suggest that concerns about framing effects in this context may be unwarranted.

Our findings complement prior work that has consistently found that individual’s understanding and beliefs related to the concepts of equipoise and randomization may be determinants of engagement in research. More specifically, patients’ have been found to have significant difficulty understanding the concept of equipoise (Mills et al., 2003) and to have difficulty believing that their provider may not be sure of the best option (Robinson et al., 2004). Similarly patients’ often fail to understand the idea of randomization to treatment (Falagas, Korbila, Giannopoulou, Kondilis, & Peppas, 2009), and even when they understand the concept, they often disagree that randomization to treatment is appropriate if the provider does not know which treatment is best (Robinson et al., 2005). This study adds to this prior literature by identifying specific beliefs that may drive engagement in pragmatic clinical research.

The relationship between intention to participate in POC-R and willingness to complete a blanket consent points to the difficulties to be addressed as comparative effectiveness research programs are implemented on a large scale. The Office for Human Research Protections (OHRP) has recently proposed a draft policy to guide researchers in rules for consenting patients in comparative effective studies where both arms reflect accepted standards of care (equipoise). They argue that those risks under evaluation that would be different for some patients must be communicated to participating patients. Recently the editors of the New England Journal of Medicine argued that the OHRP draft policy be revised to reflect the fact that the very purpose of pragmatic trials is to determine if an actual differential effect exists, in situations in which no difference is currently known (Editors, November 27, 2014; OHRP, 2015) Others have argued that when comparing two accepted standards of care, describing them in terms of risks and potential harm makes the research seem more risky than it is and therefore make the consent process more complicated (Lantos & Spertus, 2014, November 27, 2014). Regardless of the outcome of the debate on the presentation of risks in consent for pragmatic research, health systems that seek to implement pragmatic trials on a systematic basis will need to determine whether to implement a blanket or periodic consent procedure. This decision may increase the complexity of communicating with patients (Falagas et al., 2009), and, as our results suggest, may impact enrollment.

Strengths

The primary strength of this study is the randomized design. In addition, the measurement instrument was grounded in prior qualitative work, thereby improving generalizability. In addition, many patients in this study had experience participating in research thus allowing them to base their responses on prior experience rather than on a purely hypothetical scenario.

Limitations

This study has limitations that should be addressed in future work. Although we piloted the items in the survey, it is possible that patients may not have understood the differing frames of the purpose of POC-R and this may have affected our results. The instrument itself was not a fully developed validated tool. However, the high levels of reliability in the created scales and the fact that the items were derived from prior qualitative work enhance the validity of measurement. In addition since this study was conducted with a veteran population it is possible that our findings may not generalize to other groups, therefore this study should be replicated with other populations. Finally, our correlational data does not provide causal proof regarding the relationship between attitudes and intentions and behavior.

Future work

Future work could focus on improving the content validity of the survey instrument, In addition, more work is needed to clarify the relationship between how patients understand randomization, equipoise, and differing consent models and their beliefs and attitude towards pragmatic research. Finally it is likely that individuals’ beliefs may change as awareness of pragmatic research increases so that the focus of patient’s concerns may be different in the future.

Conclusions

In conclusion, patients appeared to have a generally positive attitude toward pragmatic trials, with those believing that participating in research is a duty and that research can improve care being more likely to participate. No evidence was found that differences in framing of the purposes of the research made a difference in attitudes or in willingness to participate.

Conflict of interest

Bryan Gibson, none declared, Jorie Butler, none declared, Lacey Lewis, none declared, Charlene Weir, none declared.

The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs or the United States government.

Appendix A. Supplementary material

Supplementary material