Therapeutic Misconception, Misestimation and Optimism in Subjects Enrolled in Phase I Trials

Abstract

Background

Ethical concerns about Phase I trials persist. Important conceptual advances have been made in understanding concepts used to describe misunderstanding. However, a systematic empirical evaluation of the frequency of misunderstanding incorporating recent developments is lacking.

Methods

We queried 95 Phase I subjects to provide a more sophisticated estimate of the proportion who had therapeutic misconception (TM) - misunderstands the research purpose or how research differs from individualized care - and therapeutic misestimation (TMis) -misestimates the chance of research trial benefit as >20% or underestimates risk as 0%.

Results

65/95 (68.4%) respondents had TM, associated in a multivariate analysis with lower education and family income (p= 0.008, p = 0.001), but TM was not associated with the vulnerability of having hardly any treatment options. 89/95 (94%) had TMis, though only 18% reported this was a factual estimate. Although risks of investigational agents and those exacerbated by research, such as uncertain outcomes, were mentioned (39%, 41% respondents respectively), risks novel to research, such as research biopsies, were rarely mentioned (3%). Although most of these respondents thought their chance of benefit was higher and risk lower than the population chance (optimists) (54.6%), a substantial minority (37.6%) were pessimists.

Conclusions

TM continues to be prevalent. Estimates of personal benefit were not usually meant to report facts so it is unknown whether respondents had TMis. Although not more vulnerable, Phase I participants need improved understanding of key TM concepts, with attention to risks not found in standard of care.

Introduction

Ethical concerns about Phase I trials have been discussed for over 15 years ^1–3 with considerable attention in the last seven^{4, 5}. The crux of the problem is that reports show that most patients misunderstand the purpose of a Phase I trial ^{2, 6, 7} and join for personal benefit ^8–11 when, at best, the chance of benefiting from a Phase I trial is uncertain ¹² and likely small. ¹³ This disconnect between chance of benefit and expectation of benefit has led commentators to worry that Phase I participants do not understand Phase I research. Two notions have been used to capture this misunderstanding: therapeutic misconception - misconstruing research as personal medical care –and therapeutic misestimation (TMis) - misestimating the chance of benefit or risk. TM and TMis have undergone careful conceptual development in the last decade, with reasons offered why they may not be as prevalent or as easily measured as first assumed ^{5, 12, 14–18}. In addition, a third category, therapeutic optimism (TO), which was originally intended to show an ethically acceptable reason why Phase I participants may overestimate personal benefit, has recently been criticized as ethically problematic¹⁹. A systematic empirical evaluation of the frequency of misunderstanding, which takes into account these recent developments, is lacking. Our objective was to document the frequency of TM and TMis with more sophisticated measurement based on the new conceptual insights and to look for associations of TM and TMis with TO.

The concept of therapeutic misconception has evolved since it was coined by Appelbaum et al in 1982 ^{20, 21}. A consensus panel of experts suggests the following definition: “Therapeutic misconception exists when individuals do not understand that the defining purpose of clinical research is to produce generalizable knowledge, regardless of whether the subjects enrolled in the trial may potentially benefit from the intervention under study or from other aspects of the clinical trial.²²” Two core concepts found in this and other definitions offered are: (1) subjects need to understand that the main purpose of the research protocol is to produce generalizable knowledge; and (2) conducting a research protocol differs from providing individualized care^{14, 15, 22, 23}.

In order to more clearly describe respondents’ views on risks and benefits of Phase I trials, we measured TMis separately from TM^{17, 22}, taking into account two important caveats about measuring estimates of clinical trials’ risks and benefits. First, there are two types of statements about uncertainties, frequency statements reporting the probability of risks or benefits in the general population (20 out of 100 patients will experience tumor shrinkage) and belief statements about one’s own probability (I am 80% certain that my tumor will shrink)^{18, 24}.

Second, belief statements can serve purposes other than reporting facts; they can also be used to express a positive attitude or demonstrate hope (I know it is important to have a positive attitude so I am telling myself confidently that my tumor will shrink)¹⁸. In sum, an analysis of estimates of risk and benefit must differentiate frequency from belief statements and determine the intention of the belief statement.

Therapeutic optimism(TO), being optimistic about the success of the a treatment, was introduced as an ethically unproblematic explanation for why patients may overestimate their personal chance of medical benefit¹⁷: patients are optimistic about their prospects and this reduces anxiety and gives them hope^{25, 26}. Recently, concerns have been expressed that TO can be unrealistic and cloud a patient’s judgement^{19, 27}.

Given these conceptual advances, the three objectives of this project were: (1) to determine if misunderstanding, measured with careful attention to recent conceptual advances, is as widespread as feared; (2) to identify the characteristics of participants who suffer from TM or TMis so these groups can be offered interventions to enhance the consent process; (3) to look for associations between TM and TMis and TO.

Methods

This research was approved by the Emory University Institutional Review Board and all respondents consented to participate.

Population

We interviewed and surveyed Phase I trial participants at an academic center, during the first month of their Phase I trial participation. All patients had an ECOG performance score of 0–2, and thus were ambulatory and “up and about” at least 50% of the time.

Instrument development

Since no validated instrument measures the current constructs of TM and TMis, we developed two new instruments, an in person face-to-face structured interview with open-ended questions and a paper and pencil self-administered quantitative survey. Using standard instrument development technique^28–30, we first conducted a comprehensive literature review to identify domains and relevant past studies that used scales of interest. The resulting questions were reviewed by a focus group of Phase I investigators, revised, and then cognitively tested with 12 cancer patients who were asked to assess question wording, to demonstrate comprehension by “thinking out loud” as they framed their answers and to determine if there were missing categories to query.

The final domains included motivation for participation, perceptions and estimations of risks and benefits, understanding of Phase I research and an expansive section on demographics, including attitude questions, measured on a 10 point Likert scale. The attitude questions were derived from two sources: (1) we queried strength of and comfort from religious or spiritual belief and strength of optimism based on Weinfurt et al²⁹; (2) we measured trust in medical research, trust in the phase I investigator and how many cancer treatment options respondents thought they had left based on suggestions that trust^{31, 32} and availability of options^{33, 34} may contribute to misunderstanding.

Operationalizing concepts

Therapeutic misconception

To gauge TM we asked two questions: (1) “Is the research study mostly intending to help research and gain knowledge or mostly intending to help you as a person?” (2) “Does the research study or your doctor decide the treatments?” ^{15, 23}. The first question focuses on the main intent of producing generalizable knowledge and the second on the difference between protocol-driven treatment and individualized care, the two core concepts found in most definitions of TM.

Therapeutic misestimation

In an attempt to obtain a complete picture of the respondents’ views of risks and benefits, we asked four open-ended questions: what was the main reason the respondent decided to participate in the Phase I study, what benefits the respondents hoped to gain by joining the study, what risks the study entailed and whether any risks were particularly concerning to the respondents.

We also asked respondents to estimate the probability of benefit and risk in the closed-ended survey. To differentiate the two uncertainty statements, frequency and belief, we asked the two questions suggested by prior research²⁹: “On average, how many patients out of 100 will benefit by having the growth of their cancer stop or slow down by the end of the study for at least six months? “(to elicit a frequency statement about the probability that the Phase I population will benefit -- population benefit question); “What is the chance that YOU will benefit and have the growth of your cancer stopped or slowed down by the end of this research study for at least six months?”(to elicit a personal belief statement about the probability that the patient will benefit -- personal benefit question). We poised similar frequency and belief questions for risk. Therapeutic misestimation is best understood as misestimating one’s personal chance of risk and benefit ²⁹, so our main analyses focused on estimations of personal benefit and risk.

Finally, since belief statements can serve purposes other than reporting facts, we asked what subjects meant by their estimate of personal benefit with the closed-ended choices of “1)Those are just the facts; 2)That is what I hope will happen; 3)That is what I fear will happen; 4)I think it is important to have a positive attitude.”

The correct chance of medical benefit for patients was identified as ≤ 20%, based on literature review ^{13, 35, 36} and our center’s experience. Since the Phase I trials in which the respondents were enrolled had different levels of risk, some posing very minimal risk, the Winship Cancer Institute Phase I investigative team determined that any probability of risk other than zero should be considered correct, a generous interpretation.

Therapeutic optimism was measured in two ways: Cohen’s scale²⁵ and Weinfurt’s attitude question measuring optimism on a 10 point Likert scale²⁹.

Method of administration

Phase I participants who consented to participate in this study were interviewed by the ethics research assistant at the first clinic visit after enrolling on the Phase I protocol. Open- ended answers were recorded verbatim on the structured interview form. The patient was then given the survey to self-administer, preferably at that clinic visit. However, it was possible to mail the survey. If the survey was not received by the next scheduled clinic visit, the patient was contacted at subsequent clinic visits over the next month and the survey was distributed and collected at that time. All interview and survey data were entered on a web based database by the ethics research assistant and double checked by the research coordinator.

Statistical Method

To identify factors associated with TM and TMIS, a two-sample t- test and/or Wilcox sum rank test was used for the continuous predictors, and a Chi-square test was used for categorical predictors. Logistic regression model was carried out for the multivariable analysis for TM and TMIS. Spearman correlation coefficient was calculated for the association between two continuous variables. The SAS statistical package V9.2 (SAS Institute, Inc., Cary, North Carolina) was used for all data management and analyses.

Open-ended questions querying benefits and risks were coded independently by the Phase I nurse practitioner and the ethics research assistant. After coding the first ten respondents, a standard code book was developed for each question, which was reviewed and revised by the principal investigator. The two coders recoded all the questions using the standard code book and all differences of opinion were resolved by the principal investigator. In addition, the codes developed for the two questions about benefits, that is, what benefits the respondents expected from trial participation and the main reason why respondents participated in the Phase I trial, were grouped by the ethics research assistant into the types of benefits identified in King’s seminal article, direct benefit from the trial intervention and collateral benefit resulting from being a research subject, such as extra testing, access to experimental therapies, and the gratification of altruism³⁷. Though King identifies a third benefit, aspirational benefit, it was not included in our study because King defines aspirational benefit as a benefit to society, and our study focuses on benefits for the participant. We report altruistic collateral benefit separately from the other collateral benefits in order to compare our data with previous reports of altruism^{8, 9}. The principal investigator reviewed and finalized the categorization of benefits into King’s categories.

Results

Of the 114 patients approached, fourteen refused, five did not complete both the interview and survey, yielding a sample of 95/114(83%). The median age was 57 (range 28–85), with 56% males, 67% white, 50% college graduates and 57% incomes ≥ to $60,000. The only significant difference between the respondents and refusers was that refusers were more frequently female (p= 0.029). Most respondents reported their strength of [81 (85%] and comfort from [81 (85%)] religious or spiritual belief as high. Trust, both in research [83 (87%)] and in the Phase I investigator [84 (88%)], was also high. Twenty-eight (29.5%) reported they had lots of options left in their cancer care, 38 (40%) a few options, 25 (26%) hardly any options.

Since similar measures of attitude were highly correlated we combined them into a single score to simplify the analysis: Strength of and comfort of religion belief (Spearman correlation = 0.92, p-value < 0.001) were combined into a single Religion score (REL); trust in research and in the investigator ((Spearman correlation coefficient = 0.40, p-value < 0.001) were combined into TRUST; the score on Cohen’s optimism scale and the single strength of optimism question (Spearman correlation coefficient = 0.52, p-value < 0.001) were combined into a single optimism score (OPT).

Therapeutic Misconception

Sixty-five (68.4%) respondents had therapeutic misconception, not correctly answering the two core questions. The question whether the research study or the doctor determines the study treatments was more frequently left blank or answered “do not know” than the question about the main purpose of the study (14.7% v. 1%, p=0.01) (Table 1).

Table 1.

Answers to Therapeutic Misconception Questions.

n=95		Question 1: Main Purpose?
Question 2: Treatment is decided by?		Correct	Incorrect	DNK/blank
	Correct	31 (32.6%)*	22 (23.2%)
	Incorrect	18 (18.9%)	9 (9.5%)	1 (1%)
	DNK/Blank	8 (8.4%)	6 (6.3%)

^*Lack TM because correctly answer both questions

Respondents with less than a college degree (p = 0.008), with family incomes less than $60,000 (p =0.001), and with a higher religion score (REL) (p=0.031) more frequently had TM. A multivariate analysis identified two significant factors associated with TM: education and income (Table 2).

Table 2.

Multivariate analysis for TM using logistic regression

Variable	Level	Odd Ratio	p-value
EDUCATION	< College degree	5.0	0.02
	≥ College degree	-	-
INCOME	< $60k	10.7	0.001
	>= $60k	-	-

Therapeutic misestimation

Only 3(3%) respondents correctly answered both the personal risk and benefit question, 3(3%) left at least one estimate blank and 89 (94%) misestimated. Most of the misestimations were overestimations of benefit, not surprising given our interpretation of correct risk estimates as anything other than zero. However, 13 (14%) respondents asserted that their personal risk was zero. TMis was only associated with OPT (p-value =0.007).

Overall, the estimates of personal benefit were high with 59 (62%) estimating ≥70% chance of benefit. Only 17 (18%) stated that their estimation of personal benefit was intended to be just the facts, while 31 (33%) stated it was what they hoped would happen, 2 (2%) what they feared would happen 43(45%) that it was important to have a positive attitude (2 missing). The estimates of personal risk were low with 59 (62%) estimating ≤ 30%.

Benefits of trial participation

When asked to list all the benefits they hoped to get from trial participation, respondents mentioned 155 benefits, with 102 (66%) direct medical benefits, 29 (19%) altruistic benefits (helping others and helping science) and 24 (15.5%) other collateral benefits (access to an academic center, financial help, access to cutting edge new treatments, and closer monitoring of my care). Forty-nine (52%) mentioned only direct medical benefit, 6 (6%) only altruistic, 9 (9%) only other collateral benefits and 31 (33%) mentioned more than one type of benefit, with all but 3 of those 31 including direct medical benefit. In sum, 77 (81%) mentioned direct medical benefit. When asked to rank benefits, direct medical benefit was most frequently ranked first [85/93 (91%)] and second [35 (38%)], with helping others ranked most frequently third [29 (31%)].

Motivation for participation

Sixty-three (66%) of the respondents offered only one type of reason for entering the trial: 45 (47%) joined to obtain direct medical benefit, 12 (13%) to help science or others, 3 (3%) to obtain other collateral benefits, (access to a cutting edge treatment, chance to fight my cancer), and 3 (3%) joined the trial based on the doctor’s recommendation. The remaining 32 (34%) offered more than one reason with the most frequent combination being direct medical benefit and altruism (17; 18%). Seventy-two (76%) had direct medical benefit as at least one reason for entering the trial.

Risks of Trial Participation

In the close-ended survey, 13(14%) estimated that their personal risk was zero. However, when asked in an open-ended question, 26 (27%) said there were no risks, and 56 (59%) said no risks were concerning to them. In total 90 risks were identified by 69 (73%) respondents and 46 risks were considered concerning by 39 (41%) participants. Codes that emerged from the verbatim description of risks were: (1) Risks directly attributed to standard of care interventions/medications; (2) Risks directly attributed to the research intervention/medications; (3) Risks due to non-therapeutic features of the research protocol, such as, the trial can be stopped at any time or there is a non-therapeutic biopsy; (4) participation can lower quality of life; (5) Risks inherent in standard of oncologic care but arguably exacerbated in research: unknown side effects; uncertainty of outcome; fear it won’t work; need to postpone other, perhaps more effective, treatment; breach of confidentiality. The frequencies of the respondents mentioning these risks and the risks that are concerning to them are tabulated in Table 3.

Table 3.

Frequency of Risks Mentioned in Response to an Open-Ended Query

	Risks mentioned by respondents # respondents (%)	Respondent identifies the risk as concerning # respondents (%)
1. Side effects of the investigational agent	37 (39%)	31 (33%)
2. Side effects of standard of care medications	8 (8%)	3 (3%)
3. Risks novel to research due to research design	3 (3%)	0
4. Lowering quality of life	3 (3%)	4 (4%)
5. Risks of any cancer treatment that may be exacerbated in research	39 (41%)	8 (8%)
a. Breach in confidentiality	1	0
b. Need to postpone other, perhaps more effective, treatment	3	1
c. Unknown side effects	19	4
d. Uncertainty of outcome	7	2
e. Fear treatment won’t work	9	1
6. None	26 (27%)	56 (59%)

Therapeutic Optimists

Thirty-nine (41%) were classified as therapeutic optimists(TO) using Cohen’s scale²⁵ and most [82(86%)] ranked their optimism as high. Most respondents (54.6%) estimated their probability of personal benefit higher and their probability of personal risk lower than their estimates for the population. However, there was a significant minority (36.7%) who either estimated their personal benefit as lower or their personal risk as higher than the population (Table 4).

Table 4.

Comparisons of Respondents’ Personal and Population Estimates of Risk and Benefit

Frequency (percent)		Higher personal benefit than population benefit
		Yes	No
Lower personal risk than population risk	Yes	42 (54.6%)	15 (20%)
	No	14 (16.7%)	5 (6.7%)

Discussion

Although the study sample’s demographic characteristics are not representative of the study’s catchment area (more white and higher income), they are representative of Phase I populations in general, both in regard to standard demographics³⁸ and to trust³⁰.

On the face of it, even with a more sophisticated analysis of TM and TMis, understanding levels were low. Therapeutic misconception was widespread (68%) and therapeutic misestimation, misestimating one’s personal potential for benefit and risk, was nearly universal. The concern that Phase1 participants’ main motivation for participation is direct medical benefit, when the chance of such benefit is not high, also appeared to be substantiated.

However, the conceptual advances made in the last decade raise serious concerns about interpreting this data. First, although most could not correctly answer both questions posed as the test of TM, more than half of respondents could identify the main purpose as helping research, a higher percentage than found by Daugherty (9/27; 33%)² though lower than found by Jansen(51/70;72.9%)¹⁹, both of whom poised open-ended questions. Second, 14.7% did not answer the query whether their doctor or the research determined the treatment, though all but one answered the question about the main purpose of the trial. This result hints that it may be difficult for subjects to provide a confident answer about how the physician is operating in clinical research, supporting Wendler’s and Grady’s contention that misconstruing the patient-investigator relationship as a patient-clinician relationship is a source of misunderstanding²³.

The conceptual advances also call into question conclusions about the prevalence of TMis. Seventy-six (80%) respondents stated they were not making a statement of fact when reporting their estimate of personal benefit, but rather were expressing their hope or a positive attitude. Therefore, even though most participated for direct medical benefit, we do not know if they also overestimated their chance of personal medical benefit, since most were not offering factual estimates.

Some have reported that the high estimates of personal benefit indicate that Phase I participants suffer from unrealistic optimism¹⁹, assuming they will do better than other patients similar to them. Arguably, unrealistic optimism can cloud one’s appreciation of what is entailed by Phase I participation and threaten decision making capacity. The association we found between OPT and TMis may support this claim. However, we found an intriguing subgroup (36.7%) who appear to be pessimists, estimating their personal benefit lower than the population’s chance of benefit or their personal risk higher than the population’s chance of risk. A tendency toward pessimism has not been documented before and reveals a more complex picture of some early phase trial participants’ thinking about risks and benefits. Perhaps, just as many of the estimates of personal benefit were expressions of hope and a positive attitude, the estimates of risk were expressions of preparation for the worst to happen. A limitation of our study was that we did not specifically query what the respondents meant by their estimates of personal risk, leaving this question open for future research.

Although incorporating the various conceptual caveats into our study design resulted in casting doubt on any conclusions about the rate of therapeutic misestimation, we did glean a fair amount of information about these respondents’ views of benefits and risks.

Even though most participated in order to obtain direct medical benefit, about half cited medical benefit as their sole reason for participation and a third mentioned multiple benefits including collateral benefits, supporting Agrawal’s and Emanuel’s⁴ contention that at least some phase one participants have a broad view of benefits.

The two main categories of risk identified were the risk of the investigational agents (41%) and risks inherent in any cancer treatment that may be exacerbated in research (43%), such as the uncertainty of outcome and unknown side effects. One could argue²³ that the risks of the interventional agents are exactly the ones to be focused on. Interestingly, many of these respondents, all of whom had exhausted standard therapy, were aware of risks found in any cancer treatment that may be exacerbated in research, but only 3 mentioned the novel risks of trial participation, such as extra biopsies, PKs, variable and non-individualized dosing. This result supports the concern that participants either do not understand or do not focus on the key differences between individualized care and research, namely that being on a clinical trial introduces procedures and practices not found in standard care.

More of these participants (13%) were motivated by altruism than previously reported for Phase I participants ^{8, 9}, though this level is similar to the population at large found by the President’s Advisory Committee on Human Radiation Experiments, which also found, like this report, a mixture of motives for trial participation²⁸.

The oft-found demographics associated with misunderstanding of research surfaced again: TM was associated with lower education and lower family income. However, we queried an extensive set of attitudes as well as standard demographics and uncovered several noteworthy results. The supposed vulnerability of perceiving oneself to have hardly any options for cancer treatments was not significantly correlated with TM. This finding lends support to the contention that Phase I participants are not more prone to misunderstanding than others due to their lack of treatment options³⁸. Nor did our data support the theory that trust or optimism may account for TM³² since neither TRUST or OPT were significantly associated with TM, though OPT was associated with TMis. This results supports Jansen et al’s data that an optimistic bias is not associated with misunderstanding¹⁹. Like Weinfurt, we also found a correlation between REL and TM ²⁹, suggesting a connection that deserves further research.

Since we could not identify a single demographic associated with both TM and TMis, we suggest that, rather than targeting subpopulations to improve understanding, the three concepts found most lacking in this study be emphasized in the consent process. More than two-thirds could not identify both the main purpose of research and lack of individualized care, so each of these should be emphasized. In addition, the largest deficit in appreciation of risks was appreciation for the novel risks introduced by Phase I participation such as variable dosing and additional non-therapeutic tests. These risks should be highlighted.

This study has several limitations. It is a single institution study at an academic Phase I cancer research center, so the results may not be generalizable. The study population was heavily white with higher incomes, though this is typical of Phase I studies. We do not know if estimations of personal risk were intended as statements of fact. However, the study design did take advantage of the many advances in conceptual analyses for TM and TMis and specifically queried estimates of risks as well as benefit, an advantage over much previous research. Finally, since the interviews and surveys were administered up to a month after the consent document was signed, faulty recall may in part be responsible for the high level of TM, though even directly following the consent conversation patients do not remember that key pieces of information were discussed³⁹.

Conclusion

Conceptual advances allowed us to measure TM and TMis more carefully, substantiating that the majority of these respondents suffer from TM. TM was associated with lower income and higher education, but not with the perception of few treatment options, undermining the view that Phase I participants are vulnerable due to limited treatment options. The majority participated in hope of direct medical benefit though many listed other benefits as well. However, whether or not these participants overestimated their chance of personal benefit is unknown since more careful measurement revealed that proffered estimates were often not intended to report facts. This careful measurement also resulted in a novel finding. A substantial minority were pessimists, estimating their chance of benefit as lower or their chance of risk higher than that of the population. Although a fourth did not list any risks at all, just less than half were aware of both the risks of the investigational agent and the risks that may be exacerbated by research, both important to understand. What was missing was attention to the novel disadvantages introduced by research participation, such as non-therapeutic procedures, which were almost never mentioned. Building on the conceptual advances made in the last decade it is now clearer that TM continues to be a problem and a major deficit in these respondents understanding is grasping the novel ways in which research differs from standard of care.