Patient familiarity with, understanding of, and preferences for clinical trial endpoints and terminology

Abstract

Background:

While there is increased attention to designing and explaining clinical trials in ways that are clinically meaningful for patients, there is limited information on patient preferences, understanding and perceptions of this content.

Methods:

Using maximum difference scaling (MaxDiff) methodology, we developed a survey assessing understanding of 19 clinical terms and perceived importance of nine endpoint surrogate phrases employed in clinical trials and consent forms. The survey was administered electronically to individuals with metastatic breast cancer affiliated with the Metastatic Breast Cancer Alliance. Analysis was performed using the “bayesm” package in the R Statistical Package.

Results:

Among 503 respondents, 77% had a college degree, 70% were diagnosed with metastatic disease ≥2 years prior to survey completion, and 77% had received ≥2 lines of systemic therapy. Less than 35% of respondents reported understanding “fairly well” the terms “symptomatic progression,” “duration of disease control,” “time to treatment cessation,” and “endpoints.” Income level and time since onset of metastatic disease correlated with comprehension. Patients who had received six or more lines of therapy perceived “time until serious side effects” (P<0.001) and “time on therapy” (P<0.001) to be more important compared to patients who had received one line of therapy. Positively phrased parameters were associated with increased perceived importance.

Conclusions:

Even among educated, heavily pre-treated patients, many commonly used clinical research terms are poorly understood. Comprehension and perceived importance of trial endpoints varies over the course of disease. These observations may inform the design, discussion, and reporting of clinical trials.

Precis:

Regulatory and professional organizations increasingly recognize that patients should play an important role in the design of cancer clinical trials. Our findings indicate several commonly employed endpoints are poorly understood by patients, and that the relative perceived importance of endpoints may change over the course of the disease, suggesting that these observations be considered in the design, discussion, and reporting of clinical trials.

The selection and prioritization of endpoints for cancer clinical trials remains an area of ongoing discussion among clinicians, researchers, regulatory authorities, and patients.(1, 2) The U.S. Food and Drug Administration (FDA) recently updated guidance for clinical trial endpoints for the approval of cancer therapies. These revised documents emphasize the importance of clinical endpoints that demonstrate prolonged survival and/or reduced symptoms, among other tumor-centered endpoints.(3, 4) Additionally, recognizing the importance and value of patient perception, in 2012 the FDA committed to seeking patient feedback for specific diseases. From a 2015 meeting of breast cancer patients or representatives convened by the FDA, it emerged that quality of life was the primary consideration when deciding among treatment options.(5)

A separate survey found that patients with metastatic breast cancer believed that PFS was an important endpoint, even if overall survival was unchanged. Specifically, respondents perceived PFS to be associated with increased physical functioning and emotional well-being.(6) In another study, the majority of patients with metastatic breast cancer were willing to accept all proposed toxicity scenarios if accompanied by a 27 percent to 33 percent likelihood of therapeutic benefit.(7) By contrast, a study of individuals with ovarian cancer found that most were willing to forego more than six months of PFS time to reduce toxicities of nausea, vomiting, neuropathy and abdominal pain.(8) However, it is not clear to what extent patients comprehend these concepts. A recent meta-analysis of 17 studies (representing more than 3,600 patients with advanced cancer) evaluating understanding and values toward PFS as an endpoint found that wide variation in definition and methodology limited result interpretation.(9)

While the pharmaceutical industry, regulatory agencies, and research community have historically determined which clinical trial endpoints to prioritize,(10–12) the notion of patient-centered care has gained increasing importance.(2, 13–15) For patients to contribute meaningfully to these discussions, however, they must be able to understand concepts central to disease diagnosis and management. Nevertheless, it has been demonstrated that most patients may not even have a basic understanding of the role of chemotherapy to treat their cancer.(6, 16) Other cancer-related wording may also be poorly understood, including terms such as “oncologist,” “pathologist,” and “radiologist.”(17)

Given these observations, we sought to determine patient understanding and perceptions of clinical trial endpoints and associated terminology employed in protocol documents and informed consent forms. We also determined how patient preferences change over the course of disease.

Methods

Survey Design

The survey was designed by the Research Advocacy Network (RAN) and approved by the Chesapeake Institutional Review Board (IRB). The initial endpoints and terms were identified based on terms commonly used in clinical trials and items under debate in the literature. In addition, patients with metastatic breast cancer and patient advocates participated in focus groups intended to help create survey options focused on clinical endpoints in patient-friendly language. This set of items was reviewed and revised with the goal of reducing redundancy and improving patient friendliness of language. Finally, test phrases were administered to a volunteer cohort of five patients with metastatic breast cancer. Each patient was interviewed after test survey completion, with questions focused on the perceived length, understandability, ease, and importance of the survey. Based on this feedback, phrasing and formatting were adjusted and incorporated into the final version, which consisted of 9 items (Appendix A).

Respondents provided informed consent prior to starting the survey. Demographic and disease-related information were collected as reported by patients, including time since onset of metastatic disease, lines of systemic therapy, and current treatment situation. Respondents selected the most important and least important information to know about a new treatment using maximum difference scaling (MaxDiff) methodology. MaxDiff is a type of best-worst scaling methodology that evaluates all possible pair combinations and has traditionally been utilized in economic studies.(18) Respondents select the most and least important parameters from a set of options. Recently, the approach has been applied to health care-related surveys.(18–21) Patients were queried about nine concepts (Table 1) through a series of multiple questions posed in various permutations. Responses were analyzed using a multinomial Logit model with the “most important” selections coded normally and the “least important” selection reverse coded. The output of the routine is a set of non-linear regression coefficients with higher scores reflecting increased relative importance.(22)

Table 1.

Survey Concepts and Endpoint Representation

Survey Concept	Endpoint Correlation
Time until treatment is stopped because disease gets worse	Progression-free Survival
Time until treatment stopped for any reason (disease gets worse, treatment too toxic, etc.)	Time to Treatment Failure
Length of time disease remains stable (unchanged)	Stable Disease
Time patients remain on treatment because they perceive some benefit even though disease getting worse	Time on Treatment
Time patient continues everyday life at an acceptable level	Everyday Life
Time until patient experiences serious side effects that may require hospitalization or cause organ damage	Adverse Events
Length of time overall quality of life is acceptable to patients	Quality of Life
Time until new metastasis is identified or found	Time to New Metastasis
Length of time cancer responds to treatment	Duration of Response

Respondents were asked to report if they were familiar with and understood the following terms “fairly well,” as performed in prior comprehension studies(23): quality of life, no evidence of disease, stable disease, overall survival, progression-free survival, adverse events, time on treatment, time to new metastasis, response rate, time to progression, patient-reported outcomes, disease response, progression-free duration, time to treatment failure, duration of response, symptomatic progression, duration of disease control, time to treatment cessation, and endpoints. Respondents also identified methods of self-education of advances in breast cancer treatment that they engaged in, their preferred measurement for monitoring treatment response, and their preferences for knowing about disease progression in the absence of symptoms.

Respondent Selection

We distributed the survey to organizations who are part of the Metastatic Breast Cancer Alliance (MBCA), who then sent it to their members via blast emails, Internet postings, and social media. Screening questions identified individuals with metastatic breast cancer, who were invited to complete the full survey. Respondents were provided a $20 gift card as compensation for their time and effort.

Survey Analysis

The targeted sample was 500 respondents to allow for subgroup analysis. Bayesian P-values are reported and the analysis was performed using the R platform with the “bayesm” package in addition to custom code for statistical analysis.(24–26)

Results

Initially, 1,038 individuals accessed the survey through the provided hyperlink. Among them, 191 (18%) did not advance beyond the introductory screen, 189 (18%) did not complete screening questions (confirming a diagnosis of breast cancer and metastatic stage), 13 (1%) did not report metastatic stage and therefore were not eligible to continue, and 68 (7%) did not complete the online consent process and therefore did not initiate the survey. Among the 577 respondents who started the survey, 503 (87%) completed it. All responses were received within one day of distribution. Median time to survey completion was 17 minutes. Respondents who initiated but did not complete the survey (n=77) tended to abandon the survey early in the process. Their responses were removed from the dataset because reasons for not completing were not available. Consequently, there was no missing data to report. Because demographic data were collected toward the end of the survey (screens 33–39 of 43 total), only seven respondents who did not complete the survey provided this information, precluding any analysis of this cohort.

Table 2 shows the self-reported demographic and disease-related characteristics of respondents. Almost all respondents were female, 90% were white, 77% had a college degree or greater, and 64% had an annual household income of $50,000 or greater. Seventy percent of patients were diagnosed two or more years prior; 77% had received two or more lines of systemic therapy; and 70% were receiving active treatment at the time of survey completion.

Table 2.

Demographics and clinical characteristics

Clinical Characteristic	N (%)
Age
<40	53 (11)
40–49	112 (22)
50–59	150 (30)
60–69	132 (26)
≥70	53 (11)
Not stated	3 (1)
Sex
Female	500 (99)
Male	2 (0)
Prefer not to answer	1 (0)
Race *Multiple responses allowed
White/Caucasian	455 (90)
Hispanic/Latin American	24 (5)
African American/Black Caribbean	13 (3)
Asian	10 (2)
Native American	7 (1)
Indian (Indian Subcontinent)	5 (1)
Middle Eastern	3 (1)
African	2 (0)
Other	9 (2)
Not stated	6 (1)
Household Annual Income
<$15,000	16 (3)
$15,000-$24,999	25 (5)
$25,000-$49,999	62 (12)
$50,000-$74,999	71 (14)
$75,000-$99,999	73 (15)
>$100,000	180 (36)
Don’t know/prefer not to answer	76 (15)
Time since metastasis discovered
< 2 years	153 (30)
2 to 5 years	216 (43)
> 5 years	134 (27)
Number of regimens for metastatic breast cancer
None	7 (1)
1	111 (22)
2 to 5	291 (58)
6 or more	94 (19)
Current clinical situation
In remission or bone metastasis only, no evidence of active disease	144 (29)
Currently receiving treatment (stable, too early, or not responding)	351 (70)
No standard treatment options, palliative, seeking trials	8 (2)

Respondents’ familiarity with and understanding of specific oncology clinical trial-related terminology were stratified by time since onset of metastatic disease (Figure 1) and annual household income (Figure 2). In the overall cohort, the term “quality of life” was most commonly understood (89%), with the term “endpoints” least frequently understood (13%). Overall, commonly used terms such as “response rate,” “patient-reported outcomes,” and “time to progression” were understood by 40–50% of respondents. Diagnosis of metastatic disease at least five years prior to survey completion was significantly associated with improved understanding of all listed phrases compared to respondents with diagnosis of metastatic disease within two years (Figure 1). Respondents with household annual income greater than $100,000 were significantly more likely to understand all terms evaluated, with the exception of “quality of life,” compared to those with household annual income less than $50,000 (Figure 2).

Figure 1.

Proportion of respondents reporting familiarity with and understanding of clinical trial terminology stratified by time since onset of metastatic disease*

Figure 2.

Proportion of respondents reporting familiarity with and understanding of clinical trial terminology stratified by annual household income*

*Phrases with significant differences shown

Table 3 lists the perceived importance of clinical trial endpoints according to lines of therapy received. “Time cancer responds to treatment” was considered the most important endpoint, regardless of respondents’ treatment history (P<0.001), followed by “time disease remains stable” and “time quality of life remains acceptable.” For some endpoints, we observed trends according to respondents’ own prior treatment history. Respondents who had received two or more lines of therapy considered “time until serious side effects” more important than those with less prior therapy. Conversely, “time cancer responds to treatment,” “time disease remains stable,” and “time quality of life remains acceptable” were considered more important by respondents who had received only one line of therapy. The most heavily treated patients (≥6 lines of therapy) attributed greater importance to “time until treatment stopped – disease gets worse,” “time on treatment – some benefit though disease worsening,” “time until serious side effects (e.g., hospital, organ damage),” and “time until treatment stopped – any reason.”

Table 3.

Perceived importance of endpoints stratified by lines of therapy received

Survey Question	MaxDiff score(a)			P-value(b)
	Lines of therapy received
	1	2–5	≥6
Time cancer responds to treatment	1.67	1.43	1.31	<0.001
Time disease remains stable	1.44	0.98	1.07	0.007
Time quality of life remains acceptable	1.14	0.61	0.67	0.015
Time everyday life at acceptable level	0.67	0.63	0.11	<0.001
Time until serious side effects (e.g., hospital, organ damage)	−0.63	0.14	−0.10	<0.001
Time until new metastasis	−0.22	−0.42	−0.42	0.076
Time until treatment stopped – disease gets worse	−0.78	−0.58	−0.33	<0.001
Time until treatment stopped – any reason	−0.88	−0.62	−0.41	<0.001
Time on treatment – some benefit though disease worsening	−2.41	−2.16	−1.89	0.003

^(a)Higher MaxDiff scores indicate greater perceived importance.

^(b)For comparison of 1 line of therapy and ≥6 lines of therapy.

More than two-thirds of patients selected “monitoring tumor number and size” as their preferred method to measure treatment effect, and less than one-third selected “cancer has progressed” or “treatment has stopped working” as their preferred measure. Almost all (97%) respondents reported wanting be informed of tumor progression in the absence of new symptoms. At least 75% of respondents reported receiving education about breast cancer advances through the following methods: “read about treatments that may apply to them,” “read about new treatments,” and “read articles in oncology/health newsletters” (Table 4). Over half of respondents reported reading scientific research and discussing advances with others, and almost half reported attending breast cancer conferences or meetings.

Table 4.

Preferred methods of education about breast cancer treatment advances

Education Method	N (%)
Read about treatments that may apply to me	399 (79)
Read about new treatments or approaches	384 (76)
Read articles in cancer/health org newsletters	376 (75)
Discuss treatment advancements/research with others	321 (64)
Read scientific articles/research about what is being studied	292 (58)
Attend breast cancer conferences/meetings on treatment advancements	231 (46)
Read newspaper articles	209 (42)
Other	53 (11)
Do not try to do much	17 (3)

Discussion

In an era when patient perspectives are considered paramount in clinical trial design and conduct, relatively little is known about their preferences in this domain. We undertook the present study to provide a comprehensive assessment of metastatic breast cancer patients’ familiarity with, understanding of, and perceived importance of oncology and clinical trial terminology and endpoints. We found that comprehension varied widely across concepts, with almost all patients reporting understanding “quality of life,” and about half of patients understanding “response rate,” “adverse events,” and “time to progression.” Yet even in this highly educated population (more than three-fourths college graduates), less than one-third of respondents understood such commonly used phrases as “symptomatic progression,” “duration of disease control,” “time to treatment cessation,” and the term “endpoints.” Although potentially similar in content to “quality of life,” the term “patient-reported outcomes” was understood by less than half of respondents. Furthermore, actual terminology understanding may be lower, as there may be a discordance between perceived and actual comprehension. For instance, in one phase 1 trial, more than 90% of subjects stated they understood most or all of the information provided, but only 33% were able to state the purpose of the trial when asked directly.(27)

These observations are particularly noteworthy because survey respondents were not only highly educated, but also tended to have extensive personal experience with the disease and treatment. More than two-thirds had been diagnosed with metastatic disease for at least two years, and more than one quarter for at least five years. More than three-quarters had received multiple lines of prior systemic therapy. Low rates of comprehension could reflect lack of familiarity with certain terms, even if medical providers have discussed related concepts. Alternatively, patient uncertainty regarding specific terminology could arise because the definition of some endpoints remains debated even among disease experts and clinical trialists.(28)

Why do these findings matter? Meaningful communication of a clinical trial’s endpoints, design, and results appear closely linked with patient interest and acceptance. “Patient-centric trials”—a broad term that reflects the involvement of patients in trial design to ensure that trial outcomes are relevant and meaningful to patients, and that participation is as convenient as possible—appear to have greater likelihood of achieving various measures of success. One recent analysis conducted by The Economist found that, across multiple therapeutic areas including oncology, drugs developed via patient-centric designs were more than 20% likely to be approved, and that recruitment rates for patient-centric trials were almost twice those of other studies.(29) Similarly, in a United Kingdom study, among 276 patients with gastrointestinal and hematological malignancies approached about clinical trial participation, 50% would have preferred more information on study interventions and procedures.(30) Given the low rates of trial participation by U.S. cancer patients and the resulting increase in accrual timelines and rates of premature study closure,(31) attention to these considerations may benefit investigators, sponsors, and regulatory authorities.

Indeed, it has been suggested that the public may be reluctant to participate in clinical trials precisely because they do not feel that they are considered partners in the process.(32) Instead, current messaging may reinforce knowledge gaps between scientists and the public. Half of U.S. adults have low health literacy; that is, they do not understand how to read a prescription label or what the dosing means.(32) Clear language is necessary but not sufficient to create understanding, because messages often assume more scientific knowledge than people have. Indeed, only 5–15% of the U.S. population is considered science literate.(32) As a striking example, reported interpretations of the word “trial” include invoking a process involving a judge, a hardship, or within the context of “trial and error,” all of which have negative connotations.(32)

In the current era of clinical research expansion and de-centralization, these considerations may be more important than ever. Clinical trials are now more likely to be conducted in private practice settings than at academic medical centers.(33–35) With less extensive clinical research infrastructure, these sites may particularly benefit from sponsor-provided patient-centric materials to communicate clinical trial goals and details.

Our findings may also have relevance to the communication of clinical trial results to participants and the general public. On average, more than 90 percent of clinical trial participants report never learning about the trial’s results.(36) Yet more than 70 percent of trial subjects prefer to be informed of trial outcomes,(30) and most patients report that the prospect of receiving trial results is a major factor in the enrollment decision.(37) One study found that provision of a lay summary increased participants’ understanding of basic points about a trial (such as why it was conducted) by 65 percent.(36) Furthermore, certain regulatory authorities now require clinical trial layperson summaries. For instance, European Union Clinical Trials Regulation states that such summaries should not “assume any prior knowledge of the trial, of medical terminology or clinical research in general.”(38) The Institute of Medicine has recommended that data sharing from clinical trials be the “expected norm.”(39)

In our study, patient preferences varied widely across various clinical trial endpoints. Notably, positively phased terminology (eg, “time cancer responds,” “time disease remains stable,” “time quality of life remains stable”) was associated with greater perceived importance than was negatively-phrased terminology (eg, “time until treatment stopped,” “disease gets worse,” “some benefit though disease worsening”). We also observed differences according to respondents’ own treatment history. Those who had received less treatment assigned greater importance to disease stability, response to treatment, and quality of life. Respondents who had received multiple lines of therapy were more likely to prioritize time until serious side effects and time until treatment stopped. In advanced lung cancer, patient preferences regarding toxicities to be avoided change over the course of therapy. Shortness of breath was the leading concern initially, but was overtaken by fatigue later in the clinical course.(40) Almost half of individuals with lung cancer also change their definition of treatment success. Among those who initially consider success exclusively according to survival metrics, 80 percent subsequently modify the definition to include quality of life.(41) These earlier studies and our current observations suggest that the patient perspective may change over the course of treatment. Early on, patients may be less familiar with the nature and potential severity of toxicities, and may assign greater weight to disease stability and treatment response. Conversely, after multiple lines of therapy, patients are likely to have already experienced metastatic progression and adverse events. Having already received numerous lines of treatment, they may perceive that ongoing treatment opportunities are limited, have greater apprehension about exhausting therapeutic options, and therefore prioritize treatment continuation. Whatever the etiology of these differences, they demonstrate the importance of considering patient input tailored to the specific clinical scenario under study in a given clinical trial.

Most respondents pursued various sources of information for updates on advances in breast cancer treatment. Interestingly, those with household annual income of $50,000 or less were just as likely to educate themselves about breast cancer using all options except newspapers, suggesting a potential opportunity to create appropriate tools and language that broadly meets patient needs. How the lay press conveys the design, status, and results of cancer clinical trials is likely to become an increasingly complex consideration, as many of these diverse electronic and printed media sources may not be subject to rigorous editorial oversight of terminology and endpoint descriptions. To address the low rates of patient understanding of certain comments, this survey identified key educational opportunities. For instance, almost half of respondents reported attending breast cancer conferences or meetings. However, rates of engaging in such events are likely far lower in the general breast cancer population compared to the self-selected and motivated sample queried in the present study.

The relative unfamiliarity with the term “patient-reported outcomes” (PROs) also merits consideration. One option would be to identify alternate terminology that is more recognizable and comprehensible to patients. Given patients’ high-level understanding and prioritization of quality of life, such an approach may improve adherence to these protocol metrics, as well as the reach of clinical trial results. Patients with breast cancer have reported that access to PRO endpoints assists them in navigating oncology treatment decisions and that the oncology community can no longer tolerate the lack of PROs in clinical trials.(42) Without access to PROs, patients may instead turn to the internet (website discussion boards, blogs, Facebook, and Twitter posts) for information of uncertain reliability.(42) Despite the growing interest in PROs within the clinical research community, utilization of these data remains suboptimal.(43) Specific limitations include heterogeneity in constructs and measures, inconsistent use in trial design, missing data, and novel statistical considerations.(14, 44, 45) Only one-third of breast cancer phase 3 trials collect PROs; even when collected, PROs results are published only half of the time.(46, 47) A reasonable step forward, therefore, may be to work to improve understanding by the public that PROs represent many of the parameters prioritized by patients, or to identify alternate terminology as mentioned above. In either case, heightened public awareness of PRO inclusion in current and future trials may increase patient interest in research, sponsor attention to the findings, and reporting of PRO results.

The current results suggest the following steps moving forward: (1) Ensure use of “patient-friendly” language in materials describing a clinical trial’s goals and procedures; (2) Assess comprehension of language by patients, caregivers, and the general public not only by asking if terms are understood, but also evaluating their ability to respond to content-specific questions; (3) Consider modification and reporting of trial endpoints according to patient priorities in a particular clinical scenario (such as line of therapy); (4) Routinely provide trial results in “patient-friendly” language, which may improve not only engagement and interest by potential study subjects, but also by the general public; (5) When feasible, preferentially employ positively framed terminology in patient- and public-facing materials about clinical trial design and outcomes.

Key strengths of this study include the large number of respondents and the recording and diversity of disease and treatment experience. Indeed, the on-line survey format proved not only feasible, but also remarkably efficient, with more than 500 surveys completed within 24 hours. Limitations include a single-disease focus, the potential for respondent bias due to selection of respondents from among members of a breast cancer organization, and lack of quantification of patient preferences. For instance, in a survey of clinical trial endpoints in patients with gynecological cancers, most respondents reported that a meaningful extension of overall survival and progression-free survival for a new agent should be at least five months.(48) In addition, the respondents in this study were younger, whiter, and had higher household income compared to the overall population with metastatic breast cancer, and were also required to be competent utilizing the internet to complete the survey. Unfortunately, demographic and clinical data for the general MBCA membership are not collected, so comparison with our responder cohort is not feasible. In addition, total number of MBCA members is unknown. Similarly, we do not have sufficient demographic data on the respondents who initiated but did not complete the survey to characterize this cohort.

In conclusion, this study demonstrates that patient understanding of oncology and clinical trial terminology may be quite limited, even in a highly motivated, experienced, and educated cohort. Additionally, patient preferences and priorities appear to change substantially over the course of disease. These observations may inform the design, discussion, and reporting of clinical trials, which in turn may impact patient interest and participation.