Evaluation of an Educational Program to Improve Communication with Patients About Early-Phase Trial Participation

Results from an evaluation of a Phase I educational initiative modeled around a method shown to be successful at enhancing health care professionals' discussions about randomized clinical trials with their patients are presented. A brief workshop on this method changed communication skills competency and self-confidence of participants.

Abstract

Background.

Discussing early-phase cancer trials is challenging; most offer little personal benefit to patients with life-limiting illnesses who frequently have a poor understanding of and misconceptions about the therapeutic aims. We evaluated an evidence-based training program aimed at enhancing communication.

Methods.

Prior to and after the intervention, 47 health care professionals (HCPs) experienced in early-phase trial recruitment were audio taped discussing trials with patient simulators who completed postinterview evaluations. Coders rated the interviews for the presence of information areas required to elicit ethical consent. HCPs reported their self-confidence on 15 different aspects of trial discussion.

Results.

Significant objective and subjective improvements in communication were found after the workshop. Analyses of audio tapes showed positive shifts in: establishing the patient's knowledge of their prognosis (odds ratio [OR], 2.7; p = .002), discussing symptomatic care (OR, 3.8; p < .001), the aims of the trial (OR, 2.6; p =.002), and the unlikelihood of medical benefit (OR, 3.0; p = .021). Patient simulator ratings showed improvements in: the awareness of palliative care and symptom control (OR, 2.1; p = .004), the voluntariness of participation (OR, 3.7; p = .015), the opportunity to ask questions (OR, 2.9; p = .044), and the time to consider participation (OR, 6.1; p = .009). HCPs' self-confidence increased significantly for all 15 items (OR range, 1.5–2.9; p ≤ .001).

Conclusion.

This short, intensive workshop changed communication skills competency and self-efficacy in ways likely to promote valid, ethically informed consent from patients contemplating trial entry.

Introduction

Phase I cancer trials involve the first scientific testing in humans of novel drugs or treatments outside the laboratory primarily to examine safe dose ranges and monitor adverse events. Because phase I trials are not usually conducted with an expectation that participants will derive any direct personal medical benefit, recruitment of patients presents many communication challenges. For example, patients' vulnerability heightens the risk that they may pursue experimental treatment out of desperation, and some assume that if a trial is being offered to them by a physician then it must be beneficial.

Eligible patients have generally exhausted all standard treatment options aimed at halting disease progression, yet continue to be functionally well [1]. Trial enrollment frequently demands considerable effort from these patients, with the burden of extra hospital visits for tests, treatment administration, and monitoring. Studies reveal that many patients have a poor understanding about numerous aspects of trials necessary for fully informed and ethically valid consent [2]. Ideally, prior to any discussion about trial entry, putative early-phase trial patients should have had a clear “end of treatment” interview with their physician during which understanding of their prognosis was established and the positive aims and benefits of good palliative care and symptom control were discussed [3]. However, research shows that many oncologists find such consultations uncomfortable, omitting important issues or using obtuse and ambiguous terminology that can confuse patients and subsequently lead to misinterpretation of the trial's aims [4]. For example, a recent U.K. study examining the informed consent process between oncologists and patients showed several deficiencies, omissions, and misunderstandings in early-phase trial discussions; these included a failure to check understanding about the prognosis, a lack of clarity about the experimental nature of the trial, and inadequate discussion of symptomatic care options [5]. Interviews that lack such key information elements may leave patients and their relatives believing that trial entry is their only option and with unrealistic hopes that the novel treatment will be of significant personal benefit to them. These “therapeutic misconceptions” or mistaken beliefs “that the research, like the treatments patients have received previously, is designed and will be executed in a manner of direct benefit to them” [6] call into question the validity of the patient's consent and spur criticism of health care professionals' (HCPs) communication skills.

Another interpretation is that these patients are in fact “therapeutic optimists,” continuing to hope for the possibility of benefit despite being aware of their prognosis and recognizing the experimental nature of the trial [7]. Irrespective of the predisposition toward optimism that patients and relatives might have about putative therapeutic benefits, improving communication about phase I trial recruitment is critical to ensuring that prospective participants make informed decisions.

Here, we report results from an evaluation of an educational initiative modeled around a method shown to be successful at enhancing HCPs' discussions about randomized clinical trials with their patients, which is now being used in oncology centers worldwide [8].

Methods

The Educational Intervention

The workshop lasted 8 hours, split over 2 days. It comprised a mix of didactic presentations of relevant data plus facilitated group discussion around five DVD-based scenarios created specifically for the program. The content was developed in close collaboration with experienced doctors and research nurses and from original evidence-based research [5]. A handbook accompanying the five DVDs was developed for use by facilitators. The DVDs all had a time code matching a time-coded commentary in the facilitator handbook about the communication issues being raised and with suggestions about appropriate places to stop and engage groups in exercises or discussion. Dummy referral letters outlining the patients' histories together with trial proformas were provided. Finally, there was a bibliography of relevant reading materials and a section on the ethical perspectives related to early-phase trial recruitment.

Modules

The five program modules have scenarios depicting a wide range of trials, tumor sites, and patient characteristics. Module 1 involves a trial of a standard drug plus a new agent and illustrates some of the communication demands when dealing with a distressed younger man with metastatic malignant melanoma still desperate to find potentially curative treatment. Module 2 shows an oncologist talking about a novel monoclonal antibody trial with a metastatic colorectal cancer patient. In module 3, an oncologist conducts an “end of active treatment” interview with a woman with metastatic breast cancer. Her husband, who wishes to leave no stone unturned in a quest for further anticancer treatments, including experimental ones, accompanies the patient. In module 4, the same couple is seen in a specialist early-phase trial center, where they learn about a dose-escalation trial. This module also provides opportunities for participants to consider the communication difficulties when seriously ill patients fail the necessary screening tests. The final module illustrates some of the issues that may arise when attempting to discuss phase II trials in a balanced manner, enabling realistic goal setting and expectations. The scenario shows a non-small cell lung cancer patient and his wife.

During the workshop, participants were encouraged to consider how they: structured early-phase trial discussions with patients; gave prognostic information and checked patients' understanding of this; discussed symptomatic, palliative care; described the aims of the experimental trial; and mentioned the risk for unknown side effects and the extra effort and burdens involved with trial participation. At the end of the workshop, participants generated a list of optimal and necessary points that should be covered when talking to patients about early-phase trials.

Participants

We sent a general letter of invitation to managers of phase I cancer trial units in the U.K., inviting HCPs to participate in the workshop: 50 enrolled in the study and 47 (12 men and 35 women) participated. Seventeen were oncologists, 29 were research nurses, and one was a trial data manager. All participants specialized in the recruitment of patients to early-phase trials.

Audio-Taped Interviews

Before attending the workshop, participants nominated a phase I or phase II trial that they were about to start or to which they were currently recruiting patients. A wide variety of dose-escalation, monoclonal antibody, and new agent plus standard treatment trials were chosen. Interviews about these trial were audio taped prior to (T1) and after (T2) the workshop.

The patient simulators were all experienced in improvisation as “cancer patients” and were well briefed about their disease, its symptoms, and previous treatments. To enhance authenticity, different actors were used at T1 and T2, but the trial discussed remained constant. We did not conduct an assessment to measure practice effects because in the evaluation of a similar training course about communication in phase III randomized clinical trials, no such effect was found [8].

Assessments

Both objective and subjective measures were used. Because there were no existing standardized assessment tools appropriate for this study, we modified, employing standardized methodology, instruments similar to those developed for use in a previous assessment of an educational intervention for HCPs discussing phase III randomized trials [8]. The phase I study assessment measures all have both face and content validity, being empirically derived from: (a) recordings of interviews with real patients and physicians in clinics and (B) incorporating items regarded as vital for ethical informed consent obtained from good clinical practice (GCP) [9] guidelines and expert scrutiny. Prior to use in this evaluation study, all assessments for the coding of taped interviews and simulated patient perceptions were piloted [5].

Objective Assessments

Audio-Taped Assessment.

All audio-taped interviews were randomly assigned a number, and two senior researchers analyzed the tapes using a checklist (Table 1). The researchers were unaware as to whether or not the discussion they were hearing between an HCP and patient simulator occurred before or after the HCP underwent the educational intervention. Analysis involved checking for the presence of key information areas suggested in the GCP and ethical guidelines [9], for example, the voluntary nature of trials, the right to withdraw, and descriptions of side effects. Other key topic areas, such as checking the patient's understanding of their prognosis, discussing symptom control and palliative care, and describing the main aim of the trial, were also assessed. Each of the two coders performed rate–rerate reliability checks on their own assessments and intercoder reliability checks for 10% of each others' interviews. Both rate–rerate and intercoder reliability were examined using the κ coefficient.

Table 1.

Key content areas and elements used in coder analyses

Patient Simulator Assessments.

Following each taped interview, the patient simulators completed a 15-item questionnaire probing whether or not they had understood the nature of the trial, the risks for unknown side effects, and other key elements specific to early-phase trials. The questionnaire had a Likert-type scale of 0–4 (0, not at all; 1, a little; 2, somewhat; 3, quite a bit; 4, very much).

HCP Subjective Assessment

Prior to the workshops, at T1, participants rated their self-confidence on a study-specific visual analog scale (VAS) from 0 (no confidence) to 10 (very confident) about 15 different aspects of early-phase trial discussion. Participants repeated these assessment scales after the workshop, at T2, and also rated the quality of the educational materials and other aspects of the workshop content.

Hypotheses

Our a priori hypotheses were that, after the workshop: (a) participants' communication skills in the key areas necessary in early-phase clinical trial interviews would improve, that is, their competence would be measurably better, and (b) participants would feel more confident about discussing specified aspects of early-phase trials, that is, their self-confidence/self-efficacy would be enhanced.

Statistical Analyses

Coders rated each information area as present or absent using the checklist. Comparisons were then made between preintervention (T1) and postintervention (T2) assessments for each participant for every item. Possible outcomes were “improved,” that is, absent at T1 but present at T2, the “same,” that is, present or absent on both occasions, and “worsened,” that is, present at T1 but missing at T2. Some items were grouped together to cover one area. For example, the “Aims of trial” measure had four elements and scores were in the range of 0–5 when combined with the “checked understanding” measure (Table 1). Possible scores were in the range of 0–3 for prognosis, symptomatic or palliative care, medical benefit, extra effort, and voluntary nature. Scores were in the range of 0–2 for the areas “unknown side effects” and “right to withdraw.” The patient simulator assessment questionnaire scores were in the range of 1–3, with 1 for “not at all” or “a little,” 2 for “somewhat,” and 3 for “quite a bit” or “very much.” The 0–10 VAS scores from the participants' self-confidence questionnaire were used directly.

We conducted conditional logistic regression models to compare the two scores observed at T1 and T2 for each participant. Given two scores, x₁ and x₂, observed before and after the course for an individual, we modeled the probability that x₂ is observed after the intervention rather than before. This is an example of a matched study. Conditional logistic regression provides a robust method to analyze matched studies because it does not depend on the stratum parameters that define the matched sets [10]. For an individual, the probability that x₂ is observed after the intervention rather than before can be expressed in terms of the difference between the two scores as exp[B(x₂ − x₁)]/{1 + exp[B(x₂ − x₁)]}. The interpretation of the parameter B is that, for two observations from a score that differ by one unit, exp(B) is the relative odds of the larger observation being taken after the intervention versus being taken before. Hence, the larger the odds ratio (OR) exp(B), the more likely it is that higher scores are observed after the course. No distributional assumptions are required for the scores because they are entered as explanatory variables into the conditional logistic regressions. Thus, this approach provides a robust method for before-and-after comparisons. The key data used by the estimation procedure are the numbers of participants with different scores at the two time points. Large positive ORs occur when shifts occur toward larger score values rather than smaller values.

Results

Objective Analysis: The Audio Tapes

Examination of inter-rater (two coders) and rate–rerate reliability (for 10% of interviews) showed good agreement (κ = 0.867; SE, 0.041; κ = 0.707; SE, 0·058; κ = 0.907; SE, 0·034, respectively).

A summary of the numbers and percentages of participants who improved, stayed the same, or worsened at T2 for each of the elements of the audio-taped data can be seen in Table 2. The estimated ORs of improved scores after the training intervention are shown in Table 3. More participants (30 versus 5) discussed or checked the understanding of prognostic issues after the workshop whereas 12 remained unchanged from the baseline score. The odds of observing improvement in this area was 2.7 times higher at T2 than at T1 (p = .002). Similarly, participants were more likely at T2 than at T1 (30 versus 6) to discuss symptomatic and palliative care options alongside descriptions of the trial (OR, 3.81; p < .001). Discussions about the aims of the trial (25 versus 6) were greater at T2 (OR, 2.57; p = .002). Likewise, 16 participants explained the probable lack of medical benefits, compared with 5 at T1, whereas eight remained at their baseline performance (OR, 3; p = .021). There was an improvement of borderline significance in the number of participants discussing the voluntary nature of the trial (22 versus 10; OR, 1.59; p = .053) and also a greater tendency to explain the likelihood of unknown side effects (22 versus 13; OR, 1.72; p = .058), and some evidence of improvement in the area of describing the extra effort involved with trial participation (19 versus 9; OR, 1.63; p = .096). We found no demonstrable effects of the workshop on mentioning the right to withdraw.

Table 2.

Summary of changes from baseline (T1) derived from coded audio-taped interviews (n = 47)

Table 3.

Odds ratio (OR), 95% confidence interval (CI), and p-value of improved scores after the training course for the presence of key information in participants' audio-taped interviews

Simulated Patient Ratings

Table 4 show the ORs associated with the simulated patient responses. After the intervention, participants were perceived as significantly more likely to explain that trial entry was voluntary (OR, 3.67; p = .015), give opportunities to ask questions (OR, 2.88; p = .044), discuss symptom control (OR, 2.11; p = .004), and permit time for consideration of trial entry (OR, 6.11; p = .009). There were no demonstrable changes in other areas, including: “The health care professional used clear language,” “I understood I could leave the trial at any time,” “I understood the aims of the trial,” “The health care professional was sensitive to my concerns,” “I was not left confused,” “I felt listened to,” and “I trusted the health care professional” because they were already present at T1, but these positive ratings were maintained after the workshop.

Table 4.

OR, 95% CI, and p-value of improved scores after the training course for patient simulator assessments of audio-taped interviews

^aThese variables were reversed so that an OR >1 denotes improvement.

Abbreviations: CI, confidence interval; HCP, health care professional; OR, odds ratio.

Subjective Assessments

HCP Self-Rated Confidence Levels

HCPs' ratings of self-confidence significantly improved after the workshop in all 15 areas (p ≤ .001) (Fig. 1). Topics included discussion of Phase I and phase II trials in general, explaining the unlikelihood of personal medical benefit, describing palliative care options, and a discussion of prognosis. The participants rated the workshop highly in all nine areas, with a lowest mean score of 8.8 (SD, 1.1) for PowerPoint presentations and a highest mean score of 9.7 (SD, 0.8) for the quality of the DVDs, informativeness of the workshop, usefulness of group discussions, and interest.

Figure 1.

Odds ratios (95% CI) of improved scores post training for participants' self-rated confidence.

Discussion

Early-phase trials are pivotal in the lengthy developmental process of bringing potentially exciting novel drugs from bench to bedside. Communicating well about these types of trials with patients who often have exhausted all conventional cancer treatments is challenging. Early-phase recruiters may face a more difficult task if the referring clinician, with whom the patient has an established relationship, has not already had a thorough and sensitive discussion about the prognosis, the limited proven treatment options, and the importance of good symptomatic and palliative care. In these cases, patients and their relatives may arrive at the specialist phase I center unprepared or with unrealistic expectations about the therapeutic intent, confusing research aims with treatment for personal benefit, thus increasing the communication problems for all parties involved [1]. A recent U.S. study examined the communication strategies used by oncologists when discussing phase I cancer trials and highlighted the complexity of discussing such trials and the need to develop strategies to aid oncologists and patients with these difficult conversations [11].

Our educational intervention was aimed at helping HCPs learn how to provide clear information about early-phase cancer trials in order for patients to make an educated decision about whether or not to participate. The choice of assessment elements were derived from results of real-life phase I clinic interviews [5]. The results show both objective and subjective improvements in participants' information giving, specifically in the discussion of the prognosis, addressing symptomatic and palliative care options, describing the aims of the trial, and explaining the low chance of patients obtaining any personal medical benefit.

The participants attending the workshops were all experienced early-phase trialists, yet they still demonstrated omissions in their interviews that were similar to those found in clinical settings [5]. Although the majority showed measurable improvements in key areas after the workshop, some omitted discussing the right to withdraw. The most likely explanation for this is that participants ran out of time. They were given only 20 minutes to conduct interviews, which was approximately the median length of phase I consultations we found in real clinical settings [5]. Furthermore, scrutiny of the interviews showed that the majority of participants had completely restructured the ordering of their interviews after the course. More time was spent on issues such as establishing whether or not the patient was aware of their prognosis, whether or not their current symptoms were under control, and whether or not patients understood the aims of the trial and the unlikelihood of personal medical benefit from participation. At the conclusion of each workshop, participants all generated a list of optimal and necessary issues to include in their phase I interviews. These topics were discussed and ordered as shown in the key content areas and elements shown in Table 1. It is quite likely that participants then used this as a template for structuring their own interviews, and because the right to withdraw was last on the list, it was omitted when time ran out.

Other positive findings from the course include an increase in participants' subjective feelings of confidence on a whole range of different aspects of trial recruitment, and objective analyses revealed noticeable behavioral changes in the style and content of trial discussions. Some researchers have suggested that actual behaviors changed during a course might not be transferred into clinical practice without support [12] or further consolidation courses [13]. However, there is strong evidence to show that, if both competence and self-confidence are improved, then many areas of communication do transfer successfully into the clinical setting and endure [14, 15].

Some may question the use of patient simulators in both the DVDs and for the evaluation because they do not face the same life-threatening conditions as real phase I patients. The participants rated the scenarios depicted in the DVDs as authentic and typical of patients they themselves had seen. As far as the use of simulators in the evaluation interviews is concerned, the primary focus of the educational initiative was to enable effective communication of the informational elements crucial to informed decision making and consent. A recently published randomized trial comparing patient simulators with real patients reported that, although the former may not always be entirely authentic, they are better informed about the intended tasks of the interview and consequently are able to provide more specific feedback and evaluation for learners [16].

Following on from earlier successful communication training interventions, a prominent educational feature of the modular program reported herein is the integration of different activities designed to create simultaneous rather than sequential skills development, knowledge acquisition, and awareness of how these affect both the patient and the HCP. The use of this model of communication training, although structured, does allow participants to focus on their own perceived areas of difficulty and makes the course work pertinent to their needs. The early-phase trial intervention reported here was valued by all participants, who rated the five modules together with the exercises, presentations, and group discussion as highly useful, interesting, informative, and enjoyable.

Further research is now required to determine whether or not the improvements in skills shown with our intervention transfer into the clinic setting and endure. Importantly, the benefits observed in our study for experienced trialists might be more enhanced if the workshops were offered to new, less experienced HCPs, and facilitators are currently being trained in the U.K. to roll out the program targeted at this group.

Referring oncologists not directly involved with early-phase trials might also find some of the more generic components of good communication highlighted in the DVDs helpful in their own practice, especially the sections on discussing prognosis and setting realistic expectations. Furthermore, improving referring oncologists' discussions about these issues might then help the subsequent interviews that patients have with physicians at specialist centers.

Finally, more extensive evaluation of real rather than simulated patient outcomes is warranted.

Improving recruitment to all phases of clinical trials is a vital part of establishing safety and efficacy and therefore speeding up the introduction of new treatments into clinics. Arguably, early-phase trials present particular communication challenges that need addressing. We have prima facie evidence that this short, intensive educational workshop changed communication skills and self-efficacy for the majority of the HCPs involved in ways likely to promote better understanding and valid, ethically informed consent from patients contemplating early-phase trial entry.

Author Contributions

Conception/Design: Valerie A. Jenkins, Lesley J. Fallowfield

Collection and/or assembly of data: Valerie A. Jenkins

Data analysis and interpretation: Valerie A. Jenkins, Ivonne Solis-Trapala, Lesley J. Fallowfield

Manuscript writing: Valerie A. Jenkins, Ivonne Solis-Trapala, Lesley J. Fallowfield

Final approval of manuscript: Valerie A. Jenkins, Ivonne Solis-Trapala, Lesley J. Fallowfield