The impact of an online training program about cancer clinical trials on primary care physicians’ knowledge, attitudes and beliefs, and behavior

Abstract

Introduction:

Participation in cancer clinical trials (CCTs) is critical to improving cancer treatments and quality of care. However, rates of patient participation remain low. Research has shown that a trusted physician recommendation is an important influence on patients’ decisions to enroll in a CCT. Improving primary care providers’ (PCPs’) knowledge, attitudes, and beliefs about CCTs is a promising potential path for improving CCT participation. The aim of this pilot study was to test the effect of an online educational course for PCPs about clinical trials on primary care providers’ knowledge, attitudes and beliefs, and behavior.

Methods:

Forty-one PCPs in the New York City area participated in a 1-hour online training session on cancer clinical trials. These PCPs had self-selected to complete the training in a previous survey. The objectives of the training module were to (1) educate the PCPs about clinical trials, with a focus on overcoming misconceptions; and (2) discuss roles of PCPs in partnering with oncologists to help patients gain access to clinical trials. The training module included didactics, audio excerpts and case descriptions. Participants completed a pre-test immediately before taking the course, a post-test immediately after taking the course, and a 3-month post-course survey. All three assessments included: a general T/F knowledge test, a 7-item attitude/belief scale, and a knowledge test focused specifically on local resources and access for clinical trials.

Results:

Forty-one PCPs completed the module and the pre-post course surveys. Eighty percent (33/41) also completed the 3-month post-course survey. General knowledge and local knowledge increased significantly (p<.05) from pre- to post-course. At 3 months post-training, both general and local knowledge scores remained significantly increased from baseline. For those who completed the 3-month post-course survey, attitudes and beliefs increased significantly from pre- to post-course, but this change was not sustained at 3 months post-training. At 3 months post-training, 52% of the PCPs who had an interaction with a recently diagnosed cancer patient reported speaking with patients about CCTs as a result of the training.

Conclusions:

A brief online course showed significant and sustained improvement in PCPs’ general and local knowledge about cancer clinical trials, which translated into self-reported behavior change. Future dissemination of the course and further research into its impact are important next steps.

Introduction

Primary care providers (PCPs) often play an important role for patients diagnosed with and treated for cancer. As a first point in the referral process, PCPs often provide initial referral to, and later insight about, potential cancer treatments. Indeed, PCPs are sometimes tasked with giving patients their cancer diagnosis [1], and are frequently involved in the referral process. In a national survey of 1,694 primary care physicians caring for cancer patients, conducted by the Cancer Care Outcomes Research and Surveillance Consortium, Klabunde and colleagues found that 64.2% of the PCPs reported being involved in assessing patient preferences for treatment, 28.2% were engaged in helping to determine the first treatment modality, and 19.3% reported discussing possible participation in clinical trials [2]. There has been a call for PCPs to play an even more central role around during their patients’ cancer care due to the trust placed in PCPs [3].

One potential way in which PCPs can be centrally involved in their patients’ cancer care is by preparing patients for the referral to a specialist and introducing key concepts related to cancer care, such as clinical trials. Patients’ access to CCTs is often cited as a measure for delivery of quality cancer care: An Institute of Medicine (IOM) committee stated that the “therapies offered through CCTs should ideally be considered the preferred treatment choice for physicians and patients, if they are available” and recommended that all oncologists should work toward achieving high accrual rates [4]. The Commission on Cancer has increased the expected percentages of accrual to clinical trials for cancer treatment entities seeking its accreditation for quality [5].

Lack of awareness around clinical trials clearly affects patient participation [6, 7]; one way this can be addressed is by PCPs preparing patients for the referral to a specialist. One major barrier to trial accrual is that a significant percentage of eligible patients are never approached by their treating healthcare provider about the opportunity to participate in a trial. A review by the American Cancer Society Cancer Action Network found that among institutions that offer clinical trials to patients, anywhere between 17% to 76% of those eligible patients are not offered the opportunity to participate in a clinical trial [8]. Contributing to this are the underlying attitudes and beliefs of treating specialists who play a critical role in whether or not eligible patients are offered a clinical trial. Studies have shown that: a) assumptions about treatment preferences (i.e., standard care over a clinical trial) [9]; b) varying quality of how clinical trials are introduced to patients in relation to standard of care [10]; c) provider bias [11]; and d) lack of clarity about the role of other providers on the treatment team [12] are all important factors in determining whether or not a patient is approached to participate.

Consequently, as part of the referral process, it is critical to encourage and normalize patient inquiry about CCTs prior to beginning treatment. In order to effectively inquire about clinical trials, patients need to understand that such inquiry is encouraged and supported, that there is an expectation that they will be involved in making decisions about their treatment, and that they are able to take on this role. Several studies have suggested that education before the first oncology visit improves knowledge, attitudes, and preparation for treatment decision making, including the possibility of receiving treatment through a clinical trial [13]. The patient’s PCP has an important role in beginning this education process of helping patients get the information they need to make an informed decision. As part of the referral process, PCPs can play a critical role in helping prepare patients for discussions about treatment, increasing their awareness about the potential to receive treatment through a CCT, and if possible referring them to specialists that offer trials to their patients.

As one of the first to publish work on educating PCPs about clinical trials, a national non profit organization – the Education Network to Advance Cancer Clinical Trials (ENACCT) – developed a continuing education program to address documented knowledge and attitudinal barriers to clinical trial discussions with patients [14, 15]. This face-to-face program reached almost 400 providers in three states and showed that as a result of the program PCPs had increased knowledge and improved attitudes around CCTs. The course was later adapted for use with PCPs in Hawaii, reaching almost 130 providers throughout the state, and showing an increase in understanding impacting CCT myth awareness, and self-perceived ability and willingness to bring up CCT [14, 15].

The present study extends this previous work by implementing a similar intervention in a different geographical setting, using an online (rather than face-to-face) course for the intervention, and measuring behavior change at a farther time point (3-months). Furthermore, this study focused on PCPs who practice in low-income minority communities. Improving the ability of PCPs to speak with their patients about clinical trials hold great promise, and an online intervention is more scalable than a face-to-face program. Thus, the aim of this pilot study was to test the effect of an online educational course for PCPs about clinical trials on primary care providers’ knowledge, attitudes and beliefs, and behavior.

Methods

Participants

Course participants were 41 PCPs in the New York City area. These PCPs had previously participated in a survey study we conducted about PCP knowledge, attitudes and beliefs, and practices about cancer clinical trials [16]. In that survey study, one question asked: “Would you be interested in participating in an educational training, with CME credit, that provides information about cancer clinical trials, and informs you about how primary care providers can enhance patient access to these trials?” Those who responded yes or maybe (n=97) received an email asking them to participate in this pilot educational intervention study. Our descriptive analyses in the survey study [16] showed that Black and Latino physicians tended to be more interested in training than did White and Non-Latino physicians. Participants were able to receive either CME or a $75 incentive for completing the training. The study was approved by the Memorial Sloan Kettering Cancer Center Institutional Review Board.

Intervention

The pilot training module was designed based on the work of ENACCT described previously. The objectives of the one-hour online pilot training module for the present study were to: (1) Clarify common misconceptions among PCPs about cancer clinical trials; (2) Invite PCPs to reflect on their critical role, being the first point of contact for patient access and participation in clinical trials; and (3) Describe practical strategies referring physicians can take to effectively prepare patients for discussions with their oncologist about cancer clinical trials.

The training module used didactics, audio excerpts, and case descriptions. The training module began by establishing cancer incidence and mortality rates in the NYC area, followed by basic education about CCTs, including a definition, types, and facts about participation and eligibility. The majority of the training module focused on six “Myths,” each followed by “Facts” as a way to introduce certain teaching points. For example, Myth 3 was “Clinical trials are only for patients who have run out of other options.” It was followed by several facts to correct that myth.

The emphasis in the latter part of the training module was on improving PCPs’ ability to talk with their patients about CCTs. This was introduced through Myth 6: “Helping my newly diagnosed patients (or patients being referred for a diagnostic work-up) gain access to cancer clinical trials is going to take too much time and effort and should only occur with the oncologist, not me.” We countered this myth by helping PCPs realize that they already played a role helping to manage patients during cancer treatment and gave the PCPs examples of how they might talk to their patients about CCTs. These examples showed how this could be done efficiently. The training ended with three case studies, giving the PCPs the opportunity to consider ideas on how to approach each of three patients.

Evaluation

Our evaluation plan was based on the Kirkpatrick Assessment Model, which measures the success of a course through four hierarchical levels of assessment. In this model, the complexity of behavior change increases with each level: reaction, learning, behavior, and results [17, 18]. Guided by Kirkpatrick’s model, our primary evaluation questions addressed the first three levels of the Kirkpatrick Assessment Model, which are as follows: 1) Reaction: How did the participants react to the course overall? 2) Learning: What knowledge did the participants gain (i.e., information learned), and how do they plan to apply this knowledge to their practice? How did the course information alter participants’ attitudes towards and beliefs about clinical trials? 3) Behavior: How do the participants change their behavior, i.e., act upon their improved knowledge, attitudes, and beliefs?

Participants completed a pre-test immediately before taking the course, a post-test immediately after taking the course, and a 3-month post-course survey. All three assessments included: the Knowledge of Cancer Clinical Trials T/F knowledge test Knowledge of Cancer Clinical Trials measure that we have used in our previous study [16], the 7-item Cancer Clinical Trials Attitudes and Beliefs Survey [16], and a knowledge test focused specifically on local resources and access for clinical trials. The pre-test survey also included collection of professional information and demographic data. The immediate post-test survey included participants’ self-efficacy ratings regarding the course learning objectives, participants’ ratings of aspects of the training material and content, and four open ended questions about their learning, including a question asking the participant to describe any specific changes they planned to implement regarding communicating with patients about cancer treatment clinical trials.

In addition to repeating the T/F and Attitudes/Beliefs items, the 3-month post-course survey asked participants to report about changes to their practice since the training. Specifically, we asked participants if, as a result of the training, they had done either of the following with their patients who had been diagnosed with cancer: (1) spoken with any of these patients about CCTs as a result of the training; (2) encouraged any of these patients to consider participating in cancer treatment trials. We also asked them to what extent the frequency of their discussions with professional colleagues improved based on the training.

Results

Of the 97 PCPs invited to take the online course, 41 (42%) completed the course and the pre- and post-course surveys. Participants were majority female (58.5%), white (53.7%), and practiced at an academic institution (53.7%). Additional participant demographics and practice characteristics can be found in Table 1. Thirty-three participants chose the cash incentive ($75) and eight participants chose the CME credit. Of the 41 course participants, 80% (33/41) also completed the 3-month post-course survey.

Table 1.

Physician Participant Demographics and Practice Characteristics (n=41)

Gender	% (n)
Female	58.5% (24)
Male	41.5% (17)
Age	Mean (SD)
	41.0 (8.2)
Race	% (n)
White (non-Hispanic)	53.7% (22)
Asian/Pacific Islander	29.3% (12)
Black (non-Hispanic)	12.2% (5)
Mixed race	2.4% (1)
Other	2.4% (1)
Years in practice	% (n)
0–5	34.1% (14)
6–10	24.4% (10)
11–15	14.6% (6)
More than 15	26.8% (11)
Practice setting	% (n)
Academic Institution	53.7% (22)
Community Hospital	9.8% (4)
Private Practice	7.3% (3)
Clinic	26.8% (11)
Other	2.4% (1)
Average number of patients seen per week	% (n)
0–39	34.1% (14)
40–79	29.3% (12)
80–159	29.3% (12)
160 or more	2.4% (1)

Level 1: Reaction

Overall, participants had a positive reaction to the course. Immediately following the course, they highly rated seven items of the course related to the training materials and content. On a scale from 1–5 (5=strongly agree), average ratings ranged from 3.78 to 4.29 (SD ranges= .56 to 1.26). On the same 1–5 scale (5=strongly agree), participants’ self-efficacy rating averages ranged from 3.80 to 4.15 (SD ranges = 0.69 to 1.19). Of the 41 providers, 37 indicated they would change something specific based on the course.

Level 2: Learning

We measured learning by evaluating change in pre-and post-scores on knowledge and attitudes/beliefs. Pre-post mean comparisons showed that general knowledge and local knowledge were significantly higher at the immediate post-test than the pre-test (p<.01). For those who completed the 3-month post-course survey, both general and local knowledge were sustained at 3 months (see Table 2).

Table 2.

Knowledge and Attitudes/Belief Scores Changes over Time

	Pre-course Mean (SD)	Post-course Mean (SD)	3-month post Mean (SD)
General Knowledge (maximum 9)
Full sample (n=41)	6.51 (1.27)	7.90 (1.18)*
3-month completers (n=33)	6.39 (1.30)	7.82 (1.23)*	7.12 (1.45)*
Local Knowledge (maximum 15)
Full sample (n=39)	7.79 (2.62)	10.69 (2.73)*
3-month completers (n=31)	8.19 (2.7)	10.87 (2.93)*	10.26 (2.7)*
Attitudes & Beliefs Mean (1 lowest, 5 highest)
Full sample (n=41)	3.95 (.84)	3.79 (.65)
3-month completers (n=33)	3.53 (.50)	3.81 (.70)*	3.72 (.62)

^*differs significantly from pre-course mean, p<.01

We computed average mean scores for the attitude/beliefs scale. For the full sample (n=41), there was no significant change in attitudes/beliefs after completing the course. For the subset that completed the 3-month post-course survey, the post-test scores differed significantly from the pre-test (p<.01), but this was not sustained at 3 months (see Table 2).

Level 3: Behavior

As shown in Table 3, at 3 months post-training, nearly 70% of participants reported that since they completed the training, they had interactions with one or more patients recently diagnosed with cancer. Of those who had interactions, a little more than half (52.2%) said they spoke to patients about CCTs as a result of the training, while 59.1% said that they encouraged patients to consider participating in cancer treatment trials as a result of the training. Of the 33 participants who completed the 3 month post-course survey, about three-fourths reported that as a result of the training, the frequency of their discussions with professional colleagues improved either some or a little.

Table 3.

PCPs Self-Reported Changes to Practice since the Training (3-months post)

	Yes	No
I have had interactions with one or more patients recently diagnosed with cancer within the past three months.	69.7%	30.3%
Of those who reported yes (n=23)
Did you speak with any of these patients about CCTs as a result of the training?	52.2%	47.8%
Did you encourage any of these patients to consider participating in cancer treatment trials as a result of the training?	59.1%	40.9%
	Some	A little	None
As a result of the training, to what extent did the frequency of your discussions with professional colleagues improve?	36.4%	39.4%	24.2%

Discussion

Cancer clinical trials are vital to the advancement of science and medical care. Attaining the full benefit of clinical research requires the participation of informed and willing cancer patients that represent the diversity present in the U.S. However, accrual to cancer clinical trials is low across the U.S. In this study, we tested a novel, brief online intervention for PCPs to educate them about clinical trials and examined the effect of the intervention on PCPs’ knowledge, attitudes and beliefs, and behavior.

We found that the PCPs reacted positively to the online course, the course impacted participants’ knowledge, and the increase in knowledge was sustained over a 3 month period. However, there was little impact on the participants’ attitudes and beliefs. Only the subset who completed the 3 month post-course survey had a significant change in attitudes and belief after taking the course, and this was not sustained. The subset had a lower pre-course mean than the group as a whole, which resulted in the significant pre-post change.

However, the minimal effect of the course on attitudes and beliefs did not seem to impede the impact of the course on PCP behavior change, a more complex evaluation level according to the Kirkpatrick model. Of the PCPs who had the opportunity to do so, the majority reported talking to their patients about cancer clinical trials or encouraging their patients to consider cancer clinical trials as a result of the training. We are very encouraged by this finding.

Overall, our results demonstrate a fairly robust impact from a one-hour online course. It may be that a longer course would have resulted in more change. However, our awareness of the demands on PCPs’ time influenced our decision to use a one-hour course for the pilot training.

Our study is a natural extension of the research literature showing that a trusted physician’s recommendation was the primary factor influencing patients’ decisions to enroll in a clinical trial [19–21]. Indeed, the National Cancer Institute-American Society of Clinical Oncology Cancer Trial Accrual Symposium found that a best practice for increasing cancer clinical trial recruitment was to engage with primary care providers as referral points [6].

Limitations of this study include the small sample of PCPs that self-selected to participate in the training. The self-selection likely indicated either interest or an openness to learning new things that may not be generalizable to all PCPs. Additionally, we relied on PCP self-reports to measure change in behavior based on the course, as it would be very time and resource intensive to objectively monitor all PCP behavior after taking the course. Given a small number of non-completers of the 3-month post-course survey, we were unable to compare them to completers. Furthermore, we had limited time in which to cover all of the pertinent topics related to clinical trials. For example, we were not able to fully address the myth that trials are only conducted at academic medical centers. Finally, although we were able to evaluate the intervention on the Kirkpatrick Model Levels 1–3, we were not able to measure Level 4 – the actual impact of the intervention on patient outcomes (i.e., if patients enrolled in a clinical trial).

Conclusion

This study demonstrated the impact of a one-hour online training on improving the knowledge and behaviors of PCPs about cancer clinical trials. Though the findings are encouraging, there is still a great deal more that needs to be done in this area.

Specifically, the greatest potential for future research in this area is in the area of dissemination, particularly figuring out in what form, how, and to whom to disseminate this course. In addition, future research could compare varying lengths of course and/or delivery methods (online v. in-person) in order to determine the most effective and feasible delivery and to look at more distal time points for outcomes. Further, examination of the impact of the course on trial accrual would be useful.