Improvement initiative to develop and implement a tool for detecting drug-drug interactions during oncology clinical trial enrollment eligibility screening

Abstract

Objectives:

Screening subjects for drug-drug interactions (DDI) prior to enrollment in oncology clinical trials is integral to ensuring safety, but standard procedures or tools are not readily available to screen DDI in this setting. Our objective was to develop a DDI screening tool for use during oncology clinical trial enrollment and to test usability in single-center and multi-center pilot studies.

Methods:

A multi-stage approach was utilized for this quality improvement intervention. Semi-structured interviews with individuals responsible for DDI screening were conducted to develop a prototype tool. The tool was used for screening DDI in subjects enrolling in National Clinical Trials Network trials of commercially available agents during a single-center 3-month feasibility pilot. Improvements were made, and a 3-month multi-center pilot was conducted at volunteer SWOG Cancer Research Network sites. Participants were surveyed to determine tool usability and efficiency.

Results:

A screening tool was developed from semi-structured interviews. A critical feature was reporting which medications had specific pharmacokinetic and pharmacodynamic characteristics including transporter and CYP450 substrates, inhibitors, or inducers and QT prolongation. In the 12-site study, average DDI screening time for each patient decreased 15.7 minutes (SD: 10.2, range 3–35 minutes, p<0.001). Users reported the tool highly usable, with >90% agreeing with all positive usability characterizations and disagreeing with all negative complexity characterizations.

Conclusions:

A DDI screening tool for oncology clinical trial enrollment was created and its usability confirmed. Pending further testing, this tool could be used during eligibility screening to improve safety and data accuracy collected within clinical trials.

Introduction

Drug-drug interactions (DDI) can directly impact patient outcomes by decreasing drug efficacy or increasing drug toxicity(1). Toxicity risk in patients with cancer increases with the number of DDI(2) and up to 40% of patients receiving treatment have at least one DDI(3–6), confirming a need to improve DDI identification and management. DDI screening aided by DDI tools embedded within the electronic medical record (EMR) is a standard process within clinical care(7). However, DDI screening for subjects enrolling in multi-center oncology clinical trials, where DDI can affect both subject safety and the validity of the trial results, does not always proceed through this same process(8, 9). Study medications are not always entered into the EMR and DDI information for unapproved investigational agents is not included within tools embedded in the EMR. In cases where the EMR is used for screening across National Cancer Institute’s National Clinical Trials Network (NCTN) sites, different EMR systems are utilized, which could provide inconsistent DDI information(10). This is further complicated by the increasing complexity of oncology clinical trials, such as the umbrella trial protocols that require screening for many potential study medications concurrently(11).

In a survey of sites that enroll subjects on SWOG (formerly Southwest Oncology Group) Cancer Research Network trials, around half of the respondents indicated that only DDI that are explicit exclusion criteria are screened for, and that screening was conducted by clinical research coordinators or research nurses(9). These individuals might not have DDI screening expertise(12), which could lead to high prevalence of DDI in subjects enrolled on trials. In recent studies, approximately 25% of subjects enrolling in NCTN trials at a single institution(13) and approximately 30% of subjects enrolled in two multi-site SWOG clinical studies had a major or contraindicated DDI(14) placing these subjects and trial results at risk. Furthermore, in the multi-site study, approximately 10% of subjects were enrolled with DDI that were explicit exclusion criterion(14) indicating current DDI screening practices are not effective. The high prevalence of DDI suggests a critical need to establish a standard of care process for detecting DDI in subjects enrolling in multi-site oncology clinical trials.

There have been recent proposals for comprehensive pharmacist-led DDI screening for all potential oncology trial subjects (8). This recommendation is not currently feasible considering 3%−6% of SWOG sites do not have access to a pharmacist, and currently only 23% of sites report always using a pharmacist for DDI screening(9). Consequently, a tool that aids clinical research staff with DDI screening could be beneficial to improve patient safety and trial data validity. A tool optimized for clinical trial enrollment screening would use language familiar to clinical research staff and could screen only for interactions involving the trial drug. However, DDI screening tools designed specifically for clinical trial enrollment are not readily available(15). The purpose of this quality improvement study was to utilize a multi-stage approach to develop a DDI screening tool specifically designed for multi-site oncology clinical trial enrollment screening and determine usability of the tool at diverse sites within an NCTN group. The study will be reported using the Standards for QUality Improvement Reporting Excellence (SQUIRE) guidelines (16).

Methods

Context

The quality improvement study was created within the SWOG Pharmaceutical Sciences Committee, a SWOG Research Support Committees created to provide special expertise for the clinical trial network. The initial pilot study was conducted at a single academic medical center, the University of Michigan Rogel Cancer Center, to develop the initial drug-drug interaction screening tool. At the Rogel Cancer Center, the tool was used by data managers who screen subjects for eligibility to enroll in NCTN trials. The study was then expanded to a multi-center pilot. Sites that enroll patients onto SWOG trials were included based on their voluntary commitment on a first-come first-served basis to participate in the study. The tool was used by individuals responsible for trial eligibility screening at their respective sites.

Intervention

A multi-stage approach(17) was used to develop and test the screening tool for usability (Figure 1)(18). Building on the results from a SWOG survey(9), a pre-implementation assessment of DDI screening procedures and the implementation environment at the Rogel Cancer Center was conducted to inform the implementation strategy (Figure 1, Stage 1). Based on the workflow and needs described by key stakeholders, a first-generation prototype tool was developed by ©PEPID LLC (Figure 1, Stage 2)(19). Prior to the three-month trial of the tool, data managers who screen subjects for enrollment onto NCTN trials at the Rogel Cancer Center provided feedback informing the development of the second-generation prototype tool.

Figure 1. Multi-stage approach to develop and trial a drug-drug interaction screening tool.

Quantitative data was collected prior to the study to justify the development and use of a drug-drug interaction screening tool. Stages 1, 2, and 3a focused on qualitative data to develop and trial the tool. Stage 3b included both qualitative and quantitative data to inform future studies.

Study of Intervention

A three-month trial of the tool was conducted with two data managers who screen subjects for enrollment onto NCTN trials at the Rogel Cancer Center (Figure 1, Stage 3a). Data managers were provided a 4-minute training video and an instructional handout. Online access and log-in information for the tool were also provided. Data managers then used the tool to screen DDI for potential subjects undergoing eligibility assessment for any NCTN trials of commercially available agents. Study team contact information was provided if questions arose about the training materials.

Next, a three-month multi-center pilot study was conducted at twelve volunteer SWOG sites, which included thirteen individuals (Figure 1, Stage 3b). All individuals responsible for screening DDI were provided a link to the 4-minute training video, online access and log-in information. Individuals were instructed to utilize the tool for three months to screen potential subjects for enrollment onto SWOG trials.

Measures and Analysis

Prior to the intervention, data managers who are responsible for eligibility screening for NCTN trials were interviewed to identify workflow procedures for DDI screening, and facilitators and barriers to implementation (Figure 1, Stage 1). A semi-structured interview template was designed utilizing the Consolidated Framework for Implementation Research (CFIR) Interview Guide focusing on the five CFIR domains of implementation research(20). To assess workflow procedures, data managers were provided a prototype image of the DDI screening tool and asked pre-determined questions (Appendix, Survey A). Permission was requested to take notes during interviews, and a thematic analysis was conducted by the research team(21). The first-generation prototype was presented to the NCTN data managers at the Rogel Cancer Center for feedback (Figure 1). They were asked to reflect on their needs and to identify required characteristics for the tool to positively impact workflow. This information was used to develop the second-generation prototype (Figure 2). Informal interviews were conducted with identified key stakeholders including clinicians and data managers to reach a consensus about workflow procedures being used to guide tool development and to increase buy-in for the intervention.

Figure 2. Characteristics and example display of first-generation drug-drug interaction screening tool.

A. Key characteristics determined during semi-structured interviews with end users were utilized to select features to be incorporated into the first-generation drug-drug interaction (DDI) screening tool for oncology clinical trial enrollment. Separate entry of study medication and concomitant medication allows for filtered drug-drug interaction information from ©PEPID, LLC (Phoenix, AZ) to be displayed. B. In the second-generation prototype, a medications characteristics feature was added to identify pharmacokinetic and pharmacodynamic mechanisms of interactions. C. Detailed DDI monographs were added including mechanism of interaction, effect, and references.

At the conclusion of the single-center pilot study (Figure 1, Stage 3a), data managers were contacted for a 15-minute semi-structured phone interview to collect feedback about their experience (Appendix, Survey B). Questions were developed to determine overall experience, efficiency of screening, usability of the PDF report, and suggestions for improvement. Notes were taken during the phone call for thematic analysis(21). After using the tool for three months in the multi-center study, individual participants at the sites were sent a Qualtrics® survey (Provo, UT) (22) to assess usability and obtain open-ended feedback for improvement (Appendix, Survey C). Usability was assessed using the system usability scale(23), which was minimally modified to use the word “awkward” instead of “cumbersome” in question 8 to improve understandability(24). Descriptive statistics were conducted. A paired t-test was conducted to determine if using the tool was associated with a decrease in the time it took to screen patients for DDI. Open-ended feedback was reviewed using a thematic analysis approach(21).

Ethical Considerations

This study was submitted to the University of Michigan Institutional Review Board and was determined to be not regulated.

Results

During the pre-implementation assessment in Stage 1, the workflow subtheme of screening aids indicated that screening was based primarily on the trial protocol guidance (Table 1). Specifically, they consistently noted that protocol guidance generally suggests avoiding concurrent use of medications with certain pharmacokinetic (i.e. transporter or CYP450 substrates, inhibitors, inducers) or pharmacodynamic (i.e. QT prolongation) characteristics. In some instances, the protocol appendices include lists of medications to avoid. Explaining this process, one data manager conveyed:

If the protocol, points out to a potential interaction in the inclusion and exclusion criteria, and say if the patient has these drugs or pathways, they are excluded, then I screen that. Data manager, Stage 1.

Table 1.

Themes and Subthemes Identified in Stage 1

Theme	Subtheme	Description	Summary
Workflow	Screening aids	The resources used to screen subjects for drug-drug interactions during clinical trial enrollment	• Primarily use protocol guidance and appendices • Using a variety of web-based resources
	Communication	The way in which participants communicated information	• E-mail drug-drug interactions identified to treating physician • If using clinical pharmacists to review DDI reports, need to communicate interpretation within hour
	Efficiency	The ability or perception the tool has to reduce the time it takes to screen patients for DDI	• Tool could decrease time it takes to screen patients • Tool could decrease time it takes for physician to review information
	Implementation climate	The absorptive capacity for change, shared receptivity of involved individuals to an intervention, and the extent to which use of that intervention will be rewarded, supported, and expected within their organization.(28)	• Can see tool fitting into workflow • Willing to try using tool • Think tool would be helpful for screening investigational agents
	Tension for change	The degree to which stakeholders perceive the current situation as intolerable or needing change.(28)	• Felt current methods out of date and/or incomplete • Perceived need to standardize screening to improve patient safety
Tool function	Desired feature	A characteristic the participant wanted added to the tool.	• Ability to e-mail the tool’s results to physicians as a PDF • Ability to screen for herbal supplements • Incorporation of investigational agents • Avoid seeing duplicate DDI • Contact information to clarify training instructions • Pre-populate the study medications based on trial number

DDI = drug-drug interactions

Data managers explained that when screening for a DDI based on a pharmacokinetic or pharmacodynamic characteristic, concomitant medications were searched for these properties using a variety of resources including protocol appendices, Google searches, CredibleMeds®(25) (©AZCERT, Inc., Oro Valley, AZ), Lexicomp®(26) (©Wolters Kluwer Clinical Drug Information, Inc., Hudson, OH), and the Drug Interactions Flockhart Table™(27) (Bloomington, IN). Data managers used protocol appendices, when available, for screening but use of additional resources varied among data managers. Another important workflow theme identified was that potential DDI identified during screening were e-mailed to the treating physician to determine appropriate action.

The final key themes identified focused around efficiency, willingness to change, and communication. Specifically, data managers requested a simple process to screen for specific pharmacokinetic and pharmacodynamic characteristics that cause DDI with only the study drug and to have a way to easily communicate this information to the physicians. The following statement from a data manager exemplifies the need for an efficient process for both the data managers and the physicians reviewing the reports:

I can tell you if I am entering 20 medications and then I get 50 alerts, I am not going to look at that and the physician is definitely not going to look at that. Data manager, Stage 1.

The implementation climate and tension for change(28) indicated receptiveness to the tool as evidenced by the possible positive impacts to workflow efficiency and patient safety:

If it’s going to make my life easier, if it’s going to make the physician happier, if it’s going to make the patient safer, I am all for it. Data manager, Stage 1.

Additionally, data managers provided feedback for the first-generation prototype (Figure 2A). Data managers were concerned that they would have to interpret the mechanism of DDI presented in the first-generation prototype and subsequently determine which DDI should be sent to the physician, for which they felt they lacked expertise. Consistently, they requested a process to select pharmacokinetic and pharmacodynamic characteristics that align with protocol guidance for DDI screening.

The themes from Stage 1 were taken into consideration when developing the second-generation prototype during Stage 2 (Figure 2B). Similar to currently available DDI screening tools, the tool identified interacting pairs of medication, explanation of the mechanism of the DDI, and provided a severity level. Severity levels displayed were “no interactions found”, “1 – minor”, “2 – moderate”, “3 – serious”, “4 – major”, and “5 - life threatening”. Data managers placed emphasis on screening for DDI that involved the study agent to improve efficiency, therefore, a feature of the prototype was to allow for entry of trial medications separate from the subject’s concomitant medications. The DDI panel display could be filtered to report all DDI, only DDI involving the study agent, or only DDI affecting the study agent. The severity thresholds could be set to display only DDI exceeding a particular severity level (i.e., 2+ or 3+). Additional information could be accessed about the interactions displaying the mechanism of interaction, effect of the interaction, recommended action to manage the interaction and the source of that recommendation, and literature references.

To meet their needs of screening for specific pharmacokinetic and pharmacodynamic drug characteristics, a separate panel was added to the tool. The tool incorporated a feature that automatically generates a PDF report with all settings (i.e. severity threshold) and identified DDI to facilitate communication with the treating physician. One month into the multi-center pilot to further enhance the tool, DDI monographs were added to the tool providing detailed information about the mechanism of the interaction, effect, level of concern, action, source of the recommendation, and references (Figure 2C).

Two data managers used the DDI screening tool for three months in Stage 3a during which approximately five subjects were screened. The thematic analysis outlining the desired features of the tool, useful features of the tool, and improvement needs are included in Table 2. A strength of the tool was that it increased DDI screening efficiency; one data manager estimated the tool reduced DDI screening time for a single potential subject enrolling in an umbrella trial protocol from one hour to five to ten minutes. This informed the decision to include a quantitative question to measure efficiency change in the multi-center study. Overall, data managers felt it was worthwhile to expand the tool to multi-center use.

Table 2.

Themes and Subthemes Identified in Stages 3a and 3b

Theme	Subtheme	Description	Stage	Summary
Workflow	Efficiency	The ability or perception the tool has to reduce the time it takes to screen patients for DDI	3a	• Easy to use tool • Fast to enter in medications to tool
			3b	• Easy to use tool • Fast to enter in medications to tool • Saves time during screening
Tool function	Desired feature	A characteristic the participant wanted added to the tool.	3a	• Incorporating more drug characteristics including pharmacodynamic interactions into filters • Separate the moderate and strong DDI • Adding in additional herbal supplements • Ability to prioritize DDI based on severity
			3b	• Incorporating more drug characteristics into filters • Adding in additional herbal supplements • Automatically populate study medications when study number entered • Adding in ability to screen if drug on medication list are cross reactive with patient allergy • Adding investigational agents to tool
	Useful feature	A characteristic of the tool the participants found beneficial.	3a	• PDF report was helpful way to send information • Great at screening for CYP450 strong inducers and inhibitors
			3b	• PDF report was helpful way to send information • Provides documentation for eligibility screening
	Improvement needs	A characteristic of the tool that the participants felt needed improvement.	3a	• PDF would be more useful if the most important DDI were displayed first
			3b	• Severity defaulted to 3+, which excluded some interactions that were expected • “Med List” and “Results” display overlapped on smaller display screens

DDI = drug-drug interactions

The Stage 3b multi-center survey participants (n=13) represented five different SWOG site types (Table 3). Qualitative findings are included in Table 2. Matched comparison of individuals who provided time to screen both with and without the tool (n=13) indicates the tool decreases time to screen a patient by 15.7 minutes (SD: 10.2, range 3–35 minutes, p<0.001, Table 3). Individuals reported that a DDI was detected by the tool in 28.2% of patients, with only 6.7% of patients requiring a treatment modification before enrollment and 3.7% of patients requiring trial exclusion due to a DDI.

Table 3.

Patients Screened for Drug-Drug Interactions Using Tool During 3-month Multi-site Pilot

		N (%)
Respondent site type	NCI Community Oncology Research Program	6 (46.2%)
	Academic teaching hospital	2 (15.4%)
	Community cancer center	2 (15.4%)
	Private practice office	2 (15.4%)
	Non-academic hospital	1 (7.7%)
		Mean	Standard deviation	Range
DDI Screening Effectiveness	Number of patients screened	9.15	7.43	3–25
	Number of patients with detected DDI	2.08	1.73	0–7
	Number of patients with detected DDI requiring treatment modification	0.54	0.75	0–2
	Number of patients with detected DDI requiring trial exclusion	0.23	0.58	0–2
DDI Screening Efficiency	Time to screen with tool (mins)	6.92	3.17	3–12
	Time to screen without tool (mins)	22.62	7.80	10–40
	Screening time difference (mins)	15.7*	10.2	3–35

^*statistically significant difference (p<0.001)

The usability survey included five positive (usability) and five negative (complexity) questions. Overall, individuals agreed or strongly agreed that they would use the tool frequently (100%), the tool was easy to use (100%), the system could be learned quickly (100%), the functions were well integrated (100%), and they felt confident using the tool (92.3%, Figure 3). Individuals disagreed or strongly disagreed that the tool was complex (92.3%), a technical person would be needed to use the tool (92.3%), there was inconsistency in the tool (92.3%), the tool was awkward to use (100%), and they would need to learn a lot to use the tool (92.3%). The findings of the usability survey are exemplified by one participant writing:

I love this system! It saves us time and provides documentation for eligibility. Participant, Stage 3b.

Figure 3. Perceived usefulness of the drug-drug interaction screening tool.

Survey results (n=13) after the multi-center pilot study were overall supportive of the usability of the tool.

Suggestions for improvement were included in the open-ended feedback, such as adding the ability to screen subject medication allergies with the tool, adding additional dietary supplements, and adding additional drug characteristics as filters. These suggestions will be explored for future studies of the tool.

Discussion

DDI are systematically screened for patients with cancer as a standard element of clinical care using the EMR(7) and various external tools are also available to screen patients for DDI (29). DDI clinical decision support tools are mandated for use within the EMR by the Centers for Medicare and Medicaid Services(7) and recommended for use during chemotherapy preparation by American Society of Health-System Pharmacy to reduce medical errors(30). However, screening processes are not standardized across sites(9), leading to high prevalence of DDI in trial subjects(13, 31). Standardizing DDI screening procedures and utilizing technology to facilitate the process has the potential to improve treatment outcomes and accuracy of clinical trial results.

Utilizing a single DDI screening tool, such as that developed in our study, across sites enrolling into a multi-site clinical trial could standardize DDI detection and allow for comprehensive DDI screening, improving patient safety and trial data accuracy. The tool detected DDI in 28.2% of subjects screened prior to enrollment during the 3-month multi-center pilot, which is consistent with rates of DDI detected in oncology trial subjects(13, 14). DDI identified led to a medication change in some patients (6.7%), while a small number of patients (3.7%) had to be excluded from enrollment to ensure patient safety and trial data validity.

Ideally, this tool would not increase workload during trial eligibility assessment. At sites that regularly conduct DDI screening, often without access to DDI screening resources, our data indicate that implementing this tool will increase efficiency of DDI screening and reduce staff workload. The tool shortened the time to screen DDI for a patient for DDI by about 75% (15.7 minutes), which could lead to a meaningful reduction in workload. At sites that only screen for DDI that are explicit exclusion criteria(9), implementing systematic DDI screening using our tool may somewhat increase workload, but this is justified given the high prevalence of DDI in subjects enrolled on NCTN studies(13, 14) and the potential harm of enrolling subjects with serious DDI onto trials. The tool has several features to ensure implementation is as efficient as possible, including the ability to generate a PDF export that can be e-mailed or printed for sharing or documentation. Future generations are anticipated to automatically import subject’s medications from the EMR, which will reduce documentation errors and further increase the efficiency of using the tool.

Anti-cancer agents can have both pharmacokinetic and pharmacodynamic interactions(1). Anti-cancer therapy is shifting from intravenous to oral therapy(32), which increases the potential for absorption related pharmacokinetic interactions. Additionally, pharmacokinetic interactions are often due to inhibition or induction of cytochrome P450 (CYP450)(13, 31). Our survey results indicated that the tool is a great resource for screening CYP450 interactions. However, not all pharmacodynamic interactions were included as selectable medication characteristics in the tool (i.e., sedation or antiarrhythmic potential). Feedback also requested that additional herbal supplements be incorporated into the tool for screening and identification. Many protocols prohibit the use of herbal supplements due to concern for DDI, supporting the importance of identifying and managing drug-herbal interactions(8). We are working to integrate additional pharmacodynamic interactions and herbal supplements into future generations of our tool.

Clinical trials of investigational agents are particularly challenging to screen for DDI since investigational agents are not built into the EMR or included in currently available DDI screening tools. We plan to integrate a feature into future generations that allows DDI information to be pre-built into the tool for investigational agents. Integrating investigational agents into the tool could address a major gap in the functionality of currently available tools and further improve the usefulness of this novel tool for screening DDI for potential oncology clinical trial subjects.

A limitation to this study was that all interviews were not recorded. While detailed notes were taken during the interviews, it is possible important themes were missed. The sample size for single-center pilot study and multi-center study were small, and it is possible the tool was not accurately characterized. However, despite the sample size, a variety of SWOG site types were represented in the multi-center pilot study (Table 3). We are planning a larger multi-center prospective randomized trial within an NCTN group to assess the effectiveness (i.e., prevalence of DDI in enrolled subjects) and efficiency (i.e., time required to screen potential subjects) of DDI screening using our novel tool. Successful demonstration that the intervention increases the effectiveness of DDI screening without an unacceptable increase in workload would justify expanding its use within and beyond this NCTN group, potentially enhancing treatment outcomes for clinical trial subjects and ensuring clinical trial data integrity.

Conclusion

A DDI screening tool was developed to use during oncology clinical trial enrollment, and implementation was found to be feasible and useful in a multi-center pilot study. Implementation of a DDI screening tool for oncology clinical trial enrollment has the potential to standardize DDI screening for subjects and increase screening efficiency and effectiveness. Future generations of this tool could be expanded across NCTN and industry studies, which could positively impact safety for many oncology trial subjects.