Evolution of Hematology Clinical Trial Adverse Event Reporting to Improve Care Delivery

Abstract

Purpose of Review

Reporting of adverse events on hematology clinical trials is crucial to understanding the safety of standard treatments and novel agents. However, despite the importance of understanding toxicities, challenges in capturing and reporting accurate adverse event data exist.

Recent Findings

Currently, adverse events are reported manually on most hematology clinical trials. Especially on phase III trials, the highest grade of each adverse event during a reporting period is typically reported. Despite the effort committed to AE reporting, studies have identified underreporting of adverse events on hematologic malignancy clinical trials, which raises concern about the true understanding of safety of treatment that clinicians have in order to guide patients about what to expect during therapy. In order to address these concerns, recent studies have piloted alternative methods for identification of adverse events. These methods include automated extraction of adverse event data from the electronic health record, implementation of trigger or alert tools into the medical record, and analytic tools to evaluate duration of adverse events rather than only the highest adverse event grade.

Summary

Adverse event reporting is a crucial component of clinical trials. Novel tools for identifying and reporting adverse events provide opportunities for honing and refining methods of toxicity capture and improving understanding of toxicities patients experience while enrolled on clinical trials.

Introduction

Therapy for hematologic malignancies can cause significant treatment-related toxicities. Capture of these adverse events (AEs) on clinical trials is crucial for identifying the safety and tolerability of treatment regimens. This is important for ensuring that physicians, patients, and families understand risks of therapies and the data from these trials can be used for medical decision making. AE reporting is therefore mandated on clinical trials [1]. Despite the importance of this key component of clinical trials, there are challenges in capturing AEs in a comprehensive and accurate manner on hematology trials. This article will review the current methods of AE reporting, changes over time, key challenges, and recent studies that provide insight into areas of potential future approaches to improved AE ascertainment.

Adverse Event Ascertainment

On most clinical trials, AEs are identified and reported manually by clinical research associates (CRAs) or research nurses [1, 2]. CRAs and research nurses manually review the medical record for indication of toxicities and report AEs that are identified based on published AE dictionaries. In North America, the National Cancer Institute (NCI) Common Terminology Criteria for Adverse Events (CTCAE) is the most common AE dictionary. On most trials, CRAs and research nurses are instructed to confirm AEs with the site principal investigator (PI) or treating clinician, who also assist with assignment of attribution to the study therapies. Once the AEs are confirmed, the CRA or research nurse manually enters the AEs into the clinical trial electronic data capture system. This system is essentially unchanged from the first clinical trial in childhood leukemia, which was published in 1948 [3].

AE reporting is a labor-intensive process. In addition, it is only one of many tasks that CRAs and research nurses perform [4]. Despite the importance of the data that needs to be collected, Roche et al. identified that CRAs only spend 18 min per day on AE reporting [5]. This may not provide sufficient time to accurately report AEs given the potential volume of AEs on hematologic malignancy trials and the complexity of identifying and grading AEs based on clinical documentation. Furthermore, CRAs and research nurses may have variable background education that can lead to differential understanding of toxicities. In addition, CRAs and research nurses at different hospitals may receive varied levels of training on AE capture. Different protocols often have dissimilar AE reporting requirements, which can be challenging for CRAs and research nurses who follow patients on a range of studies.

Evolution of the CTCAE

The CTCAE is the primary dictionary for AE classification in oncology and malignant hematology clinical trials. In 1983, the NCI published the first version of the CTCAE in order to standardize the way that AEs are reported across oncology clinical trials [1, 6–8]. The CTC has subsequently been updated using multidisciplinary input over time. The initial goal of the first version of the Common Toxicity Criteria (CTC v1.0) was to provide guidance on AE identification and grading for phase I clinical trials; however, the definitions were used in other trials as well [5, 9]. The NCI CTC core committee published an updated version, CTC v2.0, in 1999 that included adverse events related to radiation therapy. In addition, CTC v2.0 added definitions using activities of daily living (ADLs) to delineate between grades for some AEs [8]. This version was expanded further in 2006 with the publication of CTCAE v3.0, which included surgical AEs, late AEs once off of protocol therapy, and pediatric-specific AE criteria for some definitions [10]. Three years later, CTCAE v4.0 was published. This version included global mapping to the Medical Dictionary for Regularly Activities (MedDRA) System Organ Class [11]. MedDRA is not specific to malignancies and is a medical terminology dictionary used internationally for public health monitoring and data management and analysis. The most recent version, CTCAE v5, was published in 2017 and provided additional AEs and clarifications to prior AE definitions [12].

In addition to the CTCAE, which intends to capture AE data based on clinician documentation, the NCI has developed the patient-reported outcomes version of the CTACE (PRO-CTCAE) [13–16]. The PRO-CTCAE aims to capture self-reported data regarding AEs from patients and is intended to be used as a complement to the CTCAE. Clinician-based CTCAE reporting has inherent limitations related to what data are both elucidated by clinicians and comprehensively documented in the medical record [17]. The PRO-CTCAE can be used for capture of subjective AE data and has been found to be reliable and acceptable [16]. Basch et al. reported that of the 790 AEs in CTCAE v4, 78 AEs are suited to patient self-report and the PRO-CTCAE was developed to address ascertainment of those toxicities [14]. As part of its development, the developers of the PRO-CTCAE created a self-report software program that can be used for capture of data from patients either over the internet or telephone. While initially developed in English, there are ongoing efforts to translate the PRO-CTCAE into other languages as well [14]. The pediatric PRO-CTCAE was also recently validated for children who are at least 7 years of age or their caregivers to complete as a proxy report [18]. This tool has been demonstrated to be acceptable and there are ongoing efforts to integrate pediatric patient-reported AE capture into trials to complement clinician-based AE capture [19].

Adverse Event Reporting Challenges (Table 1)

Table 1.

Adverse event reporting challenges and potential solutions

Challenges	Solutions
Variation in training and background education on AE reporting	Increase availability of training regarding AE reporting for clinical research associates Provide training for clinicians on CTCAE and documenting AEs in the chart Provide additional guidance on interpreting AE definitions to account for variation in medical knowledge
AE reporting is labor intensive	Implement automated extraction of electronic health record data to identify AEs, for example, a tool such as ExtractEHR Incorporate trigger tools and alert systems into the EHR so that clinicians, CRAs, and research nurses are alerted to potential AEs that need to be reported Create an automated system to move data from the EHR into the clinical trial data capture system Limit the scope of AEs required to be collected on clinical trials
Complexity of the CTCAE	Streamline AE definitions to reduce potentially overlapping definitions and subjectivity in definitions Provide guidance about comprehensiveness of AE reporting (i.e. to report the syndrome and the individual symptoms or to report just the syndrome) Add more pediatric-specific guidance for AE definitions
Current AE reporting does not provide a complete understanding of AE experienced	Incorporate methods to capture duration of AEs rather than only the highest grade, such as ToxT Include patient-reported outcome measures into clinical trials to supplement clinician-identified AEs

AE, adverse event; CRA, clinical research associate; CTCAE, Common Terminology Criteria for Adverse Events; EHR, electronic health record; ToxT, toxicity over time

Despite the effort and time devoted to AE reporting, prior studies have shown underreporting of AEs on hematology clinical trials [2, 20, 21]. In order to quantify the extent of AE underreporting, Miller et al. compared AEs reported on a clinical trial for pediatric acute myeloid leukemia (AML) to gold standard physician chart abstraction at fourteen hospitals across the USA. This study found that there was less than 50% sensitivity for 8 of 12 clinically relevant AEs included in the study [2]. These results indicate that clinical trials may not provide accurate information regarding rates of the range of AEs captured on trials. This means that clinicians may not always have sufficient data to guide all clinical decisions and discussions with patients and families.

The complexity of the CTCAE may lead to challenges with accurate AE reporting. Over each subsequent version, the number of AEs included in the CTCAE has increased exponentially. The number of AEs in CTC v1.0 was 49, and CTCAE v5.0 includes more than 800 AEs [1]. CTCAE includes AEs that may have complex, subjective, or potentially overlapping definitions [22]. This may lead to variation between CRAs or research nurses in what AE is chosen to be reported. Furthermore, there is no standardized guidance regarding comprehensiveness of AE reporting across all oncology clinical trials. Some CRAs or research nurses may choose to report every individual symptom or sign a patient experiences, while others may choose a parsimonious approach and will only report the syndrome or specific AEs [1]. For patients with pediatric hematologic malignancies, current CTCAE definitions provide additional challenges. In CTCAE v5.0, the term activities of daily living (ADLs) appears in 524 grading definitions [12]. In many AEs, the delineation between grades 2 and 3 is based on if the AE impacted instrumental or self-care ADLs. This can be challenging to delineate due to both limitations in knowledge of differences between instrumental and self-care ADL definitions and gaps in clinical data that are collected from patients. In pediatric patients, ADLs vary significantly by age and developmental ability, which adds additional complexity to identifying impact on ADLs and therefore AE grade [1].

Future Directions for Adverse Event Reporting

In order to attempt to overcome challenges inherent in manual AE reporting, investigators have begun to trial automated ascertainment of clinical data from the electronic health record (EHR). Miller et al. described development of an automated method to ascertain to ascertain 12 laboratory AEs from CTCAE v4.0 to extract laboratory result data directly from the EHR at a single institution [23]. This automated method leveraged a software package in the R programming language called ExtractEHR [24]. Using this automated method, data were extracted, cleaned and processed, and graded according to CTCAE v4.0 criteria in an automated fashion. When compared to gold standard physician chart abstraction, the extraction package had sensitivity and positive predictive value greater than 98% for each AE [23]. ExtractEHR was subsequently implemented at three hospitals and successful extracted laboratory result data to describe accurate laboratory AE result data for pediatric patients receiving therapy for AML or acute lymphoblastic leukemia [24].

In addition to extracting laboratory data from the EHR, investigators are working to extract other EHR components to identify non-laboratory-based AEs. ExtractEHR and other tools have successfully extracted these data, and testing is underway in many of these platforms to use these data to accurately identify complex AEs [25]. Given the fact that some of these EHR data are free text fields, informatics techniques of natural language processing (NLP) and machine learning are necessary to identify AEs and other outcomes [26–28]. Hong et al. recently described use of NLP of EHR notes of patients who received radiation therapy to identify AEs with high accuracy. The sensitivity of the NLP process was low, however, when the notes include mentions of negated symptoms related to the AEs [26]. NLP and ML algorithms need to be honed further to improve sensitivity for detecting when an AE has not occurred based on EHR data to truly inform understanding of these AEs on malignant hematology clinical trials.

Another approach to improving AE ascertainment focuses on leveraging capabilities of EHRs by incorporating methods to capture and track AEs directly into the local EHR system. Lencioni et al. integrated a toolkit into the EHR that permits investigators to track ongoing AEs, resolve AEs, and create new AEs when they occur. Once the AEs are reported in the EHR, the data transfers to the institution’s AE tracking systems [29]. Another method that has been attempted is the use of trigger tools that search the EHR for specific terms that indicate an AE may have occurred. Weingart et al. developed a trigger tool to retrospectively search claims data for evidence of AEs in oncology patients [30]. Other investigators have trialed implementation of real-time triggers in the EHR; however, these have been reported to have low positive predictive value to date [31, 32]. Integrated AE reporting systems provide an opportunity for streamlined ascertainment of AEs. These systems are advantageous because they simplify the number of systems that are used for tracking AEs. Furthermore, they permit clinicians to be alerted as to ongoing AEs or potential AEs in real time, which may alter clinical decision-making and also may improve accuracy of AE reporting by prompting identification and grading at the time of the event. However, challenges with accuracy remain with these systems, and trigger alerts in EHR systems are often ignored due to alert fatigue [32–34]. Challenges also exist in scaling these methods to multiple hospitals and across multiple EHR vendors, and these have not thus far been reported in multi-center clinical trials.

Registry data may also be leveraged to identify AEs. While billing data alone may not be sufficient to identify AEs [2, 35], it may be feasible to use post-marketing surveillance of adverse event reports to hone knowledge about safety. Han et al. used ML techniques to screen reports submitted to the Food and Drug Administration (FDA) for significant AE signals [36]. Hauben et al. also described data mining of the FDA Adverse Event Reporting System to identify fatality associated with specific oncology agents [37]. If these AE reports can be consistently identified and tracked to the specific combination regimens in which the chemotherapy agents were used, this could provide additional knowledge that would supplement the reports submitted from clinical trials.

In addition to improving methods of AE identification, recent efforts have focused on redefining the scope of AEs captured on hematologic malignancy clinical trials. There have been suggested to simplify what is required to be reported on clinical trials [38]. Many clinical trials focus on reporting of the highest grade of each AE experienced during each treatment course or reporting period [2]. This approach does not provide an understanding of the duration of AEs that may affect patient experience and the ability to receive full therapy, especially for lower-grade AEs. This is especially important with current therapies for hematologic malignancies that may require a chronic approach to treatment [9, 39]. Thanarajasingam et al. developed a novel tool, toxicity over time (ToxT), that evaluates duration of AEs in order to improve accuracy and comprehensiveness of AE data identified and ultimately reported [40]. ToxT has been successfully applied retrospectively to clinical trial data and has potential for real-time application in ongoing clinical trials to lead to more detailed, clinically relevant AE report data.

In future trials, AE reporting will benefit from continued efforts to incorporate the PRO-CTCAE and other patient-reported outcome measures. Patient report of AEs may help improve attribution to study agents by more clearly delineating baseline AEs [41]. Patient-reported AEs will provide deeper knowledge of AEs, such as subjective AEs of nausea or fatigue, that have been challenging for clinicians to capture. Furthermore, Chung et al. found that 16.6% of PRO-CTCAE reports on three multicenter, oncology trials included additional free text comments that patients added to the standard AE report drop-downs in the measures [15]. The supplemental information in these free text fields may provide valuable information about AEs that patient experience over time.

Conclusion

Adverse event reporting on hematology clinical trials has improved over time due to the implementation of the CTCAE and increased focus on AE reporting. However, despite significant effort devoted to AE reporting, significant challenges with underreporting and inaccurate reporting of AEs remain due to the complex nature of AE definitions and the manual nature of reporting systems. Implementation of automated technologies that can harness the EHR to ascertain AEs and can adjust the focus to include duration of toxicity has the potential to reduce manual effort, standardize AE capture, and improve accuracy of AE data on clinical trials. AE reporting on hematology clinical trials will continue to evolve over time, and with this evolution will come improved understanding of AEs that can be used for clinical decision-making and as baseline comparisons for future clinical trials.