Monitoring and reporting adverse events in pragmatic clinical trials testing nonpharmacological pain management interventions

Introduction

Randomized controlled trials (RCTs) are a critical tool for assessing treatment effects of clinical interventions. RCTs are guided by ethical standards and processes to ensure participant safety and data integrity. Ensuring compliance with these standards is specified by policies, and oversight is typically provided by institutional review boards (IRBs), data safety monitoring boards (DSMBs), and others as specified prospectively in a study monitoring plan. Ensuring participant safety requires monitoring and recording adverse events (AEs) as elements that can be analyzed in the context of study findings. Generally speaking, AEs are unfavorable health-related occurrences in human research participants, irrespective of relatedness to research participation.¹ They range from unrelated changes in condition to minor tolerability issues to serious safety concerns.² Guidance for reporting AEs has long been available through a Consolidated Standards for Reporting Trials (CONSORT) statement, which has been updated recently.²

Despite existing guidance to monitor and report them, AEs are underreported in nonpharmacologic pain trials. Palermo and colleagues reviewed 40 trials in adult participants testing psychological treatments for chronic pain and found that only one quarter of them reported AEs.³ A scoping review of 113 trials assessing manual therapy for neck pain found that only 56% of trials reported the tracking of AEs.⁴ A 2014 review noted that AE reporting was universally poor in 101 trials published in one of three pain journals,⁵ in which half did not report their AE collection processes and only one-quarter reported any observed AEs.⁵ Potential factors contributing to poor reporting include journal-based word limits, lack of emphasis on reporting standards by journal editors and peer reviewers, and biased reporting and interpretation of results.⁶

Pragmatic clinical trials (PCTs) are increasingly being used to inform clinical and policy decisions. They provide evidence of the effectiveness of pain management interventions when delivered in routine clinical settings,⁷ and as such they aim to address the widely acknowledged gap between what is observed in efficacy trials and implementation into routine clinical care. PCT designs and methodologies differ significantly from more common explanatory efficacy trials by prioritizing external validity and generalizability of results. Reporting of AEs in PCTs can be critical to understanding the safe use of pain management interventions in routine clinical settings.

Unique challenges complicate AE data collection, interpretation, and reporting in PCTs.⁸ PCTs typically study established interventions, are often delivered by a larger number and diversity of healthcare providers, and have broad eligibility criteria. Thus, PCTs intentionally include participants with multiple health conditions, representative of real-world clinical settings, who may be more likely to experience a wide array and frequency of AEs not directly attributable to a study intervention. Additional factors that can complicate AE relatedness determinations include complexities and the multicomponent nature of many nonpharmacological pain management interventions, lack of a comparison control arm, and reliance upon clinical documentation to identify AEs. Both under- and over-reporting of AEs can potentially lead to additional and unnecessary testing, inappropriately denying enrollment of otherwise eligible individuals in future research, errant safety profiling of nonpharmacological therapies, and/or exposing patients to care presumed to be safe but with risk not appreciated accurately.

Authors of this Commentary are members of the Pain Management Collaboratory (PMC). This trial network is conducting 13 PCTs in veteran and military health delivery systems toward closing implementation gaps for the use of nonpharmacological approaches to manage pain and common co-occurring conditions.⁹ Consistent with emerging literature, PMC teams use a wide variety of AE monitoring strategies, grading practices, and risk determinations—all dependent on factors such as specific research questions, trial design features, randomization to a control comparison group, and intervention(s) studied.^2,8,10 We looked across PMC PCTs to identify the various approaches being used to monitor and report AEs. With this information, we offer recommendations to improve and standardize this process, toward broader awareness within the PCT research community as well as to advance the discussion of benefit and safety of research participants.

Monitoring and reporting AEs in pragmatic clinical trials

A comprehensive approach to AE monitoring for a given study should be determined at the start of the study and include a clear process to assess AEs with respect to severity, expectedness, and relatedness to study participation. Comparison with AEs occurring in control groups, when present, should also be included.

Compliant with these principles, guidance, and customary practice for monitoring AEs in clinical trials, the 13 PMC PCTs follow accepted approaches (both indirect and direct) to capture observed or reported AEs (see Table 1). Indirect methods include follow-up assessments, such as analysis of previously collected clinical data extracted from patients’ electronic health records. Direct methods include proactive strategies informed by previously completed efficacy trials, in which participants are asked to identify symptoms from a checklist of potential AEs. For pain PCTs, examples of symptoms potentially experienced include increased pain, depression, insomnia, substance use, and suicidal ideation.

Table 1.

Pain Management Collaboratory (PMC) Pragmatic Clinical Trials’ (PCTs) Approaches to Capture Adverse Events (AEs).

Direct approaches: Identify potential AEs during follow-up assessments in collaboration with participants. Enable research or clinical staff to ask participants to identify symptoms from a checklist of potential AEs informed by previously completed efficacy trials and clinical experience (eg, increased pain, depression, insomnia, substance use, suicidal ideation, and others). Ask participants to respond to open-ended questions such as: “What health changes have you experienced since your last visit?” Make preliminary determinations about whether an observed AE is related to the study and if the AE is serious. Appoint a data monitor to evaluate safety data independently and periodically. Indirect approaches: Analyze previously collected clinical data retrospectively using electronic health records.

Collecting AEs indirectly through electronic health record review may mistakenly assume all AEs observed by clinicians or reported by participants are entered in a timely manner into the health record, potentially ignoring the fact that participants do not necessarily receive all health care services from the health system in which data are collected. Indirect monitoring may thus under-report the number and frequency of AEs. Conversely, active and direct methods for identifying AEs may result in over-reporting of unrelated AEs, especially if a trial does not employ a control arm for comparison of treatment outcomes as well as AEs.

Many PMC PCTs evaluate either specific nonpharmacological approaches to manage pain and co-occurring conditions or models of care and care pathways that incorporate some of these approaches. It is intuitive that monitoring AEs in the context of trials examining integrative treatment approaches and making reliable decisions about AE relatedness is more difficult than for efficacy trials or even in PCTs testing discrete interventions.

Additional complexity arises from varied AE monitoring and reporting processes that aim to meet expectations from research sponsors, IRB and DSMB requirements, and trial-specific methods. The majority of PMC PCTs are overseen by individual DSMBs. Recent coronavirus disease 2019 (COVID-19)- inspired shifts in healthcare settings and processes yielded variable methods for interacting with study participants and collecting outcome data. Such variability is mirrored in AE-monitoring methods and may contribute to differences in how PMC PCT protocols articulate methods for defining, monitoring, and recording AEs, as well as for making determinations about AE relatedness to study interventions.

Our investigation into these practices within our own trial network highlight the need for further investigation to identify optimal methods for timely and accurate capture of AEs in PCTs. This is particularly true for pragmatic trials further along the pragmatic-explanatory spectrum, in which data collection exists almost entirely within a clinical standard of care framework and there is little to no interaction between the study team and participants. Furthermore, without a usual care or comparator intervention, it can be difficult to distinguish AEs that occur as a result of a research intervention from those that occur in everyday life.

Conclusions

Although the 13 PMC PCTs vary widely in study design, most were determined by regulatory review to be of minimal risk. Nonetheless, we believe that AE monitoring within PCTs is critical to inform policy and practice decisions. An otherwise effective intervention may not be recommended for implementation if AEs are frequently observed and attributed (or misattributed) to it. Despite an a priori assumption that many nonpharmacological pain management approaches and integrated models of care are of low or minimal risk, we suggest that intentional AE monitoring, documentation, and evaluation is needed to address the reporting gap in pain-focused PCTs.

Our observations suggest there is room to improve the consistency of AE reporting for pain management PCTs and yields four recommendations. The first is to explicitly describe within a PCT study protocol the methods to be used for monitoring and reporting AEs. The second is to review and report as part of study findings all collected AEs, by treatment arm. The third is to combine both direct and indirect methods to monitor AEs. Direct monitoring might use a checklist or open-ended questions to ask participants about negative health-related occurrences. Indirect monitoring might involve retrospectively review of an electronic health record. Both approaches offer unique benefits and challenges, which should be considered along with study design, intervention complexity, and the known risk profile of an intervention under study. Finally, we recommend that continued research should explore methods for, and consequences of, AE monitoring within nonpharmacological pain management PCTs. This knowledge base will ensure responsible safety assessment, while simultaneously avoiding unintended negative consequences from unnecessary monitoring.

Over time, universal reporting of AEs related to nonpharmacological pain management interventions could contribute to a database that can be systematically queried for expected type, frequency, and duration of AEs associated with these treatment strategies. Such a resource will benefit patients, providers considering and delivering these interventions, and health care systems.

Disclaimers: The contents of this publication are the sole responsibility of the author(s) and do not necessarily reflect the views, opinions, or policies of the National Institutes of Health; the United States Department of Veteran Affairs; Uniformed Services University of the Health Sciences; the Department of Defense; the Defense Health Agency; or the Departments of the Army, Navy, or Air Force. Mention of trade names, commercial products, or organizations does not imply endorsement by the US Government.

Funding

Research reported in this publication was made possible by Grant Number U24 AT009769 from the National Center for Complementary and Integrative Health (NCCIH), and the Office of Behavioral and Social Sciences Research (OBSSR). Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the NCCIH, OBSSR, and the National Institutes of Health.

This work is supported through cooperative agreement NIH UG3/UH3 AT009763 (JF, DR), NIH UH3AT009761 (RV) and NIH UH3/UG3 AT012257 (BM) from the NIH. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

The US Army Medical Research Acquisition Activity, 820 Chandler Street, Fort Detrick MD 21702–5014 is the awarding and administering acquisition office. This work was supported by the Assistant Secretary of Defense for Health Affairs endorsed by the Department of Defense, through the Pain Management Collaboratory—Pragmatic Clinical Trials Demonstration Project under Award No. DOD W81XWH-18–2-008 (SYM). Opinions, interpretations, conclusions and recommendations are those of the author and are not necessarily endorsed by the Department of Defense.

This manuscript is a product of the Pain Management Collaboratory. This supplement was made possible by Grant Number U24 AT009769 from the National Center for Complementary and Integrative Health (NCCIH), and the Office of Behavioral and Social Sciences Research (OBSSR). Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the NCCIH, OBSSR, and the National Institutes of Health. For more information, visit https://painmanagementcollaboratory.org

Conflicts of interest: Dr Kerns is paid honoraria for his roles as Executive Editor of Pain Medicine, as a member of the Scientific Advisory Board of the Canadian Chronic Pain Centre of Excellence, and as a member of several NIH and PCORI sponsored Data Safety Monitoring Boards. He also receives payment as a consultant on a National Institute on Aging research project (NIA-R01AG058702).

Supplement statement

This article appears as part of the supplement entitled “Pain Management Collaboratory: Updates, Lessons Learned, and Future Directions.”

This manuscript is a product of the Pain Management Collaboratory. For more information about the Collaboratory, visit https://painmanagementcollaboratory.org/.