Pragmatic Comparative Effectiveness Trials and Learning Health Systems in Pain Medicine

INTRODUCTION TO PRAGMATIC COMPARATIVE EFFECTIVENESS TRIALS

Despite increased available pain therapies, more than 50 to 100 million people in the United States still live with pain and 20 million live with high-impact chronic pain that frequently limits life or work activities.^1-3 We know little about which treatments are best for which patient under their particular circumstances or the efficacy and safety of various treatments over time. There is a lack of empirical evidence regarding the effectiveness of the various approaches to anesthesia, perioperative medicine, and pain management, which are barriers to effective and consistent care. Without this empirical evidence to match the patient’s unique characteristics with the relative effectiveness of different treatment options, clinicians are likely to rely on what they learned in their training from mentors. Consequently, this phenomenon can therefore perpetuate unwarranted variability in their practice. Therefore, we require reliable high-quality clinical evidence to improve patient outcomes continuously and ultimately tailor the most effective therapy for a specific patient and their needs.

Large randomized clinical trials or aggregates of multiple large trials are considered the gold standard of clinical evidence.⁴ The results of large randomized clinical trials are more reliable because they minimize different sources of confounding and bias. When well conducted, randomized controlled trials have excellent internal validity. However, these clinical trials are not problem free. They are costly, labor intensive, and time-consuming to conduct such trials. The results of these randomized trials have limited generalizability (ie, limited external validity) unless they enroll participants in diverse clinical settings and include patient-centered outcomes.^5-9 Indeed, generalizability of randomized controlled trial data is problematic, as classic trials exclude up to 90% of patients.^10-12 Our recent literature review on the impact of exclusion criteria on clinical trials revealed frequent use of psychosocial exclusions in clinical trials.¹³ The fact that many patients with chronic pain have psychosocial distress implies that the patients most in need of being enrolled in clinical trials are those most likely to be excluded. Furthermore, concerns about growing health care costs further limit the resources necessary to conduct these clinical trials successfully with reasonable generalizability. Therefore, we must focus on more novel and efficient designs—such as large pragmatic randomized trials—to generate clinical evidence of similar quality that can be generalized to a larger portion of our population.^14-24

Pragmatic randomized clinical trials are a valuable alternative to conventional explanatory clinical trials with inevitable trade-offs. These pragmatic trials better assess the “effectiveness” of a treatment in “real-world” settings, thus offering higher generalizability. Pragmatic effectiveness trials are larger, embedded into routine clinical care, and can investigate simple or complex treatments or treatment paths. Patients are still randomized to receive different treatments but are not necessarily blinded to their allocation. Applying proper randomization, more objective data collection measures, and appropriate statistical methods preserves protection against different confounding and bias sources. The treatment protocols should be more flexible, allowing adjustment when necessary for the clinical care of individual patients. Outcome measures should better reflect what is essential in clinical practice. These trials can enroll large numbers of patients over a shorter period of time and at a fraction of the cost compared with conventional explanatory trials.^14,25-31 The ultimate aim of pragmatic effectiveness trials is to “inform real-world decision-making process.”^25,31,32

Despite numerous advantages, pragmatic effectiveness trials are not the perfect solution for all circumstances. Delivering a treatment and collecting data in a “real-world” setting increases heterogeneity. It is, therefore, difficult to estimate the treatment effect under optimal conditions. The treatment effect is also less uniform in the study population, necessitating more precise clarification about different subgroups and the range of treatment effects.¹⁴

Considering the pros and cons, large pragmatic effectiveness trials are optimal trial design choices in various clinical settings. At the end of the pragmatism spectrum, pragmatic effectiveness trials use outcome data collected in routine clinical. Clinicians and automatic recording systems gather a wealth of data in perioperative period. Anesthesia and perioperative medicine have the luxury of using these data to conduct large pragmatic effectiveness clinical trials; these trials are well embedded within routine clinical care with minimal to no individual patient contact.^14,33-40 However, embedding large pragmatic effectiveness trials within clinical care is more challenging in pain medicine.

APPLICATION OF PRAGMATIC COMPARATIVE EFFECTIVENESS TRIALS IN PAIN MEDICINE

Application of large pragmatic effectiveness trials provides multiple opportunities in pain medicine: (1) researchers can investigate more complex treatment modalities or treatment paths while giving the clinicians the flexibility to tailor the treatment to the needs of their patients; (2) this trial design also focuses on outcome measures that are more important to clinicians, for example, disability, social function, quality of life, or even cost-effectiveness instead of a simple pain score; and (3) simplicity of these trials and their lower burden allow longer follow-up periods.²⁵ When studying chronic diseases such as patients with chronic pain, assessing how treatment effect changes over a longer course of therapy is crucial. The results of pragmatic effectiveness trials can guide long-term, real-world application of different treatment modalities for patients with chronic pain.^14,25

Application of large embedded pragmatic effectiveness trials in pain medicine poses its unique challenges: (1) the outcome measures required for these studies are not routinely collected in all clinical settings, thus limiting the application of this design. Collecting all necessary outcome measures promptly and with an acceptable compliance rate requires a unique clinical infrastructure poised to streamline this process. (2) Even in practices that collect these outcome measures, they record them during clinical visits. These clinical visits are not necessarily synchronous with the time intervals important for studying a specific intervention. More advanced statistical methods can partially overcome the limitations of asynchronous data collection; however, this method cannot perfectly substitute proper data gathering at predefined intervals. (3) Obtaining more objective outcome measures on disability and physical function is very difficult in these pragmatic trials. Use of commercial and research-grade actigraphy devices might overcome this limitation soon. (4) Finally, waiver of informed consent process or simplification of the process is routinely acceptable in pragmatic clinical trials that rely on registries and medical records to collect more objective outcome measures with no participant contact. However, collecting more subjective patient-reported outcomes makes this streamlined research process more challenging.⁴¹ This process is more complicated for pharmacologic therapies considering that most medications used to treat chronic pain are “off-label.” Most of the pragmatic clinical trials in pain medicine investigate nonpharmacological therapies, for example, behavioral therapies, physical therapy techniques, treatment pathways, and so forth.²⁵

These challenges have significantly limited the application of pragmatic effectiveness trials in pain medicine. A more recent systematic review could only identify 57 clinical trials that met the criteria to be considered as “pragmatic trials.” The average PRECIS-2 score for these trials was 3.8 (±0.6), which is comparable to other fields (eg, 3.83 ± 0.78 in cardiology).⁴² However, domain-specific scores were lower for recruitment and follow-up.²⁵ The investigators of this systematic review also suggested several areas of improvement for future pragmatic effectiveness trials in pain medicine: (1) better reporting of the nature of the study center; (2) better reporting of qualifications and expertise of providers; (3) indicating the nature of pain condition, for example, duration of pain, pain diagnosis, location of pain, and so forth; (4) providing more detailed information on comparator groups and how “usual clinical care” is defined; (5) better reporting of other pain treatments participants received; and (6) better justifying of the choices made in trial design.²⁵ It is essential to know if the choice of delivery method of intervention, outcome measures, and follow-up period are made based on clinical reasons, patient preferences, clinician, preferences, or feasibility of the study.^14,25

These challenges and shortcomings should not discourage pain researchers from application of this novel design to investigate long-term, real-world applications of chronic pain treatments. Conversely, understanding these challenges and areas for improvement can guide pain researchers to design and conduct more robust pragmatic effectiveness trials by improving the infrastructure for recruitment and data collection and being more diligent in explaining their methods and findings.²⁵

INFORMED CONSENT

The first step in successfully conducting pragmatic effectiveness trials in pain medicine is improving the ability to systematically recruit more patients while minimizing the research burden for study participants and clinicians. The traditional informed consent process can be burdensome for both participants and researchers and thus hamper the streamlined recruitment process. The traditional informed consent discussion involves explaining at least 3 important factors to the potential participants: (1) the purpose of the study; (2) specifics of the research process including randomization, data collection, additional visits or tests, and so forth; and (3) potential risks and benefits of participating in research, including those of the experimental treatment.^41,43,44 By design, pragmatic, effective trials do not involve “experimental treatment,” and additional visits/test while trying to minimize alterations to “usual clinical care” of the participants. Therefore, the waived or streamlined informed consent process may be necessary for the successful conduct of pragmatic effectiveness trials.^41,45

The traditional informed consent process can cause significant distress and confusion for patients. The decision-making process can cause anxiety for some patients and have adverse emotional consequences. For patients trying to decide about the next steps of a treatment plan, adding a vast amount of information (many informed consent forms are 15 pages or more) and another decision point about participation in a research study leads to information overload and significant anxiety.^41,43

The ethical principle that advocates the necessity of a traditional informed consent discussion is based on patient autonomy. The common assumption is that we give the patients full autonomy by providing them with as much information as possible at any decision point.^41,43,46-48 However, we challenge this assumption, as it is beloieved that information overload—the state in which increasing the amount of information decreases the ability to make a rational decision—fails to improve autonomy and minimize harm because it poses an unnecessary burden and distress for patients. This process is not only an obstacle to autonomous decision-making but also tends to be more confusing for the participants when used in pragmatic effectiveness trials; the participants of these trials will receive some form of efficacious “usual clinical care” in either arm of the study.^41,43 We argue that the actual research intervention in these pragmatic effectiveness trials is the process of randomization and not the treatment modalities the participants will receive, as no “experimental” treatment option is available in these studies.^41,43

The controversy about informed consent in pragmatic clinical trials dates to 1980s.^41,49,50 Although Vickers and colleagues argue that traditional research informed consent can be even counterproductive for explanatory clinical trial⁴³; when applied for pragmatic clinical trials it can compromise trial integrity or even make it impossible to conduct.⁴¹ At best, experts doubt whether traditional informed consent is ethically necessary when we are studying existing medical practice in large pragmatic effectiveness trials. At worst, others argue that legalistic long informed consent documents for minimal risk trials may result in an “injurious misconception”; potential participants then reject being in a trial because of an exaggerated projection of risk.^41,43,51

Symons and colleagues summarized 6 models that represent a spectrum of simplified, altered, or waived informed consent.⁴¹ They discussed the acceptability of these models in different research ethics environments of the United States, Australia, and the United Kingdom.⁴¹ They also represented 3 examples showing how large pragmatic trials became feasible using these models.^36,40,41,52 They also emphasized that nearly all research regulatory bodies allow waiver of informed consent if “the research would not be feasible or practicable to carry out without the waiver, the research has important social value, and the research poses no more than minimal risks toward participants.” Most large pragmatic effectiveness trials meet the criteria for being impactful and posing no more than minimal risk; the main question is what makes a trial “impracticable.”^41,53-55

The United States Secretary’s Advisory Committee on Human Research Protections defines trial impracticability as follows: “Appropriate ethical or scientific rationales might include, for example: (i) scientific validity would be compromised if consent were required because it would introduce bias to the sample selection; or (ii) subjects’ behaviors or responses would be altered, such that study conclusions would be biased; or (iii) the consent procedure would itself create additional threats to privacy that would otherwise not exist; or (iv) there is risk of inflicting significant psychological, social or other harm by contacting individuals or families. Once the IRB has determined that the waiver or alteration does not adversely impact the research’s ethical nature or scientific rigor, logistical issues (eg, cost, convenience, speed) may be considered.”⁵³ This paragraph provides useful framework about how ethics committees can assess overall balance of risks, burden, and benefit when making decisions about alteration or waiver of informed consent process. Some further argue that consideration of cost is an important factor in determining the impracticability of a pragmatic trial; this consideration is more important for high-impact, publicly funded, investigator-led clinical trials.^41,56,57

Some opponents to this concept argue that in some cases the risks of the disclosure of data may be more harmful than the minimal risks posed by randomization in large effectiveness trials. Waiving consent for observational and retrospective studies, considering excessive cost but not taking into account the same consideration for clinical trials, would seem a double standard.⁴¹ During times of major economic rationalization and progressively more limited health care system resources, allocating valuable resources to obtaining individual consent could be unethical.^41,58

This debate has gained momentum after the Institute of Medicine advocated applying a “Learning Healthcare System” for more effective knowledge generation.⁵⁹ As we will describe in detail later, a learning health care system embeds clinical research into routine clinical practice to continually improve care and deliver value; altered or waived consent models are essential for this integration.^41,59 Different pioneer groups have proposed different models to adapt informed consent to learning health care systems: integrating research consent with the routine clinical discussion with patients; routine randomization as the default position followed by post-hoc consent or opt-out models; and integrating a rigorous, systematic evaluation into normal practice and waiving individual consent where patients and the public are better informed.^{35,36,39-41,43,45,51,52,59-61}

Many large pragmatic effectiveness trials leverage flexible approaches to use altered or waived informed consent. Most experts now agree that these flexible models are essential for successfully implementing “Learning Healthcare Systems”⁵⁹ to embed pragmatic trials in routine clinical practice. Most also agree that the application of altered or waived consent is not only ethically defensible but also can promote beneficence and justice by more effective use of limited resources to maximize knowledge generation.

THE LEARNING HEALTH SYSTEM AND HIGH-QUALITY, REAL-WORLD DATA COLLECTION

Effective systems to help practitioners integrate relevant measures and monitor patient outcomes have not existed until recently. The United States Institute of Medicine (IOM; now the National Academy of Medicine [NAM]) called for developing learning health care systems. As envisioned by the IOM, a Learning Health System (LHS) leverages an integrated digital infrastructure to provide data-based driven and coordinated care that is available just in time to the clinician and that is centered on the patient. LHSs combine science, informatics, data science, incentives, and culture that are then aligned for continuous improvement and innovation. The NAM and National Science Foundation extolled the virtues of LHSs⁶² and declared that LHSs can rapidly inform decisions that have transformative effects on improving health.⁶³ Properly implemented, LHSs can be used to optimize and tailor care as well as their future potential to help achieve the goal of precision medicine.

A defining attribute of an effective LHS is to have a data collection system that is (1) already embedded in routine clinical practice; (2) simple and short enough that minimize time and mental burden for participants and clinicians; and (3) valid for measuring the intended outcomes.^14,64 In perioperative medicine, available objective measures meet all these criteria, and we can easily extract them from medical records without frequent patient contact; this has resulted in the more widespread use of larger-scale pragmatic clinical trials in anesthesiology.^14,33-40 Although an anesthesiologist relies heavily on objective data while delivering care intraoperatively, routine clinical practice of a pain physician involves seeking several patient-reported outcome measures through asking subjective questions.

More pain clinicians and academic centers are using standardized patient intake forms or questionnaires to collect more uniform pieces of information about their patients that can better guide their practice.²⁵ Streamlined integration of these data collection systems into routine clinical care requires that these instruments are simple enough and provide value for clinicians and patients.^25,64 Vickers and colleagues have been using an embedded system for more than a decade.^64,65 They believe that their success is because of following these 10 golden rules:

1. What seems obvious to an engineer (or informatics manager) may not be obvious to a patient.

2. What seems quick and easy may strike the patient as burdensome.

3. Questionnaires developed for research may not be appropriate for clinical practice.

4. Many words used by doctors and researchers can be replaced by something simpler.

5. Mandatory fields and open text cause problems.

6. Do not ask questions for clinical care unless you are prepared to act.

7. Patients have to see that completing the questionnaire is in their best interests.

8. A subgroup of users can cause a great deal of additional work, but, unlike Amazon and Uber, you cannot ignore those users.

9. Watch patients use your tool and ask about their experiences.

10. Patient trust is hard to gain and easy to lose.⁶⁵

Once we implement such a system and integrate it into routine clinical practice, we can apply it to conduct larger-scale pragmatic effectiveness trials.^64,66,67 A dynamic learning health care system allows the researchers to adapt data collection points to gather outcome measures necessary to test their hypotheses. Reporting findings based on high-quality subjective data is valuable and represents the measures clinicians use to make decisions when delivering care in routine clinical practice.

THE STANFORD LEARNING HEALTH SYSTEM MODEL AND FUTURE DIRECTIONS

In recognizing the societal problem of pain, the IOM Relieving Pain In America report called for “greater development and use of patient outcome registries that can support point-of-care treatment decision making, as well as for aggregation of large numbers of patients to enable assessment of the safety and effectiveness of therapies.”¹ Similarly, in the Health and Human Services National Pain Strategy (Mackey; Co-Chair), the committee stated, “better data are needed to understand the problem and guide action.” In response to these calls, Stanford University Division of Pain Medicine developed CHOIR (http://choir.stanford.edu; Principle Investigator: Mackey) as an innovative, open-source, highly flexible, and free learning health care system (LHS).

CHOIR (Fig. 1) was developed to provide high-quality, point-of-care data to optimize care and for real-world research discovery. Using a Web-based interface, CHOIR captures patient-reported outcome data at each clinic visit, graphically displays real-time results that inform point-of-care decisions, and tracks patient treatment responses longitudinally. CHOIR emphasizes tracking of patient-generated information as a core component of clinical practice, allowing for individualized improvements in the health care delivery process over time, and guiding precision medicine. As a flexible platform, CHOIR has also been tailored for other medical specialties, including Preoperative Anesthesia Assessment, Pediatric Pain,⁶⁸ Orthopedics Hand and Joint Replacement, Interventional Radiology,⁶⁹ Chronic Fatigue, Psychiatry, and Primary Care/Family Medicine.⁷⁰

Fig. 1.

CHOIR.

CHOIR integrates NIH Patient-Reported Outcomes Measurement Information System (PROMIS) measures to efficiently and rapidly capture 15 to 20 domains of physical, psychological, and social functioning. The role that psychological and social factors play in the incidence, magnitude, and persistence of pain, as well as the associated costs of care, have increasingly come to light and, as noted earlier, is frequently a basis for exclusions in clinical trials. Consequently, there has been a demand to measure and monitor psychological and social factors to manage these complex interactions better. Another strength of PROMIS measures is that they allow comparisons of individual patients against national population norms. CHOIR also has a built-in computer adaptive testing engine (CHOIR-CAT) to deliver both legacy and more modern item response theory (IRT) surveys, such as those used by PROMIS. The use of computerized adaptive testing (CAT) reduces participant burden. Overall and after several hundred thousand administrations of PROMIS surveys, CHOIR-CAT reduces subject burden by approximately 75% compared with instruments based on classic testing theory. CHOIR also features an interactive, validated pain body map (CHOIR Body Map) available in CHOIR and separately as a library within Research Electronic Data Capture (REDCap).^71,72

In addition to its clinical utility, CHOIR has been an invaluable research tool to capture real-world research evidence, which has been a high priority by the Food and Drug Administration, the National Institutes of Health, and NAM.^73,74 Furthermore, CHOIR addresses the need for generation of systematic practice-based evidence by allowing low-cost, large, prospective, observational studies on thousands (or more) of patients in a “real-world” clinic setting. From real-world data collected from patients with chronic pain and using CHOIR, we have gained critical insights about the effects and impacts on chronic pain from tobacco,⁷⁵ cannabis,⁷⁶ social functioning and social isolation,^77-80 fatigue,⁸¹ perceived injustice,^82,83 pain catastrophizing,^84-86 opioid misuse,^87-91 multiple overlapping pain conditions,^92,93 cancer,^94,95 and symptom severity.⁹⁶ More than 30 manuscripts (full list at https://choir.stanford.edu/publications/) have been published using CHOIR data across multiple sites, providing unique insights into the characteristics and treatment responses of “real-world” patients with pain and other conditions.

We have successfully adapted CHOIR to conduct large-scale pragmatic effectiveness trials more effectively and at a lower cost by applying these modifications.

1. We have successfully worked with our institutional board review to implement a 2-step informed consent process. Before a clinical encounter, the patients read a one-page simplified informational sheet about pragmatic effectiveness trials and randomization. We follow with a more formal informed consent for patients who are randomized later instead of all patients.

2. If there is a true equipoise, the clinicians randomize the patients to a treatment arm instead of randomly choosing the treatment themselves. This point-of-care randomization⁹⁷ step allows the clinicians to initiate treatment immediately without waiting for the process of randomization to be completed by a research coordinator.

3. We have integrated targeted questionnaires in CHOIR for each general type of treatment (medications, invasive interventions, behavioral interventions, physical therapy, and so forth). The patients receive and complete these questionnaires as a component of their clinical care at time intervals relevant to each specific treatment modality.⁹⁷

4. A research coordinator contacts the patients to obtain formal research informed consent to use the patient information in our clinical trials.

We believe that successful implementation of this model will allow academic centers to maximize the process of generating knowledge by transforming each clinical encounter into a research opportunity.

This model has limitations including the following: (1) successful research recruitment requires the constant engagement of the clinicians in our research process. It is essential to constantly remind clinicians during their busy and stressful clinical days until this process becomes a habitual step of their routine clinical practice; (2) we still lack objective data to characterize and improve the participants’ physical and social functioning. More widespread application of wearable devices might be an opportunity to overcome this limitation in the future and is a future goal for integration within CHOIR^98-101; and (3) ethics committees might still be hesitant to accept this altered method of consenting participants, especially when investigating more invasive interventions. An ongoing dialogue and collaboration between investigators and ethics committees is necessary to better educate members of ethics committees about these advances in clinical trials and benefits of applying them; this is only feasible if we demonstrate that our research teams are fully committed to always protecting rights of all patients and research participants and advocating for them.

SUMMARY

Application of large pragmatic effectiveness clinical trials in pain medicine poses certain unique challenges, considering the lack of more uniform, objective outcome measures. These challenges have limited the number of these trials in our literature. Pain researchers are thinking more creatively to build a better infrastructure of learning health care systems to streamline the informed consent process and embed data collection into routine clinical care. We can then successfully leverage these systems’ benefits to design and conduct larger scale pragmatic effectiveness trials. These trials are essential for understanding our treatments’ effectiveness in real-world application.

KEY POINTS.

• Large pragmatic effectiveness trials generate evidence for real-world applications of treatment modalities by enrolling a large number of patients at a lower cost; the findings can be generalized to a wider population.

• Alteration or waiver of a traditional informed consent discussion is crucial in successful application of large pragmatic effectiveness trials in pain medicine.

• Learning health care systems can provide a dynamic and adaptable infrastructure that can facilitate data collection for routine clinical care; this is a crucial step for efficiently collecting subjective outcome measures needed for studying chronic pain treatments.

CLINICS CARE POINTS.

• When reading research papers, we should consider eligibility criteria more carefully to assess if the results can be applied to our patient population.

• When reading research papers, we should pay attention to outcome measures and decide if they represent what is important for our clinical practice.