Point-of-Care Clinical Trials in Sports Medicine Research: Identifying Effective Treatment Interventions Through Comparative Effectiveness Research

Abstract

Context

Recently, calls to conduct comparative effectiveness research (CER) in athletic training to better support patient care decisions have been circulated. Traditional research methods (eg, randomized controlled trials [RCTs], observational studies) may be ill suited for CER. Thus, innovative research methods are needed to support CER efforts.

Objectives

To discuss the limitations of traditional research designs in CER studies, describe a novel methodologic approach called the point-of-care clinical trial (POC-CT), and highlight components of the POC-CT (eg, incorporation of an electronic medical record [EMR], Bayesian adaptive feature) that allow investigators to conduct scientifically rigorous studies at the point of care.

Description

Practical concerns (eg, high costs and limited generalizability of RCTs, the inability to control for bias in observational studies) may stall CER efforts in athletic training. In short, the aim of the POC-CT is to embed a randomized pragmatic trial into routine care; thus, patients are randomized to minimize potential bias, but the study is conducted at the point of care to limit cost and improve the generalizability of the findings. Furthermore, the POC-CT uses an EMR to replace much of the infrastructure associated with a traditional RCT (eg, research team, patient and clinician reminders) and a Bayesian adaptive feature to help limit the number of patients needed for the study. Together, the EMR and Bayesian adaptive feature can improve the overall feasibility of the study and preserve the typical clinical experiences of the patient and clinician.

Clinical Advantages

The POC-CT includes the basic tenets of practice-based research because studies are conducted at the point of care, in real-life settings, and during routine clinical practice. If implemented effectively, the POC-CT can be seamlessly integrated into daily clinical practice, allowing investigators to establish patient-reported evidence that may be quickly applied to patient care decisions. This design appears to be a promising approach for CER investigations and may help establish a “learning health care system” in the sports medicine community.

Key Points

• Compared with traditional research methods such as the randomized controlled trial and observational studies, the point-of-care clinical trial (POC-CT) may better facilitate and support comparative effectiveness research.

• By incorporating an electronic medical record, the researchers in a POC-CT seek to embed many research-related processes without disrupting the patient care experience as well as reduce patient and clinician burden during the clinical trial.

• The use of a Bayesian adaptive design during a POC-CT can result in clinical trials that are more efficient and ethical.

As the sports medicine community continues its push toward an evidence-based practice (EBP) culture, it is apparent that a gap persists between the evidence generated by researchers and the evidence patients and clinicians need to inform patient care decisions.¹ This gap is marked by the historically limited and slow process of translating research findings from the laboratory environment to the patient care setting.¹ This incorporation of evidence into routine patient care, particularly intervention-related evidence, is often hindered by traditional approaches to athletic training research. Specifically, intervention studies are typically conducted in a well-controlled laboratory setting with a relatively small and homogeneous sample (fewer than 30 healthy college-aged participants).² Thus, these investigators generally address the efficacy (does the intervention work under well-controlled and optimal treatment conditions?) and not the effectiveness (does the intervention work under routine and variable treatment conditions?) of interventions.^3–6 In addition, athletic training researchers have generally emphasized disease-oriented outcomes as opposed to patient-oriented outcomes.² Taken together, these traditional research approaches tended to produce lower-quality studies and generate evidence with less strength of recommendation, as classified by the Strength of Recommendation Taxonomy.⁷

Because clinicians do not work in well-controlled environments and the measurements obtained typically do not provide patients with meaningful information (eg, patient-reported outcomes) to inform their care decisions, the findings from intervention studies can be difficult to apply to patient care decisions. As an example, consider the treatment of chronic ankle instability (CAI), which is a well-researched condition in sports medicine.⁸ Recent authors⁸ have used patient-reported outcome measures to identify the benefits of different treatment approaches for this condition, including various manual therapy methods. However, most of the best available evidence relied on relatively small study samples of individuals from a general university student population.⁸ Further, the treatments assessed in most of these studies were provided in research laboratory settings.⁸ Because these participant and treatment settings do not reflect usual care, incorporating research-based evidence into the routine care of patients with CAI can be challenging. Ideally, and as highlighted by the translational research model, promising laboratory-based, efficacy findings should be translated and tested in more usual patient care environments and in real patients to better estimate the overall effectiveness of a treatment.⁵

To encourage more effectiveness studies, government health care agencies^9–11 and the sports medicine community^12–14 have advocated for comparative effectiveness research (CER). Deeply rooted in practice-based research^13,15 and clinical outcomes assessment,^16,17 CER involves direct comparison of existing treatments to determine what works best. Pragmatic CER studies are conducted at the point of care in real-life environments and emphasize the importance of evaluating patient-reported outcomes when determining the effectiveness of an intervention.¹⁸ More specifically, CER compares the effectiveness of 1 intervention with another intervention for treatment of the same condition or illness.¹⁹ When studies are conducted properly, CER findings allow patients, clinicians, and policymakers to identify the most effective intervention for a specified condition while considering factors such as cost efficiency and the overall burden of the intervention (eg, duration and amount of treatment) when making patient care decisions.¹⁹ Consider again our CAI example. In a CER study, treatment of patients with this condition would compare the effectiveness of 1 common intervention with another common intervention that may be less time consuming or less costly (or both) for the clinician. To estimate the effectiveness of the intervention, the study would focus on patient-reported outcomes relating to functional capabilities and health-related quality of life. By comparing 2 common treatment approaches supported by the current best evidence and determining effectiveness based on patient-reported outcomes, findings from CER studies may provide better information for clinicians and patients and guide patient care decisions.¹⁸

Although the need for and benefits of CER in athletic training are clear,¹³ conducting such studies can be challenging. Traditional research methods, such as randomized controlled trials (RCTs) and observational studies, are ill suited for CER, so new, innovative research methods are needed to meet the objectives of CER.^19–27 One such design, the point-of-care clinical trial (POC-CT), addresses the limitations of RCT (eg, high costs, viability in a real-life setting, lack of generalizability of the findings) and observational (eg, bias, potential confounders) designs so that rigorous clinical research can be feasibly conducted during routine patient care.^22,25 Although the POC-CT design has been successfully used in medical practices and hospital-based research networks,^20,24 it has yet to be applied in athletic training research. Thus, more CER studies in athletic training are necessary to support EBP and improve patient care. The objectives of this article are to discuss the limitations of traditional research designs (RCTs, observational studies) as they relate to CER, describe the POC-CT as an alternative to traditional research designs for CER studies, and highlight major features of the POC-CT in the context of CER and EBP. To illustrate how the POC-CT can be implemented in athletic training research, we will continue to use the example of treating patients with CAI.

LIMITATIONS OF TRADITIONAL RESEARCH DESIGNS

To understand the inherent limitations of applying traditional research designs to CER studies, the inverse relationship between the internal and external validity of experimental designs and how this relationship affects the goals of CER and EBP efforts must first be examined. Internal validity determines the scientific rigor of a study. In short, investigators seek to identify known or expected sources of variability to reduce their influence and isolate the true effect of the intervention being investigated.^28,29 Threats to internal validity are many,³⁰ but all introduce systematic error that can bias the results of a study. One common threat is the introduction of a confounding (extraneous) factor that occurs outside of the study protocol: for example, a participant assigned to the no-treatment group who receives treatment from a clinician not associated with the ongoing study. Another example is the researcher's conscious or unintentional assignment of patients with more severe injuries to the treatment group, which would be a form of experimenter bias. When designing a study, investigators use multiple strategies, such as random assignment of participants into groups and control or comparison groups that do not receive the intervention of interest, to protect against threats to internal validity and to isolate any cause-and-effect relationship between the independent (treatment) and dependent (outcome of the study) variables.^28,29

External validity is concerned with the degree to which the results may be generalized beyond the context of the study: that is, the ability to apply the study findings to other patients and environments.^28,29 To enhance generalizability, investigators seek to include a sample of participants that represents the larger patient population of interest. This is typically accomplished by creating inclusion and exclusion criteria for the study that reflect the target patient population and conducting the study in an environment similar to that in which the treatment is typically provided.^28,29 Both internal and external validity are important research design considerations, but a high degree of internal validity generally limits external validity and vice versa. When designing and conducting clinical research, investigators must balance internal and external validity to maximize both scientific rigor and generalizability. To date, researchers have relied primarily on 2 methods for conducting health care research: RCT and observational designs.

The Randomized Controlled Trial

The RCT is considered the criterion standard of experimental designs because of its ability to protect against threats to internal validity.^31,32 Features of the RCT, such as strict selection criteria for participants, inclusion of a control or comparison group, random assignment of participants into treatment and comparison groups, and rigid intervention and measurement protocols, can help control for bias from potentially confounding factors, allowing for a better estimate of the effect of the intervention on the study outcome. Because of its ability to limit participant heterogeneity, optimize internal validity, and establish a cause-and-effect relationship between the intervention and the outcome, a well-designed RCT can generate the highest levels of evidence.⁷ However, this approach also has several well-known limitations, particularly when considered in the context of CER.³³

Although constraints implemented in an RCT may protect against threats to internal validity, these same constraints may limit the external validity of study findings. For example, in athletic training research, it is common for the intervention to be provided in a research laboratory by a research team member instead of in a clinic by the patient's health care provider. Scientifically, this approach is preferable because it limits variability in treatment delivery by the practitioner (usually provided by an investigator on the research team), treatment parameters (eg, frequency, duration), and protocols (eg, duration between treatments). Yet study findings may be difficult to generalize to routine patient care because the treatment environment does not reflect real-life patient care environments.^5,19,32 This is a major limitation for CER efforts because studies with low external validity do not translate into clinically meaningful interpretations of treatment effectiveness that can influence patient care decisions. This is also problematic because the goal of CER is to allow clinicians and patients to use findings to support patient care decisions.^19,32

In addition to external validity limitations, the RCT has practical limitations, particularly for athletic training. One major limitation is the substantial infrastructure required to successfully conduct the study.³⁴ This infrastructure is often cost prohibitive because it typically requires a large research team and extensive time to ensure that research processes (eg, participant recruitment, enrollment, and randomization) are completed during the study in a timely manner (Table 1).^19,20,34 Furthermore, the RCT design often requires researchers or clinicians to include appointments, clinical tests, paperwork, and questionnaires beyond the standard of care, which can be time consuming and cumbersome for the patient, clinician, and research team.^31,34 This departure from the standard of care can also add to limitations surrounding the generalizability of the study results and negatively affect recruitment efforts and patient compliance, especially if patients perceive their participation in the study as a burden with little or no financial compensation or benefits.^25,34 These practical limitations are compounded in athletic training research because few clinics have the time, money, or infrastructure of the settings (academic hospitals) where RCTs are routinely conducted.⁵ Given the substantial infrastructure required, high patient and clinician burden, and low external validity, the RCT may not be the optimal approach for CER investigations in athletic training.²⁷

Table 1.

Comparison of Randomized Controlled Trials, Observational Designs, and Point-of-Care Clinical Trials

Study Component	Randomized Controlled Trials	Observational Designs	Point-of-Care Clinical Trials
Scientific considerations
Randomization of patients?	Yes	No	Yes
Level of control	High	Low	High
Validity optimization	Internal validity	External validity	Internal and external validity
Practical considerations
Relative infrastructure	Large	Small	Small
Relative costs	High	Low	Low
Typically disrupts routine patient care?	Yes	No	No

The Observational Study Design

Researchers have also used the observational study design to conduct CER.²⁶ Unlike the RCT, the observational design does not directly control research-specific variables and is typically descriptive and exploratory in nature.^20,26 For example, investigators may observe a patient population prospectively (follow patients and collect data over time) or retrospectively (analyze data that were previously collected) to determine if a treatment is effective. In short, data from an observational design are collected as the patient encounters occur in the real-world clinical setting, and variables are not manipulated by the investigator.²⁶ In addition, inclusion criteria for participant selection in an observational design may be less stringent than for an RCT.^26,35 In combination, these characteristics offer greater external validity than RCTs, and observational findings are considered more “representative” and, thus, more generalizable to a typical patient population.^26,35 Data from observational studies form the foundation of practice-based research, and the generalizability of findings to the general population is unrivaled because of large, representative samples.^26,35 For example, the findings from the landmark Framingham Heart study, which led to the identification of major cardiovascular risk factors, such as obesity³⁶ and hypertension,³⁷ highlighted the potential effects and importance of large observational datasets for improving patient care.

Although observational research may be a more straightforward option for CER, limitations of this study design have also been identified. If the study is not conducted prospectively, missing data are often a problem, and patients are routinely lost to follow-up if they are not enrolled in a study. Furthermore, without randomization or strict selection criteria for participants, observational designs are vulnerable to confounding variables that may lead to bias and mask the true effects of the intervention on the study outcome.^20,26 For instance, a researcher cannot randomize participants into groups using an observational design because the groups are often based on a condition of interest (eg, CAI, concussion, posterior shoulder tightness). To address this concern, researchers typically stratify or match participants on known confounding factors (eg, age, sex, height, weight, sport) or attempt to address group imbalances using statistical techniques, such as analysis of covariance or propensity analysis.²⁷ However, such techniques do not address unknown confounders or those for which data are not available. This is an inherent limitation of observational studies and can create (or fail to eliminate) systematic error in the results.^26,32,34

Although RCT and observational designs include elements conducive to CER studies, neither method is fully suited to accomplish the primary aims of a CER study. In general, the RCT suffers from a lack of external validity, whereas the observational study suffers from a lack of internal validity. As a result, researchers have been encouraged to adopt new research approaches to address the limitations of traditional RCT and observational designs to better facilitate CER efforts.^19–27 For example, with increased attention on CER efforts, the pragmatic-explanatory continuum indicator (PRECIS) tool^38,39 was developed to better estimate the degree to which a study addresses efficacy or effectiveness. In brief, the PRECIS tool considers 10 components of a study design, such as flexibility of the intervention application, expertise of the practitioner applying the intervention, and nature of the primary study outcome, to determine where the study falls on the explanatory (efficacy) to pragmatic (effectiveness) continuum.^38,39 Thus, in conjunction with CER initiatives, calls have been made for more pragmatic studies that are designed to measure the effects of interventions under routine treatment conditions.^40,41 With the call for more pragmatic studies to support the objective of CER efforts, researchers have developed a new and innovative study design called the POC-CT.

A NOVEL APPROACH: THE POC-CT

The POC-CT was pioneered by researchers from the Massachusetts Veterans Epidemiology Research and Information Center as a project of the US Department of Veterans Affairs (VA),^20–22 the Center for Innovative Study Design at Stanford University,^21,22 and the Memorial Sloan Kettering Cancer Center in New York City.^24,25 The POC-CT was designed to allow researchers to address the limitations of the RCT and minimize the effects of potential sources of bias in observational designs so that rigorous clinical research could be feasibly conducted during routine patient care (Table 2).^22,25 To accomplish this, the POC-CT features 2 innovative components rarely applied in athletic training research: (1) a fully integrated, customized electronic medical record (EMR) that can be used during routine care by both the patient and clinician and (2) a Bayesian adaptive feature that can help limit the number of patients needed for the study, restricting costs and improving the feasibility of the study.

Table 2.

Study Processes in Traditional Randomized Controlled Trial and Point-of-Care Clinical Trial

Study Process	Traditional Randomized Controlled Trial	Point-of-Care Clinical Trial
Screening patients for study eligibility	Clinician or research team member collects, records, and assesses patient demographics at point of care; clinician completes physical examinations	Patient enters demographic information in EMR; EMR algorithm evaluates patient's demographics and alerts clinicians of patients who conform to inclusion and exclusion criteria of ongoing clinical trial; clinician completes physical examinations
Obtaining consent from patients for study enrollment	Clinician or research team member oversees consent process; typically completed via paper forms	Patient reviews informed consent forms in EMR; clinician available to answer any questions; patient, if interested, can sign informed consent in EMR
Randomizing patients into intervention group	Clinician or research team member follows established study procedures for group randomization for each patient	The EMR randomly assigns patient into an intervention group based on established allotment algorithm (eg, Bayesian adaptive design)
Recording study measurements	Measured and recorded by clinician or research team member at clinical site; historically disease-oriented variables	Patient completes study measurements within EMR from computer or smartphone; focuses on patient-oriented variables (eg, patient-reported outcome measures)
Tracking patients for measurements	Clinician or research team member manually tracks each patient throughout study period and provides reminders	The EMR tracks study measurements and sends clinician and patient notifications when measurements are due for study

Abbreviation: EMR, electronic medical record.

The Role of an EMR

The hallmark of the POC-CT is that study components of the RCT are embedded into routine patient care.²² When successfully implemented, the POC-CT, or clinically integrated randomized trial, integrates seamlessly into patient care, and the typical clinical experiences of the patient and the clinician are minimally affected by the ongoing study.²⁵ The use of a fully integrated EMR is a key component for preserving the typical clinical experiences of the patient and clinician during a POC-CT because both can routinely and frequently interact during patient care.^20–22 For example, in athletic training, a fully integrated EMR would allow the patient to routinely enter services into a daily sign-in or log-in form and complete patient-reported outcome measures and allow the clinician to comprehensively document all aspects of patient care.

With the POC-CT design, the EMR is used to facilitate research by embedding many of the study processes into the system,^20–22 resulting in 2 distinct benefits. First, the EMR replaces the extensive infrastructure associated with a traditional RCT, which reduces the high costs of conducting a study.^20,22 More specifically, the traditional RCT often requires a large research team to complete study processes, such as obtaining informed consent, randomizing participants into treatment groups, documenting patient care, and collecting clinical data in a longitudinal manner, whereas the POC-CT embeds these processes into the EMR so that minimal additional personnel are required to conduct the study (Table 1). Second, using a fully integrated EMR is less intrusive on the usual clinical experiences of the patient and clinician than a traditional RCT because both individuals are accustomed to interfacing with the EMR during patient care.

To illustrate the role of the EMR, consider our CAI example as an investigation of the comparative effectiveness of 2 common manual therapy approaches⁸—Maitland grade 3 joint mobilizations^42,43 and Mulligan talocrural mobilizations with movement^44,45—for improving patient-reported outcomes and dorsiflexion range of motion in patients with this condition. Figure 1 provides an overview of study processes for such an investigation and highlights the role of the EMR in a POC-CT. Based on recommendations from the International Ankle Consortium,⁴⁶ patients with CAI can be identified primarily by their responses on a series of patient-reported outcome measures, including the Ankle Instability Instrument,⁴⁷ Cumberland Ankle Instability Tool,⁴⁸ and Foot and Ankle Ability Measure.⁴⁹ Thus, for our example, most of the screening process for potential study participants can be completed by the EMR. During a routine patient encounter (eg, preparticipation examination), patients would be asked to log into the EMR to update their demographic profiles and complete a series of patient-reported outcome measures. A screenshot of the patient view of the EMR during this screening process, beginning with the Ankle Instability Instrument, appears in Figure 2.

Figure 1.

Study processes and role of electronic medical record (EMR) within a point-of-care clinical trial. Abbreviations: ADL, Activities of Daily Living subscale; Sport, Sport subscale.

Figure 2.

Screenshot of a patient's view in an electronic medical record illustrating the screening process of a point-of-care clinical trial. The Ankle Instability Instrument is pictured here and is the first patient-report measure completed by the patient.

As the patient completes each patient-reported outcome measure, an algorithm in the EMR analyzes the responses in real time to determine the patient's eligibility based on current selection criteria for patients with CAI.⁴⁶ So if the patient answers yes to question 1 and at least 4 other questions on the Ankle Instability Instrument, has a score of less than 24 points on the Cumberland Ankle Instability Tool, and scores less than 90% on the Activities of Daily Living subscale and less than 80% on the Sport subscale of the Foot and Ankle Ability Measure,⁴⁶ then the patient is eligible for participation. If at any point during this initial screening process the patient does not meet the inclusion criteria (eg, score of 26 points on the Cumberland Ankle Instability Tool), the screening process ends, and the EMR displays a message that thanks the patient for completing the forms. Unlike a traditional RCT, which usually requires the patient to complete all patient-reported outcome measures, the POC-CT relies on the algorithm within the EMR to screen patients in real time and only asks them to complete the next patient-reported outcome measure if they met the inclusion criteria for the previous form (Figure 1). This real-time analysis limits the time each patient spends on the screening process, does not require the clinician or a research team member to facilitate the process, and improves the efficiency of the process.

The EMR then flags patients who fit the patient-reported inclusion criteria of the POC-CT and notifies the clinician. At this point, the clinician approaches the patient and performs a clinical examination, including range of motion, strength, balance, and functional activities, as dictated by usual care and current evidence.^50,51 If, based on the clinical examination and study protocol, the patient may benefit from a joint-mobilization treatment protocol, the clinician describes the study and asks if the patient would like to participate. Figure 3 is a screenshot of the clinician's view in the EMR that shows a summary of the screening and enrollment status of patients. If the patient agrees to participate in the study, he or she provides informed consent and is enrolled and automatically randomized by the EMR into 1 of 2 treatment groups based on a predefined algorithm^21,22 (see next subsection for additional information related to assignment of participants). If the patient decides not to participate in the study, the clinician proceeds with the usual care and determines the most appropriate intervention based on clinical judgment. To preserve the usual clinical experience of the patient and clinician, every attempt should be made to recruit each eligible patient seen by the clinician so that these processes become a standard component of patient care.^20,22 These processes are completed in real time through the EMR and without prompting by the patient, clinician, or member of a research team, thereby minimizing the overall burden of the study (Figure 1).

Figure 3.

Screenshot of a clinician's view in an electronic medical record illustrating the enrollment status of patients.

To preserve the clinician's focus on delivery of care, the EMR is designed to capture study measurements that characterize the effectiveness (or ineffectiveness) of an intervention as part of routine patient care.^21,22 Additional clinical measurements that are commonly obtained in efficacy studies but considered beyond routine care should be limited.^21,22 For the CER example study involving Maitland grade 3 joint mobilizations versus Mulligan talocrural mobilizations with movement, investigators would likely capture changes in health-related quality of life (HRQOL) via patient-reported outcome measures, such as the Disablement of the Physically Active⁵² or the Foot and Ankle Ability Measure, and measure changes in perceived joint instability. To facilitate the collection of these measures over the study period, automated notifications from the EMR can remind the patient and clinician that study-specific measurement periods are upcoming or due at predetermined time points (eg, 2 weeks, 1 year postintervention; Figure 1).

In addition to minimizing the intrusiveness of the study from the clinician's perspective, the use of the EMR should also limit the patient's burden.^21,22 Unlike the RCT, which typically requires patients to attend additional in-person follow-up visits for data collection, the POC-CT allows the patient to complete electronic forms or reports through the EMR from a remote location, such as home, or on a digital mobile device. Although data collection from a remote location differs from that in traditional research designs, the collection of patient-reported outcomes (eg, functional limitations or changes in HRQOL) is vital for assessing the effects of the intervention from the patient's perspective, determining clinical outcomes that are meaningful and important to the patient, and supporting the primary aims of CER efforts.^9–11

In summary, the EMR is essential to the POC-CT in providing a foundation to support studies at the point of care. When designed and used effectively for a POC-CT, the EMR can simultaneously support both patient care and research so that the typical clinical experiences of the patient and clinician are minimally affected. By using the EMR to screen, obtain consent from, randomize, and follow patients throughout the study protocol, many of the research processes of a traditional RCT are automated for a POC-CT. Thus, the effective use of an EMR during a POC-CT reduces the overall burden of the study, preserves the typical clinical experiences of the patient and clinician, and makes CER at the point of care more feasible.

The Use of Bayesian Statistics for an Adaptive Design

The second essential feature of the POC-CT is its Bayesian adaptive feature. Historically, frequentist statistics (based on the theory of infinite study replication) have dominated medical research methodologic design (eg, power analysis, sample-size recommendations) and statistical analysis (eg, hypothesis testing, P values).^53–56 The rigid approach of frequentist statistics has provided researchers with a reliable method of estimating population characteristics based on data collected from a sample of participants and has revolutionized medical research during the last century.^53,54,56 However, although the rigidity of this approach complements traditional research designs such as the RCT, experts^53–55 have argued that using the frequentist approach in CER is inefficient and impractical. Ironically, the greatest asset of the frequentist approach (its rigidity) may also be its greatest drawback for CER. For example, the real-life, uncontrolled setting of CER inherently produces more variability within the study which, under the frequentist approach, would require more participants to be enrolled in the study and result in increased study costs.^19,57 To cope with the inherent variability of CER, investigators have recommended the use of Bayesian statistics for these studies^{19,53,55,56,58–60} and, more recently, for research in athletic training.^61,62

The defining characteristic of the Bayesian approach is its flexibility.⁵⁶ Unlike the frequentist approach, which makes inferences about population characteristics based solely on findings from the study sample, the Bayesian approach allows investigators to include previous and new findings from outside the study sample.^53,55,56 These 2 approaches also differ logistically. When using a frequentist approach, prior information is considered only during the design of a study but not during the study or data analysis. A Bayesian approach provides a formal mathematical method for combining prior information with current information during the design and conducting of the study and as part of the analysis.^50,52 Thus, the Bayesian approach allows investigators to update or “adapt” the design of an ongoing study as new information is accrued from the study or becomes available from other sources.^50,52 This adaptive feature plays an important role in the randomization of participants in a POC-CT and incorporates practical, ethical, and patient care benefits.

To highlight the benefits of a Bayesian approach, let us consider our CAI CER study example. The research question could be, “What is the probability that Maitland grade 3 joint mobilizations are more effective for improving patient-reported outcomes in patients with CAI than Mulligan talocrural mobilizations with movement?” Without considering findings from previous studies and setting aside anecdotal evidence, the probability that 1 treatment protocol is superior to the other would be 50%. Using simple randomization, the allocation ratio for assigning study participants to 1 of the 2 treatment groups would be 1 : 1. That is, each participant would have an equal chance of being assigned to receive Maitland grade 3 joint mobilizations or Mulligan talocrural mobilizations with movement. Initially, the Bayesian adaptive approach is similar to that of an RCT. However, a Bayesian approach allows adaptation of the allocation ratio as information is collected during the course of the study.

For example, after a predetermined number (“batch”) of participants has been enrolled, the data collected to that point are analyzed in real time to determine if 1 intervention is demonstrating better results than the other.^22,53 Based on prespecified criteria (eg, change in patient-reported outcomes scores that exceed the minimal clinically important difference), the randomization of study participants into treatment groups may be adjusted to favor what appears to be the better intervention.^63,64 Using our CAI example, if changes in the HRQOL scores after the first batch of 10 participants suggest that Maitland grade 3 joint mobilizations are resulting in better study outcomes than the Mulligan talocrural mobilizations with movement, the allocation ratio may be readjusted to 3 : 2 (60% for the Maitland grade 3 joint-mobilization group, 40% for the Mulligan talocrural mobilization-with-movement group).¹⁹ This updating process would continue throughout the study until a predetermined endpoint is achieved based on a predefined “stopping rule” with an acceptable type 1 error rate (false positive or reporting a difference when there is none).^20,22,24,25

Of course, the algorithm driving the adaptive process is not a simple count of “wins” and “losses.” It must incorporate potential predictors of treatment and failure, including known confounders and effect modifiers or unknown factors that may be identified by retrospectively reviewing the participants' complete medical records within the EMR. Propensity scores (model-based estimates of the probability of assignment to intervention arms based on all covariates) may be used in an adaptive manner.⁶⁵ In some cases, neither intervention (eg, Maitland grade 3 joint mobilization, Mulligan talocrural mobilization with movement) will be shown to be better, and the study is stopped. When that happens, other factors, such as the cost and time burden of the interventions, must be considered to determine the comparative effectiveness of the specified interventions.^19,22 For our example, if the Maitland grade 3 joint mobilizations and Mulligan talocrural mobilizations with movement are equally effective in improving HRQOL, then the clinical recommendation would favor the treatment that would require less time or cost associated with the treatment.

From a research perspective, the integration of prior and new data can help reduce the number of patients needed in the study (ie, sample size) to determine effectiveness and contain overall costs.^25,58,66 In addition, the Bayesian adaptive nature of the POC-CT can offer ethical benefits related to protecting human participants. Any study involving 2 or more interventions will inherently pose an ethical concern of exposing patients to an inferior intervention.⁶⁷ However, within a Bayesian adaptive design, the constant updating of the allocation in favor of the better treatment helps to direct more patients to the superior intervention and fewer patients to the inferior intervention.²⁵ Further, the Bayesian adaptive design allows investigators to incorporate new interventions into the design of an ongoing study.¹⁹ In our CAI example, the initial interventions were Maitland grade 3 joint mobilizations and Mulligan talocrural mobilizations with movement. If, during our study, a new intervention (eg, multimodal rehabilitation) shows promising results for the treatment of patients with CAI, then the flexibility of the Bayesian design allows the new intervention to be added to the ongoing study. This type of flexibility is particularly useful in the health care system because of the regular introduction of new and promising interventions.¹⁹ In fact, recent discussions have emphasized the need for more robust research designs, such as platform trials,^68–70 that can add and remove interventions in an ongoing manner to accommodate ever-changing treatment options and identify the best treatment for a specific condition.

Because it offers practical, ethical, and patient care advantages over the frequentist study design approach, the Bayesian adaptive design is becoming more common in medical research, and its use has been encouraged in the sports medicine community.^61,62 The advantages of this design align well with CER objectives and offer investigators the flexibility needed to conduct research at the point of care in a scientifically rigorous manner. Although the Bayesian approach is relatively new to the sports medicine community, athletic training researchers should consider its advantages, particularly when conducting a CER investigation.

Clinical Benefits of the POC-CT: A Path to a Learning Health Care System

Of the many benefits the POC-CT offers to scientific investigation, its promise lies in its potential to improve clinical practice and advance EBP efforts. One of the major barriers to current EBP initiatives is the inability to rapidly incorporate research evidence into routine patient care. However, this barrier can be addressed by using a customized EMR for routine patient care and incorporating a Bayesian adaptive design.

From a clinical viewpoint, the ability to readjust the allocation schedule of participants based on accumulated information increases the probability that more patients are assigned to the favored intervention over the course of the study.^19,22 Using our CAI example, the readjustment of group assignment to favor Maitland grade 3 joint mobilizations (assuming they displayed superior effectiveness) after the first batch of participants would ensure that more patients are assigned to the intervention that is producing better patient care outcomes. This real-time integration of accumulating research evidence is the essence of EBP and ensures that evidence is incorporated in routine patient care.¹⁹

If a clinician is participating in a POC-CT and randomizing every willing patient, the adaptive allocation increases the probability that the majority of patients will receive a superior intervention.²⁵ The size of a participant batch in a POC-CT can vary. Incorporating Bayesian algorithms into a fully integrated EMR could introduce a layer of additional “intelligence” into the clinical decision-making process, whereby a “batch” may comprise 1 participant and the allocation ratio would be adjusted in real time after each participant. This approach would have a profound effect on patient care by facilitating the establishment of a “learning health care system,” whereby clinicians learn from each patient care experience and refine practice patterns in real time.^22,25 The consistent refinement and updating of the best available evidence is the essence of evidence-based medicine and improvement in patient care.

The use of an EMR and Bayesian adaptive design may prompt concerns that the POC-CT is replacing clinical expertise with computer algorithms. In a POC-CT, just as in an RCT, the treatments being studied are assumed to be effectively equivalent to one another (“clinical equipoise”)⁶⁷; based on the current evidence and clinical consensus, one intervention is not supported over another.⁶⁷ Thus, the EMR and Bayesian adaptive feature of a POC-CT should not be viewed as technology replacing the clinician but as tools aiding clinicians in their patient care decisions. In short, just as evidence in EBP does not supersede clinical expertise, features of the POC-CT do not supersede established clinical consensus or best practices.

In addition to preserving the patient's and clinician's usual clinical experience, the use of an EMR can accelerate the accumulation of CER evidence when the EMR is connected to a larger network. For example, the Massachusetts Veterans Epidemiology Research and Information Center research group is currently conducting a feasibility study comparing different insulin interventions using the POC-CT design at its Boston Veterans Affairs site.²² Preliminary findings indicated that the POC-CT is a feasible approach and, because the infrastructure of obtaining informed consent, randomizing patients, documenting patient notes, and collecting outcomes data is already in place within the EMR, the research group can easily expand the study to either a small subset or all of the 1500 Veterans Affairs-affiliated sites using the same EMR for routine patient care documentation.^20,22 This interconnectedness among multiple sites through a centralized EMR provides researchers with the opportunity to conduct multisite POC-CTs, quickly aggregate data from a variety of diverse sites, and establish generalizable evidence to guide patient care.^12,13,15

Although the percentage of athletic trainers who document patient care through an EMR is unknown, the Athletic Training Practice-Based Research Network (AT-PBRN)¹⁵ provides an EMR infrastructure that is similar in concept to the EMR infrastructure of the Veterans Affairs health care system. As an affiliate practice-based research network of the Agency for Healthcare Research and Quality, the AT-PBRN was developed to support successful POC-CTs, and it connects multiple clinical practice sites (currently more than 70) across the country through a Web-based EMR (CORE-AT EMR).¹⁵ The EMR is fully integrated into routine care so that patients and clinicians regularly interact with it, and its customizability allows algorithms to be embedded that support aspects of the POC-CT, including the collection of patient-reported outcome measures and incorporation of a Bayesian adaptive design. Preliminary findings from the AT-PBRN suggested that patient care data can be reliably collected through the Web-based EMR and that data can be aggregated across sites to begin answering questions that are clinically relevant to athletic training practice.^15,71,72 Although the POC-CT is a novel approach to CER, the athletic training profession may be structurally positioned to begin conducting multisite research through the AT-PBRN, or other similar networks, using the POC-CT design.

CHALLENGES AND CONSIDERATIONS

As with any study design, potential challenges are associated with the POC-CT. The POC-CT offers a way for researchers to design more pragmatic studies, yet the optimization of internal validity is crucial. Potentially confounding factors must be documented and addressed if a POC-CT is to produce high-quality evidence.

As suggested by best practices for CER, it is also important to engage patient and clinicians in the planning and design process to ensure success of the study.⁷³ Patients and clinicians may identify practical real-life barriers that researchers had not considered. For example, the POC-CT aims to limit the patient burden by collecting study outcomes electronically, but Internet access may not be readily available to patients in some low socioeconomic and rural areas. Although recent data⁷⁴ showed that the majority of individuals in the US had access to the Internet (eg, 81% of adults making less than $30 000 a year and 78% of adults in rural communities), identifying alternative ways to collect study outcomes from patients who lack Internet access will be important to the success of each POC-CT.

The POC-CT aims to minimize the intrusiveness of a study on the clinical experiences of the patient and clinician; however, an ongoing study will inevitably impede the usual workflow. Thus, it will be important to gather input from clinicians to ensure that the ongoing study minimizes the overall effects on their clinical practice. Like all clinical research studies, the POC-CT relies on the comprehensive documentation of study findings by the clinician providing patient care. Recent evidence⁷⁵ suggested that athletic trainers did not document in a comprehensive manner. Common barriers to documentation include inadequate time and not knowing what to document.⁷⁶ From a POC-CT perspective, these barriers highlight both the need to design the EMR in a manner that offers clinicians an easy way to document their patient encounters during the study and the importance of training the clinicians on how and what to document. This is similar to a traditional RCT in which clinicians must be properly trained so that procedures, such as data collection and documentation and intervention delivery, are consistent across clinicians and study sites.⁷⁷ In short, although the POC-CT offers a method of studying interventions at the point of care, many of the considerations and barriers associated with the traditional RCT must also be considered when designing a POC-CT.

CONCLUSIONS

As health care professionals seeking to provide the best possible patient care, athletic training researchers and clinicians must work together to facilitate EBP. One of the greatest barriers to adopting EBP is a lack of practice-based evidence that can be applied to an athletic trainer's decision-making process. Because the POC-CT encompasses the basic tenets of practice-based research, in which studies are conducted at the point of care, in real-life settings, and during the routine course of clinical practice, it is an ideal mechanism for facilitating the collection of patient care data and providing practice-based evidence. The POC-CT combines the better elements of both RCT and observational study designs to overcome the flaws inherent in each. In essence, the POC-CT is a randomized, quasi-observational study that helps to minimize bias and implementation at the point of care while limiting cost and improving the generalizability of the findings.^22,25 Thus, because the POC-CT encompasses the basic tenets of practice-based research, its findings may be incorporated into clinical practice more readily than findings from traditional research designs.^{20–22,24,25} By using a fully integrated EMR and a Bayesian adaptive design, a POC-CT can be integrated seamlessly into daily practice in such a way that new patients enrolled in the study will be more likely to receive a superior intervention. This research design appears to be a promising new approach that the sports medicine community should consider and use to support CER and EBP initiatives.