Table 4.

Various Methods for Developing Evidence for CER and Personalized Medicine

Evidence Type	Description	Advantage or Limitations for CER	Advantage or Limitations for Personalized Medicine	Example
Randomized controlled trials	Gold standard for establishing an intervention's causal effect on patient health outcomes in predefined, carefully controlled conditions Can establish efficacy and safety for drugs and some medical devices Can establish utility for diagnostic tests Comparator is typically placebo or no treatment Typically patient-level randomization and analyses	Usually does not generate evidence for real-world effectiveness in diverse patient populations If cluster randomization is used with an active comparator (usually standard of care), this can generate high-quality CER data	Subgroup analyses and analyses focusing on variability rather than those that average main effects (ie, identify groups of individuals that do well or poorly) can identify hypothesis generating analyses for potential high-impact groups (which can be based on genomic data, if collected) to be used in future randomized controlled trials to establish causality, efficacy, safety, and/or clinical utility	MINDACT trial uses the MammaPrint 70-gene profile37; TAILORx trial uses the Oncotype Dx 21-gene assay38 To provide prospective evidence about the clinical utility of these tests, to determine the need for adjuvant chemotherapy, and to identify women who are unlikely to benefit from chemotherapy
Pragmatic clinical trials	Standard for establishing an intervention's effect on patient health outcomes in real-world settings and populations (diverse/heterogeneous) Outcomes include a broad range of measures that are relevant to decision makers, clinicians, and patients Comparator is typically standard of care Clinician-, clinic-, or system-level randomization and analyses (cluster)	Results are generalizable and useful for real-world effectiveness Effect sizes may be small (but still clinically meaningful) when comparing the impact of active treatments; thus, to generate clinically meaningful and statistically significant results, very large sample sizes may be required (with their associated high costs)	Including genomic data in study measures will begin to generate effectiveness outcomes for personalized medicine Currently, this type of data is not commonly captured. To do so will require a health information technology data infrastructure	Family History Project uses a hybrid type 2 effectiveness and implementation model39 To assess the impact and operating characteristics of a family history decision support intervention for cancer screening on patient and provider behaviors
Adaptive clinical trials	Strives for the most effective use of an intervention (optimized) to maximize its effect on patient health outcomes in real-world settings and populations (diverse/heterogeneous) Outcomes include a broad range of measures that are relevant to decision makers, clinicians, and patients Measures include data for assessing the population distribution probability of an outcome at baseline (prior distribution) and after the intervention (posterior distribution) Data are analyzed frequently (at set intervals) and used to inform the study direction Comparator is baseline (prior to intervention) and standard of care	Because the intervention's form or use is modified during the study to maximize its impact, adaptive clinical trials do not usually generate adequate effectiveness data although they do capture some They can be used to inform future effectiveness studies, once optimization has occurred	Since a goal of adaptive clinical trials is to modify study design to optimize an intervention's effect, they not only gather data on personalized medicine characteristics (including genomic data, if collected) but also shift resources to focus enrollment on these groups Results can inform the design of future definitive personalized medicine randomized controlled trials or effectiveness studies	I-SPY-2 breast cancer trial systematically assigns phase II agents while rapidly learning about the impact of these agents on patients based on specific molecular characteristics of their tumors. Drugs that show beneficial changes will be preferentially assigned and will move through the trial more rapidly. Agents that do not show the likelihood of improved response rate associated with any predefined biomarker will be dropped from the trial40
Observational cohort studies and registries	Generates hypotheses about linkages between an intervention and patient health outcomes in real-world settings and populations (diverse/heterogeneous) There is no randomization; all interventions are provided in the course of normal clinical care Outcomes typically include those available in administrative databases, such as clinical variables and health care use Measures are usually longitudinal and available at unstructured time points	Very useful for generating CER hypotheses but does not establish causality How useful it is depends on the quality and extent of data available for collection	Very useful for generating personalized medicine hypotheses and for performing biomarker development and initial validation studies but does not provide final proof for the link between personalized medicine characteristics/subgroups and health outcomes How useful it is depends on the quality and extent of data available for collection (including whether genomic data are available) Results can inform the design of future definitive personalized medicine randomized controlled trials or effectiveness studies	TCGA seeks to identify somatic mutations that are specifically associated with the type of cancer being sequenced and other types of larger-scale genomic changes, such as copy number changes and/or chromosomal translocations, and gene expression that contribute to cancer development and/or progression41

Abbreviations: CER, comparative effectiveness research; MINDACT, Microarray in Node-Negative and 1-3 Node-Positive Disease May Avoid Chemotherapy trial; TCGA, The Cancer Genome Atlas.