Table 1.

Characteristics of RCTs, PrCTs, and Real-World Observational Studies^{2,7,9,19,21,97,98}

	RCTs	PrCTs	Real-World Observational Studies
General information	Prospective design Usually phase 2 or 3 clinical trials Investigational drug vs placebo and/or an active comparator(s) Provides “gold standard” evidence for safety and/or efficacy of a drug	Prospective design Features of RCTs and real-world observational studies Provides suggestive real-world evidence on a therapeutic intervention’s value in real-world clinical practice while maintaining the strength of initial randomized treatment	Often retrospective design; can be prospective or a combination of the two Conducted using real-world data from administrative health databases, insurance and claims databases, and registries
Study population	Highly selective population(s) based on defined inclusion (eg, age, sex, severity of disease, concomitant medications, and willingness to participate) and exclusion (eg, comorbidities, risk factors, and prior use of study drugs or other confounding medicines) criteria, with exclusions applied to minimize the interference of potential effect modifiers and maximize the probability of demonstrating a treatment effect	Broad population(s) from community-based clinics Can include “all-comers” with the disease under study	Potentially a very large population Less stringent selection criteria Representative of patients in routine clinical practice likely meeting the exclusion criteria in RCTs (eg, comorbidities, nonadherence, crossover to alternative medication, and polypharmacy)
Randomization	Yes	Usually	No
Comparability	Sample is randomized for uniform distribution of all known and unknown factors affecting patient prognosis, thus ensuring that differences in outcomes are attributable to intervention(s) NOTE: Baseline differences may still occur in RCTs with smaller sample sizes	Diverse populations taking new or investigational therapies are enrolled Randomization helps ensure comparable treatment groups Limited generalizability of results owing to lax adherence measures, unrestricted treatment changes, and lack of objective endpoints	Physician preferences, formulary status, or costs may restrict new drug prescriptions in difficult to treat or treatment-resistant patients, potentially biasing outcomes when comparing different treatments Although statistical adjustments can be attempted for known variables and comparison groups can be matched using propensity scores, adjustments for unknown variables cannot be made
Study setting/data sources	Research centers, specialized trial centers, and secondary or tertiary hospitals Highly controlled environment	Usually community-based medical clinics	Diverse routine clinical practice settings, including primary care settings Large healthcare databases Registries
Assessment burden	Demanding schedule of maintaining records (eg, home diaries) and frequent study visits	Periodic telephone or clinic evaluations and recall questionnaires Few home diaries and visits Low follow-up demands	Regular, real-world physician–patient interactions
Data collection	Per-protocol using validated efficacy endpoints such as PROs Daily electronic e-diaries Predefined scheduled visits Usually 6–10 follow-up visits, with multiple objective endpoints and PROs assessed at each visit	Subjective questionnaires or PROs often used instead of objective procedure-based tests PROs provide suggestive evidence but can be prone to errors resulting from patient bias and potential lack of validation Objective tests, such as e-diary data, laboratory tests, and sequential lung function tests, are not generally obtained	Usually through hospital- or clinic-based registries, where visits are per standard of care, or insurance-based claims Some modes of data collection (eg, spirometry for COPD diagnosis or assessment of treatment effectiveness) may not be used in routine visits Overlapping/mistaken data (such as diagnoses of both COPD and asthma) may be entered in e-health record databases. Some information may be unavailable because data were not entered in the e-health database
Adherence	Strictly monitored by daily diaries or dose counts Adherence is often near-complete or maximum attainable because of continuous patient contact (eg, detailed patient education, reminders, home visits)	Adherence is loosely monitored with intermittent dosing acceptable Annual number of prescription fills may be estimated Adherence may be low and is reflective of real-world clinical scenarios	Annual prescription fills are often measured Adherence is usually much lower than that achieved in RCTs
Discontinuations/withdrawals	Patients with poor adherence or who switch therapies are discontinued	Patients with poor adherence or who switch therapies are included in the analysis	Patients with poor adherence or who switch therapies are included in the analysis
Statistical design and comparators	Usually, single- or double-blinded treatments are administered to prevent patient and clinician selection bias Statistics prespecify numbers of patients needed and power to demonstrate superior efficacy for primary endpoints Standard of care or placebo and/or an active comparator are used for treatment comparison Normally both per-protocol and intention-to-treat analyses are reported	Treatments are usually open-label Standard of care or an active treatment comparator is used in superiority trials A highly effective comparator is used in noninferiority trials Placebo is typically not dispensed Normally both per-protocol and intention-to-treat analyses are reported	Treatments are open-label by prescription Usual care, which differs by patient segment and country, can vary substantially across study centers
Follow-up data	Follow-up duration is usually short with frequent visits, often every 8–12 weeks; can be longer	Follow-up duration may be long, and frequency is usually sparse with as few as 2 or 3 mandatory visits over a year	Follow-up duration may be substantially long, often ≥1 year, and frequency of visits is determined by patients and/or physicians per usual practice
Outcomes	Prospective primary, secondary, and other efficacy and safety or pharmacokinetic endpoints are prespecified, statistically powered, and collected to objectively measure improvements vs control/comparators Validated PRO questionnaires are used Health outcomes data are obtained prospectively and concurrently, usually through daily e-diaries or paper diaries and frequent clinic visits Resource utilization data (eg, unscheduled clinic visits, emergency department visits, and hospitalizations) and risk vs benefit can be assessed Efficacy and safety outcomes assessed should be biologically meaningful	Prospective primary and secondary endpoints are prespecified for superiority or noninferiority analyses Few objective outcomes such as hospitalization and mortality, and some technician-administered outcome tests may be completed Patient questionnaires are often used, which are not always validated Rather than contemporaneous e-diaries, data are usually collected retrospectively via periodic recall questionnaires conducted via telephonic interviews/conversations	Endpoints are retrospectively selected to measure effectiveness, safety, patient experience, PROs, resource utilization, risk vs benefit (relative effectiveness), etc, as determined by a study analysis plan prepared a priori before data analysis Long-term effectiveness can be assessed, and rare adverse events may be identified Outcomes reported may be meaningful for decision-making in routine clinical practice
Data quality	Usually very good	Variable	Concerns about sensitivity and specificity of data are present, given the retrospective, nonrandomized design and possible bias in matching algorithms
Generalizability	Results are usually reproducible in the population studied, and support drug regulatory approval Results are applicable to patient populations with disease characteristics same or similar to those included in RCTs	Findings may be hypothesis generating or suggestive Can establish effectiveness in broad real-world populations. However, because of variable adherence, infrequent visits, and limited questionnaire-based endpoints, confirmation by RCTs may also be needed	Results are applicable to a broad range of healthcare databases, may apply to real-life treatment users, and may be generalizable to routine clinical practice These studies are post hoc analyses, and require confirmatory RCTs or replicate observational studies before results can be broadly accepted
Validity	Randomization and nondifferential assignment are attempted to make the treatment groups comparable at baseline and ensure that the results are valid and not confounded High level of scientific accuracy of conclusions is ensured by strict adherence, monitoring, and restrictions on disallowed medications, as well as serial, contemporaneous collection of objective endpoints	Prospective design and randomization add credibility to these findings Findings are suggestive because of the weak controls on adherence, confounding or alternative therapies, and the limited endpoints assessed Broader patient populations are enrolled If superiority is demonstrated, these trials can provide compelling data for clinicians and payers Findings of “noninferiority” are more difficult to generalize because poor adherence, crossing over between therapies (if allowed), or soft endpoints can lead to scientific uncertainty	Risk vs benefit assessment among treatment groups may be confounded by incomparability of clinical characteristics at baseline because of differential prescribing Results may not be internally valid and need to be interpreted with caution
Precision	Results may be reasonably precise in RCTs of large sample size (>1000 patients)	Precision is sacrificed to ensure higher cost-effectiveness and feasibility Evidence of superior efficacy compared with usual care/standard therapies can be demonstrated in relatively small studies Larger samples are needed for adequate power in “noninferiority” trial designs because real-life patients may not always be highly responsive or adherent to treatments	A large sample size is likely to increase the precision of the study
Cost	High cost per patient	Intermediate cost per patient Studies may be more expensive in total because larger numbers of patients are required, and real-world patients may be less sensitive to drug effects than highly selected patients	Low cost per patient
Value	Are of value for controlled scientific analysis of treatment effectiveness Required for regulatory approval	Provide suggestive value to regulators and payers May broaden populations appropriate for clinical treatments	Traditionally of value to payers

Abbreviations: COPD, chronic obstructive pulmonary disease; e-diary, electronic diary; e-health, electronic health; PrCT, pragmatic clinical trial; PRO, patient-reported outcome; RCT, randomized controlled trial.