Comparing Randomized Controlled Trials and Real-World Studies in Chronic Obstructive Pulmonary Disease Pharmacotherapy

Abstract

Analytic epidemiological studies cover a large spectrum of study methodologies, ranging from noninterventional observational studies (population-based, case–control, or cohort studies) to interventional studies (clinical trials). Herein, we review the different research methodologies or study designs and discuss their advantages and disadvantages in the context of chronic obstructive pulmonary disease (COPD) pharmacotherapy. Although randomized controlled trials (RCTs) are considered the “gold standard” for evaluating the efficacy and safety of an intervention, observational studies conducted in a real-world scenario are useful in providing evidence on the effectiveness of the intervention in clinical practice; understanding both efficacy and effectiveness is important from the clinician’s perspective. Pragmatic clinical trials that use real-world data while retaining randomization bridge the gap between explanatory RCTs and noninterventional observational studies. Overall, different study designs have their associated advantages and disadvantages; together, findings from all types of studies bring about progress in clinical research as elucidated through examples from COPD research in this paper.

Introduction

Clinical research studies can be broadly classified as descriptive (eg, ecological studies or case reports) or analytic (Figure 1).¹ Analytic studies span a large spectrum, ranging from noninterventional and observational real-world studies to interventional studies.^1–5 Observational studies include cross-sectional, case–control, and longitudinal cohort studies,⁶ and interventional studies include explanatory randomized controlled trials (RCTs) and pragmatic clinical trials (PrCTs), which bridge the gap between explanatory RCTs and real-world observational studies.^2,7 In addition to the difference in study types, study designs vary in many respects (eg, methodologies, temporal relationship, number of subjects enrolled, eligibility criteria, characteristics of included subjects, interventions administered, duration, assessments, and outcomes). These variations lead to inherent advantages and disadvantages; however, ultimately, the various study types and resultant data complement each other and form the building blocks of the research process.

Figure 1.

Overview of study designs.

Here, we review the different study types and discuss their advantages and disadvantages (also summarized in the supplementary video), in general and in the context of chronic obstructive pulmonary disease (COPD) pharmacotherapy, for the benefit of clinicians with more limited research experience.

Process of Drug Development

New drug development is a stepwise, rigorous, and prolonged process,⁸ typically involving preclinical studies, followed by phase 1 to 3 clinical trials, and phase 4 trials and other observational studies, which subsequently verify the results of phase 1 to 3 trials (Figure 2). The process may differ slightly for expanded indications (eg, additional age groups, other endpoints, and new diseases) for previously approved drugs.

Figure 2.

Research process for new drug development: possible sequence of research designs used.

Abbreviations: RCT, randomized controlled trial.

Study Designs

Explanatory Clinical Trials

Clinical trials are prospective studies in which patients receive an intervention. The designs of such studies increase in complexity with each phase of the study (Figure 2).^8,9 In general, phase 1 trials include small, nonrandomized, and noncomparative studies, whereas phase 2 trials may include randomized and comparative/controlled interventional studies. Early phase 1 and 2 trials often test or modify initial hypotheses, which are further evaluated in phase 3 and 4 trials.

Phase 1 trials are primarily safety and pharmacokinetic/pharmacodynamic trials. These trials include a small number (20–80) of healthy volunteers who receive single or multiple doses of the investigational drug to determine dosing; document absorption, distribution, metabolism, and excretion (sometimes referred to as ADME studies); and identify short-term adverse effects.⁹

In phase 2 trials, safety and preliminary efficacy (the extent to which a drug can bring about its intended effect under ideal circumstances, such as in an RCT¹⁰) are assessed.⁹ Phase 2 trials are often blinded RCTs and include approximately 100 to 300 patients (ie, people with the disease under investigation) who receive either one or more doses of the investigational drug and/or standard-of-care treatment and/or placebo over a period of 1 to 4 weeks (to achieve steady state). Generally, the objective of the dose-ranging phase 2 trials is to determine the optimal dose(s) for evaluation in larger phase 3 trials in addition to assessing safety and preliminary efficacy.⁹

Phase 3 trials, comprising RCTs with specified eligibility criteria, are considered the “gold standard” for establishing the safety and efficacy of a drug.^3,4 These RCTs are large scale—approximately 1000–3000 patients; can be single-, double-, or triple-blinded; and are often conducted over a prolonged period of time.⁹ Patients are randomized to receive one or more doses of the investigational drug, placebo, and/or a commercially available comparator agent for weeks, months, or even years. Treatment safety, efficacy, and adherence are monitored using objective, validated endpoints with the help of home diaries and periodic assessments during visits at regular intervals. The number of patients and duration of the trial may vary depending upon the disease under consideration, stage of drug development, duration of previous trials, and chronicity of the disease. For example, if the primary endpoint is a reduction in exacerbations of COPD, the frequency of which varies according to season, a year-long study is generally preferred to reduce the confounding effect of seasonality. These trials aim to meet regulatory agency approval requirements to evaluate the long-term safety and efficacy of clinically relevant doses of the investigational drug compared with placebo and/or a comparator agent, substantiated by sufficient statistical power generated with a large number of patients.

Phase 4 trials are usually prospective trials with active comparators (sometimes called head-to-head efficacy trials) or are open-label, noninterventional, observational studies conducted after regulatory approval; occasionally, these trials are agreed to by the study sponsor and are often required by regulatory approval authorities.^11–13 The objective of these trials is to collect additional information about the safety (long-term risks and rare adverse events), efficacy of a drug on an expanded indication (eg, exacerbation reduction), effectiveness (the extent to which a drug achieves its intended effect in the usual clinical setting¹⁰), and optimal use of the investigational drug in the general patient population, as well as to evaluate the investigational drug in special patient populations that are usually excluded or difficult to include and follow-up in phase 3 RCTs.¹¹

For rare diseases or orphan drugs, the number of phase 2 to 4 trials and the number of participants in each trial may be substantially reduced, and alternative clinical trial designs may be acceptable.¹⁴

Usually, statistical analyses in phase 2, 3, and 4 trials are conducted to assess whether or not the investigational drug has greater efficacy than placebo or an active control (a “superiority” design). Superiority evaluations require a prospective design with adequate statistical power, reliable objective endpoints, and adequate patient adherence. In superiority trials, usually an intent-to-treat (ITT) or an all-data-collected analysis is conducted¹⁵ by using appropriate analysis methods (eg, mixed models repeated measures [MMRM], analysis of variance [ANOVA], analysis of covariance [ANCOVA], Cox proportional hazard regression, or logistic regression¹⁶) to evaluate efficacy; however, other statistical methods may also be used. Traditionally, safety is assessed by the proportions of patients with adverse events, calculated as cumulative (at the end of the trial period) or cross-sectional (at each visit) percentages.¹⁶ Regulatory authorities often require two or more duplicate 12- to 24-week phase 3 efficacy trials and at least one phase 3 year-long safety trial (termed “pivotal trials”) to adequately characterize the safety and efficacy of a new drug for approval. Given the seasonality of COPD exacerbations, long-term phase 3 trials of up to 12 months duration are recommended for the evaluation of the effect of new investigational drugs on this outcome or for the expansion of approved indications for already commercially available drugs.¹⁷

Case Reports or Case Series

Reports of individual cases or a series of cases provide retrospective safety and efficacy details, as well as other clinical parameters (eg, quality of life [QoL]) derived from cases of interest in their natural clinical setting;¹⁸ they can be based on either on-label (supporting the approved indication) or off-label (demonstrating a potential new or expanded indication) use of a drug. These cases are easy to report and can generate new research questions, although the generalizability of their findings is limited because of selection bias and lack of controls.¹⁸

Real-World Observational Studies

Real-world observational studies include large-scale cross-sectional, cohort, and case–control studies that do not employ randomization⁶ and may be population based (Table 1). Case–control studies are retrospective, while cohort studies may be prospective or retrospective. In these studies, investigators solely observe treatment effects, generally using administrative health databases, claims databases, or registries;¹⁹ investigators have no control over the medical management of the patient or the data collected. These studies are characterized by the enrollment of real-world patients, lax controls on treatment adherence, use of concomitant medications or alternative therapies, and selection of endpoints for optimum feasibility; moreover, the endpoints may not be suitably objective or validated. Besides safety and clinical effectiveness, cost-effectiveness and other economical outcomes may also be assessed.¹⁹ When comparisons are made between the clinical effectiveness and/or safety of two or more different medications, propensity matching on selected clinical characteristics may be performed in an effort to minimize the impact of differences in these patient-related features. These studies are relatively cheaper and are especially useful when the disease of interest is rare, but can also be used when the disease of interest is common.

Table 1.

Characteristics of RCTs, PrCTs, and Real-World Observational Studies^{2,7,9,19,21,97,98}

	RCTs	PrCTs	Real-World Observational Studies
General information	Prospective design Usually phase 2 or 3 clinical trials Investigational drug vs placebo and/or an active comparator(s) Provides “gold standard” evidence for safety and/or efficacy of a drug	Prospective design Features of RCTs and real-world observational studies Provides suggestive real-world evidence on a therapeutic intervention’s value in real-world clinical practice while maintaining the strength of initial randomized treatment	Often retrospective design; can be prospective or a combination of the two Conducted using real-world data from administrative health databases, insurance and claims databases, and registries
Study population	Highly selective population(s) based on defined inclusion (eg, age, sex, severity of disease, concomitant medications, and willingness to participate) and exclusion (eg, comorbidities, risk factors, and prior use of study drugs or other confounding medicines) criteria, with exclusions applied to minimize the interference of potential effect modifiers and maximize the probability of demonstrating a treatment effect	Broad population(s) from community-based clinics Can include “all-comers” with the disease under study	Potentially a very large population Less stringent selection criteria Representative of patients in routine clinical practice likely meeting the exclusion criteria in RCTs (eg, comorbidities, nonadherence, crossover to alternative medication, and polypharmacy)
Randomization	Yes	Usually	No
Comparability	Sample is randomized for uniform distribution of all known and unknown factors affecting patient prognosis, thus ensuring that differences in outcomes are attributable to intervention(s) NOTE: Baseline differences may still occur in RCTs with smaller sample sizes	Diverse populations taking new or investigational therapies are enrolled Randomization helps ensure comparable treatment groups Limited generalizability of results owing to lax adherence measures, unrestricted treatment changes, and lack of objective endpoints	Physician preferences, formulary status, or costs may restrict new drug prescriptions in difficult to treat or treatment-resistant patients, potentially biasing outcomes when comparing different treatments Although statistical adjustments can be attempted for known variables and comparison groups can be matched using propensity scores, adjustments for unknown variables cannot be made
Study setting/data sources	Research centers, specialized trial centers, and secondary or tertiary hospitals Highly controlled environment	Usually community-based medical clinics	Diverse routine clinical practice settings, including primary care settings Large healthcare databases Registries
Assessment burden	Demanding schedule of maintaining records (eg, home diaries) and frequent study visits	Periodic telephone or clinic evaluations and recall questionnaires Few home diaries and visits Low follow-up demands	Regular, real-world physician–patient interactions
Data collection	Per-protocol using validated efficacy endpoints such as PROs Daily electronic e-diaries Predefined scheduled visits Usually 6–10 follow-up visits, with multiple objective endpoints and PROs assessed at each visit	Subjective questionnaires or PROs often used instead of objective procedure-based tests PROs provide suggestive evidence but can be prone to errors resulting from patient bias and potential lack of validation Objective tests, such as e-diary data, laboratory tests, and sequential lung function tests, are not generally obtained	Usually through hospital- or clinic-based registries, where visits are per standard of care, or insurance-based claims Some modes of data collection (eg, spirometry for COPD diagnosis or assessment of treatment effectiveness) may not be used in routine visits Overlapping/mistaken data (such as diagnoses of both COPD and asthma) may be entered in e-health record databases. Some information may be unavailable because data were not entered in the e-health database
Adherence	Strictly monitored by daily diaries or dose counts Adherence is often near-complete or maximum attainable because of continuous patient contact (eg, detailed patient education, reminders, home visits)	Adherence is loosely monitored with intermittent dosing acceptable Annual number of prescription fills may be estimated Adherence may be low and is reflective of real-world clinical scenarios	Annual prescription fills are often measured Adherence is usually much lower than that achieved in RCTs
Discontinuations/withdrawals	Patients with poor adherence or who switch therapies are discontinued	Patients with poor adherence or who switch therapies are included in the analysis	Patients with poor adherence or who switch therapies are included in the analysis
Statistical design and comparators	Usually, single- or double-blinded treatments are administered to prevent patient and clinician selection bias Statistics prespecify numbers of patients needed and power to demonstrate superior efficacy for primary endpoints Standard of care or placebo and/or an active comparator are used for treatment comparison Normally both per-protocol and intention-to-treat analyses are reported	Treatments are usually open-label Standard of care or an active treatment comparator is used in superiority trials A highly effective comparator is used in noninferiority trials Placebo is typically not dispensed Normally both per-protocol and intention-to-treat analyses are reported	Treatments are open-label by prescription Usual care, which differs by patient segment and country, can vary substantially across study centers
Follow-up data	Follow-up duration is usually short with frequent visits, often every 8–12 weeks; can be longer	Follow-up duration may be long, and frequency is usually sparse with as few as 2 or 3 mandatory visits over a year	Follow-up duration may be substantially long, often ≥1 year, and frequency of visits is determined by patients and/or physicians per usual practice
Outcomes	Prospective primary, secondary, and other efficacy and safety or pharmacokinetic endpoints are prespecified, statistically powered, and collected to objectively measure improvements vs control/comparators Validated PRO questionnaires are used Health outcomes data are obtained prospectively and concurrently, usually through daily e-diaries or paper diaries and frequent clinic visits Resource utilization data (eg, unscheduled clinic visits, emergency department visits, and hospitalizations) and risk vs benefit can be assessed Efficacy and safety outcomes assessed should be biologically meaningful	Prospective primary and secondary endpoints are prespecified for superiority or noninferiority analyses Few objective outcomes such as hospitalization and mortality, and some technician-administered outcome tests may be completed Patient questionnaires are often used, which are not always validated Rather than contemporaneous e-diaries, data are usually collected retrospectively via periodic recall questionnaires conducted via telephonic interviews/conversations	Endpoints are retrospectively selected to measure effectiveness, safety, patient experience, PROs, resource utilization, risk vs benefit (relative effectiveness), etc, as determined by a study analysis plan prepared a priori before data analysis Long-term effectiveness can be assessed, and rare adverse events may be identified Outcomes reported may be meaningful for decision-making in routine clinical practice
Data quality	Usually very good	Variable	Concerns about sensitivity and specificity of data are present, given the retrospective, nonrandomized design and possible bias in matching algorithms
Generalizability	Results are usually reproducible in the population studied, and support drug regulatory approval Results are applicable to patient populations with disease characteristics same or similar to those included in RCTs	Findings may be hypothesis generating or suggestive Can establish effectiveness in broad real-world populations. However, because of variable adherence, infrequent visits, and limited questionnaire-based endpoints, confirmation by RCTs may also be needed	Results are applicable to a broad range of healthcare databases, may apply to real-life treatment users, and may be generalizable to routine clinical practice These studies are post hoc analyses, and require confirmatory RCTs or replicate observational studies before results can be broadly accepted
Validity	Randomization and nondifferential assignment are attempted to make the treatment groups comparable at baseline and ensure that the results are valid and not confounded High level of scientific accuracy of conclusions is ensured by strict adherence, monitoring, and restrictions on disallowed medications, as well as serial, contemporaneous collection of objective endpoints	Prospective design and randomization add credibility to these findings Findings are suggestive because of the weak controls on adherence, confounding or alternative therapies, and the limited endpoints assessed Broader patient populations are enrolled If superiority is demonstrated, these trials can provide compelling data for clinicians and payers Findings of “noninferiority” are more difficult to generalize because poor adherence, crossing over between therapies (if allowed), or soft endpoints can lead to scientific uncertainty	Risk vs benefit assessment among treatment groups may be confounded by incomparability of clinical characteristics at baseline because of differential prescribing Results may not be internally valid and need to be interpreted with caution
Precision	Results may be reasonably precise in RCTs of large sample size (>1000 patients)	Precision is sacrificed to ensure higher cost-effectiveness and feasibility Evidence of superior efficacy compared with usual care/standard therapies can be demonstrated in relatively small studies Larger samples are needed for adequate power in “noninferiority” trial designs because real-life patients may not always be highly responsive or adherent to treatments	A large sample size is likely to increase the precision of the study
Cost	High cost per patient	Intermediate cost per patient Studies may be more expensive in total because larger numbers of patients are required, and real-world patients may be less sensitive to drug effects than highly selected patients	Low cost per patient
Value	Are of value for controlled scientific analysis of treatment effectiveness Required for regulatory approval	Provide suggestive value to regulators and payers May broaden populations appropriate for clinical treatments	Traditionally of value to payers

Abbreviations: COPD, chronic obstructive pulmonary disease; e-diary, electronic diary; e-health, electronic health; PrCT, pragmatic clinical trial; PRO, patient-reported outcome; RCT, randomized controlled trial.

Cohort studies⁶ are prospective or retrospective studies conducted to determine the incidence and natural history of a disease or condition. Exposure to putative risk factors precedes the outcomes, and multiple outcomes can be studied using one cohort. However, prospective cohort studies are expensive and have a substantial risk of attrition, whereas retrospective cohort studies may be impacted by recall bias.

Case–control studies⁶ are retrospective studies in which individuals with and without the disease or condition of interest are matched (eg, age, sex,²⁰ duration of comorbid diseases, and severity markers for comorbid diseases). Severity matching is imperfect and can induce bias; therefore, both pre- and post-matched baseline severity characteristics should be reported. Case–control studies are useful for understanding exposure factors for rare diseases.⁶ In these studies, investigators review large healthcare database records and determine which individuals had the suspected exposure in the past. These studies are relatively cheaper (especially when compared with prospective cohort studies) and are feasible when the disease or condition of interest is rare. However, these studies are subject to biases such as sampling, observation, or recall bias.

Pragmatic Clinical Trials

Key Features vs RCTs and Real-World Studies

PrCTs have some features of both RCTs and real-world observational studies (Table 1):^2,19,21 like RCTs, they use prospective study designs and randomization, and like real-world studies, they involve broadly inclusive populations, representative of patients receiving the treatment in everyday clinical practice, and are conducted by healthcare professionals in community-based settings, where regular patient management is ensured while still tracking some measures of treatment adherence. In PrCTs, relevant outcomes important to inform optimal healthcare treatment decisions are captured, and appropriate active comparators are generally included instead of placebo.²² In RCTs, the benefits of an intervention may be overestimated and the harms may be underestimated because they are performed with relatively small and highly selective patient populations at research sites with experienced investigators.² Therefore, findings from RCTs should not be used in formulating usual practice guidelines without further evaluation.²³ In contrast, PrCTs provide effectiveness and safety estimates in large, real-world, diverse patient populations using broad inclusion and relatively few exclusion criteria.²¹ Randomization in PrCTs confers some of the strengths of RCTs, such as credibility and limiting the allocation bias, to the PrCTs while providing external validity from the real-world component.²⁴ PrCTs may also be designed to compare the effectiveness of alternative treatments or practice procedures not supported by the industry or the Food and Drug Administration (FDA)²⁵ and allow recording of hospitalization and mortality throughout the study duration. Apart from objective clinical endpoints, inclusion of cost-effectiveness and adoption endpoints in PrCTs is critical for decision-making by health systems. Therefore, PrCTs can provide true risk/benefit assessments and value of a medicine in a routine care setting, allowing healthcare practitioners and payers to make informed decisions.²³

The limitations of PrCTs should, however, be noted—depending on whether they more closely mimic RCTs or real-world observational studies, PrCTs may have weak controls on adherence to therapy,²⁸ permit cross-over to alternative therapies,²⁸ and/or lack or include fewer objective, procedure-based outcome measures compared to RCTs. PrCTs may rely on alternative endpoints²² based on in-office or telephone questionnaires, which may provide suggestive evidence in some cases. On the other hand, lower adherence to therapies in PrCTs compared with RCTs is more reflective of the real-world clinical scenarios in COPD²⁹ and contributes to the enhanced external validity of the results.

As discussed, the key aspects of trial designs differ substantially between RCTs and PrCTs. Researchers can use the PRagmatic Explanatory Continuum Indicator Summary-2 (PRECIS-2) tool to make study design decisions befitting the intended use of the trial.²⁶ The applicability of real-world evidence (RWE) in research is expanding. With the 21st Century Cures Act, drug manufacturers can submit RWE instead of RCT results to support the expansion of additional indications for previously approved drugs.²⁷

Pragmatic Clinical Trials: Design Considerations

Study populations in PrCTs represent real-world populations likely to be prescribed treatment resembling routine clinical practice. With randomization and real-world use, the approximate effectiveness and safety of such interventions can be evaluated by establishing “superiority” vs usual care or standard treatment (ie, superiority trials),³⁰ or by showing “noninferiority” vs well-established therapies (ie, comparative effectiveness trials).³¹

Active-controlled superiority trials may require larger sample sizes than placebo-controlled trials,³² sometimes requiring several hundred patients per treatment arm to have enough statistical power to detect a superiority benefit. When the “superiority” of an intervention vs a control arm is evaluated in a PrCT, a prespecified statistical analysis plan (similar to RCTs with analysis methods such as MMRM or ANCOVA) is followed, and significant improvements in primary and secondary endpoints are sought (typically with P<0.05).

In comparative effectiveness trials, a carefully selected “noninferiority limit” is prospectively specified to ensure that a similar high level of clinical benefit is achieved for both the new and established treatments.³⁰ A noninferiority trial aims to establish that the intervention is not worse than its comparator by a prespecified degree, which is known as the “noninferiority limit or margin”.³³ Statistical analyses need to be prespecified and of high rigor for noninferiority trials, with stringent noninferiority limits. High adherence to treatment and low use of confounding therapies are important. Otherwise, sloppy design or implementation features can lead to a false noninferiority finding, thus erroneously concluding an inferior treatment to be noninferior.^15,34 Both per-protocol and ITT analyses are typically performed; however, unlike in superiority trials (where the ITT approach is considered conservative because it is likely to lead to a treatment effect closer to having no effect), the ITT approach is not conservative in noninferiority or equivalence trials because it can bias towards the null, which may lead to false claims of noninferiority or equivalence.³⁵ Usually, the sample size required to demonstrate noninferiority in an active-controlled trial is substantially larger (sometimes impossibly larger) than that for a placebo-controlled superiority trial;^15,30 small sample sizes may reduce the statistical power for proving noninferiority.³⁰

When well-conducted PrCTs report key differences or strong evidence for noninferiority between treatments, the findings may be applicable to larger, real-world populations, and the generated RWE can help clinicians understand the effectiveness and safety of the drug in clinical practice settings.

Comparing and contrasting the characteristics of RCTs, PrCTs, and real-world observational studies (Table 1) help to understand how these designs complement each other and can cumulatively create comprehensive evidence. Ideally, clinical research should address questions that are relevant to the target audience. To that effect, researchers could use the effective dissemination and implementation frameworks proposed by leading funding agencies, such as the National Heart, Lung, and Blood Institute and Patient-Centered Outcomes Research Institute.^36,37

Use of Different Research Methodologies in the Evaluation of COPD Pharmacotherapy

Determining an optimum treatment for COPD has been a conundrum for many decades. Researchers often focus on improving lung function, respiratory symptoms, and QoL; preventing and treating exacerbations; and minimizing morbidity and mortality.^38–40 Prevention of exacerbations to reduce morbidity has been crucial in COPD studies conducted as early as the mid-twentieth century.^38,39

Small Randomized/Nonrandomized Studies

Occasionally, small nonrandomized or randomized trials (pilot studies) with or without adequate statistical power are used to determine the feasibility of a therapeutic approach which, when successful, can lead to larger clinical trials. For example, a series of studies were conducted to assess the benefit of antibiotics in treating COPD exacerbations. Elmes et al conducted an RCT based on previous research which demonstrated that exacerbations of chronic bronchitis were usually associated with pathogenic bacterial proliferation.³⁸ Benefits of prophylactic oxytetracycline at the beginning of a suspected COPD exacerbation were assessed (as days ill and days off work; no lung function reported) in patients with chronic bronchitis. Among 88 patients who reported 146 exacerbations, exacerbation-associated loss of work time in the intervention group was half of that in the control group. Three decades later, Anthonisen et al documented their findings in a similar but slightly larger RCT with a crossover component conducted over 3.5 years in 173 patients.⁴¹ Of the 362 exacerbations, 182 and 180 were treated with an antibiotic and a placebo, respectively. With a treatment success of 68.1% vs 55.0% (P<0.01), a significant benefit was observed with antibiotic vs placebo use. In a much larger, well-designed and appropriately powered, parallel-group, prospective RCT by Albert et al, approximately three further decades later, 1142 patients were randomized to receive azithromycin (n=570) or placebo (n=572) in addition to usual care for 1 year.⁴² Patients in the azithromycin group had a significantly longer median time to first exacerbation, fewer exacerbations, lower risk of acute exacerbations, and greater improvement in St. George’s Respiratory Questionnaire (SGRQ) scores than those in the placebo group (all P<0.05). Findings from a subgroup analysis showed that antibiotic efficacy was confined to former smokers. Results also indicated a significantly higher risk of hearing impairment in the azithromycin group (P<0.05).

Similar to small, nonrandomized or randomized trials, a series of individual RCTs (N-of-1 RCTs) may provide preliminary data that could be confirmed in larger clinical trials. In a series of 27 N-of-1 RCTs, long-term ambulatory oxygen therapy did not improve QoL (as measured by the Chronic Respiratory Questionnaire and the SGRQ) in patients with COPD; the general application of long-term ambulatory oxygen therapy was found to be not justifiable for patients with COPD and transient exertional hypoxemia who did not satisfy the criteria for mortality reduction.⁴³ However, two patients were oxygen responders, leading the authors to conclude that select patients can be identified in N-of-1 RCTs who may benefit from long-term ambulatory oxygen therapy. Continuous oxygen therapy conferred a survival advantage specifically in patients with chronic obstructive lung disease with significant resting hypoxemia in the Nocturnal Oxygen Therapy Trial (NOTT) and Medical Research Council (MRC) studies.^44,45 However, long-term supplemental oxygen did not improve clinical outcomes in the Long-Term Oxygen Treatment Trial (LOTT), a large, multicenter, parallel-group RCT of 738 patients with COPD and moderate (less severe) resting or exercise-induced desaturation who were followed for 1 to 6 years.⁴⁶

Large Pivotal RCTs

Overall, well-conducted, large, pivotal RCTs have led to the approval of new drugs or drug combinations for COPD treatment (Table 2). The efficacy of single bronchodilators (long-acting β₂-agonist [LABA] or long-acting muscarinic antagonist [LAMA]), dual bronchodilators (LABA + LAMA), and triple therapy (LABA + LAMA + inhaled corticosteroid [ICS]⁴⁷) has been assessed in phase 3 and 4 clinical trials, which exemplify variations in RCTs with respect to design, population, and outcomes. The clinical development of tiotropium (an inhaled LAMA) and roflumilast (an oral phosphodiesterase-4 inhibitor), for example, exemplify the trajectory of different research methodologies—from preclinical studies through different phases of clinical trials.

Table 2.

Examples of Pivotal and Subsequent Randomized Controlled Trials for COPD Pharmacotherapies

Study Name	Study Type	Study Results	Drug Comparators	Duration
Tiotropium (LAMA)48
Niewoehner et al48	A parallel-group, randomized, double-blind, placebo-controlled trial (N=1829)	Tiotropium 18 µg vs placebo reduced COPD exacerbations and related healthcare utilization in patients with moderate-to-severe COPD	Tiotropium vs placebo	6 months
UPLIFT49	A phase 3, randomized, double-blind, parallel-group, multicenter, placebo-controlled trial (N=5993)	Tiotropium 18 µg improved lung function, quality of life, and exacerbations (vs placebo) over 4 years, but did not significantly reduce the rate of decline in FEV1	Tiotropium vs placebo	4 years
TIOSPIR61	A phase 3, double-blind, parallel-group, multicenter, randomized controlled trial (N=17,135)	Tiotropium SMI 5 µg and 2.5 µg were noninferior to tiotropium DPI 18 µg for mortality risk (both P<0.05) and not superior for exacerbation risk (P=0.42 and P=0.56, respectively) in this trial of 17,135 COPD patients	Tiotropium SMI 5 µg and 2.5 µg vs tiotropium DPI 18 µg	2.3 years mean follow-up
Tiotropium + olodaterol (LAMA + LABA)88,89
WISDOM76,77	A phase 4, double-blind, parallel-group, multicenter, randomized controlled trial (N=2485)	Stepwise ICS withdrawal (fluticasone propionate) was noninferior to ICS continuation for risk of moderate or severe exacerbations in patients with severe or very severe COPD receiving tiotropium + salmeterol. ICS withdrawal resulted in a modest decline in trough FEV1; in patients with high baseline blood eosinophils, ICS withdrawal resulted in increased COPD exacerbations	ICS (fluticasone propionate) withdrawal vs ICS continuation in patients on triple therapy (fluticasone propionate + tiotropium + salmeterol)	52 weeks
Indacaterol + glycopyrronium (LABA + LAMA)99
FLAME90	A phase 3, double-blind, double-dummy, noninferiority, multicenter, randomized controlled trial (N=3362)	Indacaterol + glycopyrronium significantly lowered the annual rate of moderate or severe exacerbations and significantly increased the time to first moderate or severe exacerbation or time to first severe exacerbation vs salmeterol + fluticasone in COPD patients with a history of at least one exacerbation in the previous year	Indacaterol + glycopyrronium vs salmeterol + fluticasone	52 weeks
SUNSET72	A phase 4, double-blind, triple-dummy, parallel-group, multicenter, randomized controlled, switch trial (N=1053)	Direct switch from long-term triple therapy to indacaterol + glycopyrronium did not impact COPD exacerbation risk in low-risk populations; patients with eosinophil counts ≥300/µL at both screening and baseline were more likely to benefit from continuing triple therapy	Indacaterol + glycopyrronium vs triple therapy (tiotropium + salmeterol + fluticasone)	26 weeks
Fluticasone furoate + umeclidinium + vilanterol (ICS + LAMA + LABA)
IMPACT73	A phase 3, double-blind, parallel-group, multicenter, randomized controlled trial (N=10,355)	Fluticasone furoate + umeclidinium + vilanterol in patients with COPD and FEV1 predicted <50% normal and ≥1 moderate or severe exacerbation or FEV1 predicted 50%–80% normal and ≥1 severe or ≥2 moderate COPD exacerbations resulted in significantly lower moderate or severe COPD exacerbation rates vs fluticasone furoate + vilanterol or umeclidinium + vilanterol in patients with symptomatic COPD	Triple therapy (fluticasone furoate + umeclidinium + vilanterol) vs fluticasone furoate + vilanterol or umeclidinium + vilanterol	52-week treatment period
Beclomethasone dipropionate + formoterol fumarate + glycopyrronium bromide (ICS + LABA + LAMA)78,79
TRILOGY74	A phase 3, double-blind, parallel-group, multicenter, randomized controlled trial (N=1368)	In symptomatic COPD patients, triple therapy with beclomethasone dipropionate + formoterol fumarate + glycopyrronium bromide significantly improved predose and 2-hour postdose FEV1 vs beclomethasone dipropionate + formoterol fumarate dual therapy. Triple therapy also significantly reduced the adjusted annual moderate-to-severe exacerbation frequencies vs dual bronchodilator therapy	Triple therapy (beclomethasone dipropionate + formoterol fumarate + glycopyrronium bromide) vs beclomethasone dipropionate + formoterol fumarate	52-week treatment period
Budesonide + glycopyrronium + formoterol fumarate (ICS + LAMA + LABA)
KRONOS75	A phase 3, double-blind, parallel-group, multicenter, randomized controlled trial (N=1902)	Budesonide + glycopyrronium + formoterol fumarate MDI triple therapy was efficacious and well tolerated and showed improvements, including reduced COPD exacerbation rates, vs corresponding dual bronchodilator therapies in symptomatic patients with moderate-to-very severe COPD, irrespective of exacerbation history	Budesonide + glycopyrronium + formoterol fumarate via MDI vs glycopyrrolate + formoterol fumarate or budesonide + formoterol fumarate via MDI, or open-label budesonide + formoterol fumarate DPI	24 weeks
ETHOS47	A phase 3, double-blind, parallel-group, multicenter, randomized controlled trial (N=8564 [actual enrollment])	Completed; results awaited. The trial will assess the efficacy and safety of triple therapy with budesonide + glycopyrronium + formoterol fumarate aerosol provided as MDI vs corresponding dual bronchodilator and bronchodilator + ICS therapies for COPD exacerbations in patients with moderate-to-very severe COPD	Budesonide + glycopyrronium + formoterol fumarate (2 regimens with different doses) vs budesonide + formoterol fumarate or glycopyrronium + formoterol fumarate	52 weeks

Abbreviations: COPD, chronic obstructive pulmonary disease; DPI, dry powder inhaler; FEV₁, forced expiratory volume in 1 s; ICS, inhaled corticosteroid; LABA, long-acting β₂-agonist; LAMA, long-acting muscarinic antagonist; MDI, metered-dose inhaler; SMI, soft mist inhaler.

Niewoehner et al first demonstrated that tiotropium significantly reduced COPD exacerbations in patients with moderate-to-severe COPD in a clinical trial of 6-month duration.⁴⁸ This finding was further corroborated in the UPLIFT trial, a much larger phase 3 trial with approximately 6000 patients with moderate-to-very severe COPD who were followed for 4 years.⁴⁹ Historically, tiotropium was delivered as a dry powder inhaled through HandiHaler^® (tiotropium dry powder inhaler [DPI]), but later, it became available as an inhalation solution delivered via Respimat^®, a soft mist inhaler (SMI; tiotropium SMI). Further development led to the dual bronchodilator formulation of tiotropium + olodaterol (5 + 5 µg; a fixed-dose LAMA + LABA). The development of tiotropium began with ipratropium, the first approved anticholinergic that has been shown to be safe and efficacious in both COPD and asthma treatment.⁵⁰ In initial in vitro studies, tiotropium—although structurally similar to ipratropium—had a substantially higher affinity (6- to 20-fold), greater M3 receptor selectivity, and a slower dissociation profile and, hence, longer duration of action than ipratropium.^51,52 The high potency, slow onset, and long duration of effect of tiotropium were confirmed in the airways of guinea pigs and humans.⁵³ The dose-related bronchodilator activity of tiotropium in COPD was demonstrated in an open-label, dose-escalation, crossover, pilot study of five single-inhalation doses of tiotropium (10–160 µg; N=6).⁵⁴ The efficacy of tiotropium in COPD was further confirmed in a phase 2, 4-week, dose-ranging RCT (N=169); the once-daily, 18-µg dose was well tolerated and, thus, selected for long-term trials.⁵⁵ In a larger RCT (N=288), 18 µg once-daily tiotropium DPI was significantly more efficacious than ipratropium (40 µg, four times daily), with a comparable safety profile over 13 weeks.⁵⁶ Patients in this trial continued into a 1-year RCT and their results, combined with results from another identical 1-year RCT of patients with COPD, confirmed an acceptable safety profile and significantly greater efficacy of tiotropium 18 µg once daily vs ipratropium in improving dyspnea, reducing exacerbations, and improving health-related QoL and lung function.⁴⁰

The efficacy and tolerability of different doses of once-daily tiotropium SMI (5 and 10 µg) were compared with that of 18 μg tiotropium DPI in preliminary studies,^57,58 and results facilitated the transition from tiotropium DPI to tiotropium SMI. Based on the available breadth of evidence, tiotropium DPI was used as an active comparator to assess the efficacy and safety of once-daily tiotropium SMI in pivotal trials. Further key trials of tiotropium monotherapy or dual bronchodilator therapy with tiotropium + olodaterol (5 + 5 µg) are summarized in Table 2. Of interest, initial studies of 1-year duration on efficacy and safety of tiotropium SMI indicated a numerical imbalance in the all-cause mortality signal in the tiotropium groups compared with placebo.^59,60 However, this finding directly conflicted with the results of UPLIFT, a large phase 3 trial with approximately 6000 patients, in which tiotropium DPI treatment was associated with significantly lower mortality compared with placebo during the 4-year study duration.⁴⁹ Subsequently, TIOSPIR, a large phase 3 trial with over 17,000 patients, which compared mortality and other safety endpoints between tiotropium SMI and tiotropium DPI, corroborated the findings concerning the safety of tiotropium.⁶¹ With comparable cardiovascular safety profiles for tiotropium SMI and tiotropium DPI, TIOSPIR refuted the concern that tiotropium SMI may have additional cardiovascular risks. This highlights the importance of large, well-designed, and adequately powered phase 3/4 RCTs to establish reliable efficacy and safety for new pharmacotherapies.

Similar to the clinical development of tiotropium, preclinical studies^62,63 of roflumilast were followed by phase 1 and 2 clinical trials in healthy volunteers and patients with COPD, respectively.^64,65 In the phase 2, crossover study, roflumilast significantly reduced the absolute number of neutrophils and eosinophils in the sputum of patients with COPD, and improved postbronchodilator forced expiratory volume in 1 s (FEV₁) vs placebo.⁶⁵ Later, results of a large phase 3 RCT (N=1411) confirmed the efficacy and safety of roflumilast in patients with moderate-to-severe COPD: postbronchodilator FEV₁ was improved and exacerbations were reduced.⁶⁶ Results of another large RCT (N=1513), however, did not demonstrate a reduction in exacerbations despite a significant improvement in lung function.⁶⁷ A pooled post hoc analysis of two previous replicate RCTs (including the one reported by Calverley et al⁶⁷) revealed that patients with chronic bronchitis and severe airflow obstruction with or without concurrent ICS use were most likely to benefit from roflumilast.⁶⁸ This understanding subsequently led to two multicenter RCTs in a specific subset of patients with COPD (ie, aged >40 years with severe airflow limitation, symptoms of chronic bronchitis, and history of exacerbations).⁶⁹ Indeed, the results demonstrated significant efficacy with roflumilast vs placebo; prebronchodilator FEV₁ was significantly improved, and moderate (glucocorticosteroid treated) or severe exacerbations were reduced. In the phase 3/4, postmarketing Roflumilast and Exacerbations in patients receiving Appropriate Combination Therapy (REACT) trial (N=1945), roflumilast significantly decreased the rate of moderate and severe exacerbations when added to baseline ICS + LABA or triple therapy (LABA + LAMA + ICS) over a period of 1 year in patients with severe COPD, according to prespecified secondary endpoints that considered the use of antibiotics in the definitions.⁷⁰ The primary endpoint (without consideration of antibiotics in moderate-to-severe exacerbations) was not met using Poisson regression (prespecified primary analysis method), although it was met using a negative binomial regression analysis. In the phase 4 Roflumilast Effect on Exacerbations in Patients on Dual (LABA + ICS) Therapy (RE²SPOND) trial, roflumilast reduced the rate of moderate and/or severe exacerbations in patients at risk for exacerbations despite treatment with ICS + LABA with or without a LAMA, although the results were not significantly different between the roflumilast and placebo groups.⁷¹

Large phase 3 RCTs also helped establish robust scientific evidence for efficacy and safety of ICS-containing therapeutic combinations (eg, LABA + ICS in the FLAME trial, LAMA + LABA + ICS in the SUNSET,⁷² IMPACT,⁷³ TRILOGY,⁷⁴ and KRONOS⁷⁵ trials) in unique patient populations with COPD.

In addition to evidence generated from clinical trials, practitioners, regulatory authorities, and formulary decision-makers appreciate RWE generated from observational and pragmatic trials. Examples of real-life COPD administrative database- or claims database-based studies and nonrandomized and randomized COPD PrCTs are summarized in Table 3.

Table 3.

Examples of Real-World Studies for COPD Pharmacotherapies

Study Name	Study Type	Study Aim/Results
Observational studies
DACCORD80	Real-life, prospective, noninterventional study in which patients were treated at the physician’s discretion (N=1258)	No increased exacerbation risk in over 2 years of follow-up in patients with ICS withdrawal compared with continuation of ICS therapies for COPD Patient treatment groups may not have been directly comparable because of lack of randomization and problems of severity matching
OPTIMO91	Real-life, prospective study in which patients were treated at the physician’s discretion (N=914)	No increase in exacerbation risk or deterioration in lung function upon ICS withdrawal from maintenance therapy (ICS + LABA) in patients with moderate COPD and low exacerbation risk Results from prospective randomized controlled trials need further confirmation because of lack of randomization, variable COPD severity, and cross-over between treatments
Samp et al 201792	Retrospective observational study based on an insurance claims database that included COPD patients in the United States treated with LAMA + LABA or ICS + LABA (N=478,772)	LAMA + LABA and LABA + ICS had similar effectiveness as measured by exacerbation rates in COPD patients
Voorham et al 201893	Matched historical cohort study conducted using records from the OPCRD and CPRD primary care databases (N=1647)	Significant reduction in exacerbation risk was observed with triple vs dual bronchodilator therapy, with a larger reduction in frequent exacerbators
Price et al 201894	Matched historical cohort study of real-life management of COPD patients with or without comorbid asthma Data from the OPCRD and CPRD primary care databases on patients prescribed the salbutamol comparator or a reference product were evaluated (N=1191)	The salbutamol comparator was noninferior to the reference product for the rate of moderate and severe COPD exacerbations after matching for demographic variables, indicators of disease severity, and baseline maintenance medication
Pragmatic nonrandomized controlled trial
Nyberg et al 201795	Prospective, multicenter, 12-month trial with planned enrollment of 96 patients with COPD from six participating primary care units in Sweden (N=96)	Results awaited The trial aims to evaluate the feasibility of the study design and procedures that consider the effectiveness of the COPD-web, a novel intervention, which is an internet-based program to support self-management strategies
Pragmatic randomized controlled trial
CRYSTAL96	Prospective, multicenter, 12-week, open-label, PrCT in COPD patients with moderate airflow limitation (N=4389)	Indacaterol + glycopyrronium improved lung function and dyspnea after direct switch from previous treatments, either ICS + LABA or LABA or LAMA monotherapy
Salford Lung Study81,82	Prospective, multicenter, 12-month, open-label, phase 3 PrCT in COPD patients receiving regular maintenance via inhaler therapy (N=2799)	Fluticasone furoate + vilanterol (ICS + LABA) delivered via a novel dry powder inhaler lowered the rate of exacerbations vs usual care without increasing the risk of serious adverse events
AIRWISE85	Prospective, multicenter, 12-month, open-label, phase 4 PrCT with a planned enrollment of 3200 patients across community-based sites (N=3200 estimated)	Results awaited; estimated primary completion date: February 23, 2021 The aim of the study is to compare the time to first moderate or severe COPD exacerbation in patients not controlled on their current therapy, randomized to tiotropium + olodaterol (LAMA + LABA) vs triple therapy (LAMA + LABA + ICS) over 12 months
RELIANCE86	Multicenter, 36-month, parallel-group, noninferiority, phase 3 study with a planned enrollment of 3200 patients	Currently recruiting; estimated primary completion date: February 2023 The aim of the study is to compare the effectiveness of roflumilast therapy vs azithromycin to prevent hospitalization or death in patients at a high risk of COPD exacerbations

Abbreviations: COPD, chronic obstructive pulmonary disease; CPRD, Clinical Practice Research Datalink; ICS, inhaled corticosteroid; LABA, long-acting β₂-agonist; LAMA, long-acting muscarinic antagonist; OPCRD, Optimum Patient Care Research Database; PrCT, pragmatic clinical trial.

Over the years, findings from observational studies have increased our knowledge and rekindled the evaluation of clinical conundrums by using real-world data to understand the implications of changes in COPD management. For example, the use or withdrawal of ICS and its effects on COPD management have been evaluated in RCTs.^{72,73,75–79} In DACCORD, a real-life, prospective, noninterventional study, patients were treated at the discretion of the physician.⁸⁰ The results were in agreement with similar clinical trials^76,99 and demonstrated that the risk of exacerbation over 2 years was not increased following ICS withdrawal in patients with low exacerbation risk. However, COPD phenotype groups who do benefit from ICS, including patients with elevated eosinophils and with a history of one or more exacerbations when FEV₁ is compromised (<50% of predicted),⁷⁷ have been identified in large RCTs.^73,75,77

The findings from large, community-based PrCTs such as the Salford Lung Study in COPD^81,82 have provided important RWE for COPD pharmacotherapy. The aim of the Salford Lung Study, a 12-month, open-label, randomized (1:1) phase 3 PrCT, was to evaluate the safety and effectiveness of fluticasone furoate + vilanterol (FF + VI; an ICS + LABA combination) vs usual care in patients with COPD and a history of exacerbations (N=2799) in a real-world setting (75 general practices in Salford and South Manchester, United Kingdom).^81,82 Use of FF + VI was associated with significantly lower rates of moderate or severe exacerbations without increasing the risk of serious adverse events.⁸² However, the rate of first moderate or severe exacerbation in the time-to-event analysis was not significantly different between groups (hazard ratio for FF + VI vs usual care: 0.93; 95% CI: 0.85–1.02). Limitations of this study included the fact that COPD diagnosis was not confirmed by spirometry,^81,83 and that the study population for FF + VI did not match the target population approved by the regulatory agency.⁸⁴ In addition, 22% of the subjects receiving FF + VI switched back to their previous regimen.⁸³ Although the latter finding is reflective of real-world settings, it complicates the ITT analyses.⁸³ Owing to such limitations and the possibility of the Hawthorne effect, ie, altered behavior of patients from awareness of being observed, the classification of the Salford Lung Study as a PrCT and the credibility of its results have been questioned in the literature.⁸⁴ Planned and ongoing PrCTs such as the Assessment In a Real World Setting of the Effect of Inhaled Steroid-based Triple Therapy Versus the Combination of Tiotropium and Olodaterol on Reducing COPD Exacerbations (AIRWISE)⁸⁵ and Roflumilast or Azithromycin to Prevent COPD Exacerbations (RELIANCE) trials⁸⁶ will also add to the accumulating RWE for COPD pharmacological treatments.

Efficacy and safety results of RCTs have been traditionally considered important by clinicians and regulatory authorities alike. However, generalizability of these results is limited because of highly restrictive inclusion and exclusion criteria. On the contrary, effectiveness and long-term safety results provided by real-world studies are accommodative of real-world patient populations and routine clinical practice but have multiple sources of bias for comparisons of multiple treatment arms and questionable internal validity. Considering the strengths and limitations of RCTs and real-world studies, clinicians should make data-driven decisions taking into account results from both types of clinical studies. Moreover, RCT and real-world observational study designs should be complementary in nature, such that, taken together, they provide more robust clinical evidence compared with individual study types.⁸⁷

Conclusion

In summary, different study designs have their associated advantages and disadvantages. However, when used in concert, findings from various types of studies bring about progress in clinical research. Although RCTs are considered the “gold standard” for evidence on the safety and efficacy of an intervention, observational studies conducted in real-world settings provide evidence on the effectiveness of that intervention in clinical practice. PrCTs help to bridge the gap between classical explanatory RCTs and real-world studies, with a study design that leverages the advantage of randomization in a real-world scenario, thus providing a clearer picture of the safety and effectiveness of a drug.

Funding Statement

Financial support for the preparation of this article was provided by Boehringer Ingelheim Pharmaceuticals, Inc. (BIPI). BIPI was given the opportunity to review the manuscript for medical and scientific accuracy as well as intellectual property considerations.

Disclosure

Dr. Tashkin has participated as an advisory board member and speaker for Boehringer Ingelheim, AstraZeneca, and Sunovion. He has also worked as a consultant for Mylan and Innoviva/Theravance. He also reports personal fees from AstraZeneca, outside the submitted work. Dr. Amin has worked as a consultant and/or speaker for Boehringer Ingelheim, Sunovion, BMS, Pfizer, Portola, and AstraZeneca. He also reports personal fees from Boehringer Ingelheim, Sunovion, BMS, Pfizer, Portola, and Astra Zeneca, outside the submitted work. Dr. Kerwin has participated in consulting, advisory boards, and speaker panels for, or has received travel reimbursement from, Amphastar, AstraZeneca, Boehringer Ingelheim, Cipla, GlaxoSmithKline, Mylan, Novartis, Oriel, Pearl, Sunovion, Teva, and Theravance. He has conducted multicenter clinical research trials for approximately 40 pharmaceutical companies. The authors report no other conflicts of interest in this work.