Real-world research and its importance in respiratory medicine

Abstract

Educational Aims

To improve understanding of:

• The relative benefits and limitations of evidence derived from different study designs and the role that real-life asthma studies can play in addressing limitations in the classical randomised controlled trial (cRCT) evidence base.

• The importance of guideline recommendations being modified to fit the populations studied and the model of care provided in their reference studies.

Key points

• Classical randomised controlled trials (cRCTs) show results from a narrow patient group with a constrained ecology of care.

• Patients with “real-life” co-morbidities and lifestyle factors receiving usual care often have different responses to medication which will not be captured by cRCTs if they are excluded by strict selection criteria.

• Meta-analyses, used to direct guidelines, contain an inherent meta-bias based on patient selection and artificial patient care.

• Guideline recommendations should clarify where they related to cRCT ideals (in terms of patient populations, medical resources and care received) and could be enhanced through inclusion of evidence from studies designed to better model the populations and care approaches present in routine care.

Summary

Clinical practice requires a complex interplay between experience and training, research, guidelines and judgement, and must not only draw on data from traditional or classical randomised controlled trials (cRCTs), but also from pragmatically designed studies that better reflect real-life clinical practice. To minimise extraneous variables and to optimise their internal validity, cRCTs exclude patients, clinical characteristics and variations in care that could potentially confound outcomes. The result is that respiratory cRCTs often enrol a small, non-representative subset of patients and overlook the important interplay and interactions between patients and the real world, which can effect treatment outcomes.

Evidence from real-life studies (e.g. naturalistic or pragmatic clinical trials and observational studies encompassing healthcare database studies and cohort studies) can be combined with cRCT evidence to provide a fuller picture of intervention effectiveness and realistic treatment outcomes, and can provide useful insights into alternative management approaches in more challenging asthma patients. The Respiratory Effectiveness Group (REG), in collaboration with the European Academy of Allergy and Clinical Immunology (EAACI) and the European Respiratory Society (ERS), is developing quality appraisal tools and methods for integrating different sources of evidence. A REG/EAACI taskforce aims to help support future guideline developers to avoid a one-size-fits-all approach to recommendations and to tailor the conclusions of their meta-analyses to the populations under consideration.

Introduction

Contrary to what many of the guidelines purport, patients refuse to fit into neat boxes or to behave in the logical and rational manner expected by their clinicians. Additionally, healthcare professionals (HCPs) often do not follow guidelines to the letter and as a result may offer patients less-than-optimal therapy. Daily clinical practice requires a complex interplay between experience and training, research and judgement and must draw on data not only from classical randomised controlled trials (cRCTs), but also from pragmatically designed studies that better reflect real-life clinical practice.

Asthma guidelines have been developed to support the complex decisions faced by practicing clinicians by synthesising and summarising the data available in the field. They aim to signpost appropriate treatments and should be a useful reference for clinicians, especially general practitioners and other non-specialist HCPs who need evidence-based guidelines to help diagnose, manage and provide patients with informed treatment choices. However, guidelines can only be as robust as the evidence on which they are based. To understand the true value of guidelines, it is important to understand the methodologies on which they are founded [1].

The historical systems used to rate the quality of evidence all put RCTs on top of the hierarchy and observational studies at the bottom, just above the “no evidence” level (i.e., expert opinion). More recently, the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) system was established to help developers of guidelines. More than 70 organisations around the world (among them the American Thoracic Society (ATS) [2], the World Health Organization (WHO) [3] and the Cochrane Collaboration [4]) now subscribe to this system for developing guidelines. GRADE categorises evidence as “very low”, “low”, “moderate” or “high” based on its perceived methodological quality and likelihood of outcome bias. By default, RCTs start as high-quality evidence, observational studies as low-quality evidence. But GRADE offers a unique and highly valuable feature: the opportunity to upgrade or downgrade these ratings based on the detailed methodological characteristics of studies. Accordingly, the quality of RCTs can be diminished through poorly detailed design and execution, inconsistency, indirectness, reporting bias and imprecision. Conversely, the quality of observational studies can be elevated and strengthened through use of robust methods of handling potential sources of bias (e.g. selection bias, recall bias, information bias, detection bias) and a priori study registration and planning.

The GRADE system has unequivocal value but it results in an inherent dependence on RCT evidence within clinical guidelines. This relates in part to the high robustness of evidence provided by well-conducted RCTs, but other factors need to be considered: 1) many researchers and experts have more experience in assessing the quality of evidence coming from RCTs than from real-life studies; and 2) high-quality real-life studies are still scarce. While the importance of RCTs for establishing the efficacy and short-term safety profile of new therapies is unequivocal, they are not without their limitations when seeking to evaluate how therapies will be used and perform in the real world. It is neither practicable, nor affordable (considering that funds, manpower and materials available for research cannot be endlessly expanded), and sometimes not even ethical, to address all the clinical questions that arise in daily practice within an RCT.

RCTs are typically conducted in centres of excellence and are designed to minimise extraneous variables so that a direct cause-and-effect relationship can be discerned between an intervention and an observed outcome. Patient and clinical characteristics with the potential to confound an outcome, be they lifestyle characteristics (e.g. current smoking status), socioeconomic or educational characteristics, behavioural characteristics (e.g. religious beliefs, medication attitudes) and/or clinical characteristics (e.g. comorbidities, reversibility of airflow obstruction) are systematically controlled for, as are variability in intensity of clinical care and monitoring. In respiratory medicine, efficacy RCTs exclude approximately 95% of asthma and 90% of COPD routine care populations due to their strict inclusion criteria (see fig. 1) [5]. A recent study found that 45% of patients with a physician diagnosis of COPD did not meet the typical clinical trial criteria for defining COPD and noted that patients who were younger, female, not white, had diabetes or depression were under-represented in the cRCTs. The authors concluded that efficacy trials in COPD often enrol a small, non-representative subset of patients with physician-diagnosed COPD [6].

Figure 1.

Studies have shown that classical efficacy RCTs exclude about 90% for a) asthma and 95% for b) COPD routine care populations due to strict inclusion and exclusion criteria. a) For asthma clinical-trial patients criteria included: visual analogue scale (VAS) <2; historical reversibility in FEV 12% within the last 12 months; absence of significant co-morbidity; nonsmoker or if previous smoker (XS) a smoke burden <10 pack-years; regular use of ICS; symptomatic asthma (defined as either the use of short acting β2-agonist daily or nocturnal awakening due to asthma at least once a week). b) For COPD clinical-trial patients criteria included: VAS >7.5, absence of significant co-morbidity; smoker (S) or XS; a smoke burden of >15 pack-years; no history of hay fever indicating presence of atopy. Reproduced from [5] with permission from the publisher.

Thus, while respiratory guidelines are built upon a strong foundation of systematic appraisal of the available evidence (among which the highest GRADE levels are provided by cRCTs), there are many patients or clinical patterns of care that do not feature amid that evidence and for whom practicing physicians must take a clinical “leap of faith” when making management decisions.

Combining evidence sources

Given the current economically constrained environment of research, a high proportion of RCTs are designed mostly to satisfy registration requirements. Therefore, they answer only very specific PICOT (Population details, Intervention, Comparison to controls, Outcome, Time to outcome) questions restricted to evaluation of interventions in idealised populations and in optimised standards of care. As outlined before, the unavoidable result is a number of gaps in the evidence available to practicing clinicians when they are faced with making clinical conditions in more routine scenarios. Where data are unavailable, or unevaluable through the conventional cRCT route, there is a need to consider alternative study methodologies, such as pragmatic randomised controlled trials (pRCTs; see table 1) and observational studies.

Table 1.

Common similarities and differences between the design of classical and pragmatic RCTs

Feature of trial design	Classical RCT	Pragmatic RCT
Randomisation	Yes	Yes
Control group	Yes	Yes
Setting/ecology of care	• Highly controlled • Specialised centres (secondary or tertiary)	• Pragmatically controlled • Usual care (> primary care)
Patient population	• Highly selected • Confirmed diagnosis Narrow (“pure”) population	• Pragmatically selected • Clinical diagnosis Broad (“real-life”) population
Inclusion/exclusion criteria	Many	Few
Adherence	Very good (stimulated and monitored)	Low (real-world adherence)
Therapy	• Blinded (single- or double-blind); or • Open-label	Usually open-label to allow for effects of different technologies e.g. device or mode of administration
Comparator	Placebo; and/or active treatment	Active treatment
Outcome	Efficacy	Real-life effectiveness (comparative effectiveness)
Safety	Usually short-term	Short-term and long-term

There is increasing recognition of the need to look beyond the traditional cRCT to achieve a clearer picture of treatment outcomes when used outside a clinical trial setting, as is noted in the 2014 revised Global Initiative for Asthma (GINA) recommendations. GINA’s revised recommendations are built upon a “new” definition of asthma, one that recognises its heterogeneous nature and variability, and the need to tailor patient care to individual patient characteristics, modifiable risk factors, patient preference and practical issues to maximise the benefit that can be obtained from available medications.

Responding to calls for a more integrated view of the available evidence being used to inform practical, implementable clinical guidance [7, 8], this paper considers what real-life evidence is available, how it can complement that from cRCTs and opportunities to use real-life evidence to help guide treatment decisions in asthma.

What sort of evidence 'do we have?

Asthma cRCTs provide necessary, robust short-term safety and efficacy data for available interventions. Yet their high internal validity comes at the expense of their external validity; limiting the extent to which intervention efficacy can be extrapolated to reflect real-life effectiveness in the more heterogeneous populations managed in routine care.

Intervention effectiveness (rather than efficacy) is typically the focus of real-life studies, where “real-life” refers to pragmatic (or naturalistic) RCTs and observational studies designed to better reflect aspects of routine care than most cRCTs [9]. Thus a “real-life” study often includes wide patient populations and/or is designed with a management approach that mimics that of clinical practice. Yet as Roche et al. [10] have observed, understanding whether or not a given study should be considered “real-life” is not always clear.

For instance, a trial may involve close patient follow-up (very different from usual care) yet include a broad patient population that is fairly representative of the general population with the condition of interest. Thus it is “real-life” in terms of population selection, but not in terms of its “ecology” (i.e. general modalities and context) of care. Conversely, an observational study can focus on outcomes in a highly selected patient population, yet involve no clinical intervention beyond usual care; representing a real-life study in terms of ecology of care but not patient selection.

To assist classification and interpretation of the available evidence, Roche et al. [10] propose an integrated framework of the evidence base for clinical research that classifies studies within a two-dimensional design space bound by a study population axis (ordinate) and an ecology of care axis (abscissa) (fig. 2a). The ecology of care axis categorises studies along a continuous scale from highly controlled efficacy RCT management and follow-up, at one end, to usual care (real-life practice) at the other (fig. 2c). The population axis categorises a study population along a continuum from a highly selected population through to a clinically diagnosed population and on to “managed as condition X” (i.e. managed as the condition under evaluation with or without a properly confirmed physician diagnosis) (fig. 2d). Any study can be positioned within this space depending on the degree to which its two components can be said to reflect real-life. The position of a study within the space is not a marker of its quality; it is a descriptive classification designed to aid understanding and interpretation of the resulting evidence.

Figure 2.

Integrated research framework, bounded by population and ecology of care axes running from highly selected to managed care populations (Y-axis) and from highly interventional to observational study design approaches (X-axis). Adapted from [10] with permission from the publisher.

What do cRCTs tell us?

cRCTs are the cornerstones of the drug licensing process. They typically compare the efficacy of a new product against placebo or a gold standard management approach, and capture short-term safety data to ensure a new intervention is both efficacious and safe (fig. 2b). They are designed to address specific PICOT questions, requiring identification of a highly characterised patient population, intervention, comparator, outcome and evaluation time. In other words, they tend to “purify” all study components in order to minimise the risk of missing a beneficial effect of the treatment.

Asthma treatment options

A systematic, Cochrane review of the evidence comparing anti-leukotriene agents and inhaled corticosteroids (ICS) in the management of recurrent or chronic asthma concluded that, as monotherapy, ICS display superior efficacy to anti-leukotrienes (particularly in patients with moderate airway obstruction) [11]. The review endorsed the current guideline position that ICS should be the first-choice preventer therapy in patients initiating maintenance asthma therapy (with leukotriene modifiers as an option) [12, 13]. For those who achieve suboptimal control on low-dose ICS maintenance therapy, the addition of a long-acting bronchodilator (LABA) is recommended before increasing the dose of ICS or adding a leukotriene modifier or sustained-release theophylline to low-dose ICS.

If one takes a moment to review some of the high-quality cRCT data underpinning these recommendations, it is of note that the cRCTs range in duration from 12 to 28 weeks, include compliance rates in excess of 90% and typically include a run-in period [14–17], which introduces knowledge of observation and potential Hawthorne effects [18]. Following consideration of the cRCT evidence base informing the recommendation, there is perhaps a missing qualifying statement to advise that ICS are the most effective preventer drug: in short-term use for those asthma patients with optimal inhaler technique, substantial lung function impairment, 15% reversibility of airflow limitation, if they do not smoke and have excellent medication compliance.

Inhaler devices

While manufactures invest heavily in the design and development of inhaler devices, systematic reviews and guidelines conclude that there is no evidence that alternative inhaler devices (DPIs, breath-actuated pMDIs or hydrofluoroalkane pMDIs) are more effective than the pMDIs with a spacer for the delivery of ICS therapy [12, 13, 19].

Yet, there is perhaps a certain inevitability to these conclusions. cRCTs typically train participants in the optimum use of their inhaler and require demonstration of effective use throughout, with extensive expert review at most trial visits. The missing qualifier within statements claiming device equivalence should perhaps be that all devices appear equal “when used optimally”. If the research question under consideration were: “which inhaler device is associated with the best outcomes in a broad, heterogeneous asthma population managed in a naturalistic setting?” the evidence may be less conclusive due to the high frequency of errors in inhaler technique with all types of device and the frequent lack of adequate patient education by trained HCPs.

What can real-life research tell us?

Real-life research offers insights into the interactions between patient characteristics, preferences and lifestyle and treatment outcomes that are often missed (excluded) from cRCTs. It also offers the opportunity to explore how these interactions differ between therapies and treatment modalities and to evaluate important clinical outcomes, such as asthma exacerbations, that can be underpowered in cRCTs owing to their short duration and their pre-selection of idealised patients in whom such outcomes are often relatively infrequent.

Real-life ecology of care

Studies designed to reflect a more naturalistic, real-life management approach can provide interesting insights into the differences between cRCT management and routine care management, and the potential implications of ecology of care on treatment outcomes (fig. 2c).

Medication adherence, for example, is considered a key pillar of treatment effectiveness, with optimum therapeutic outcomes only achievable if taken as prescribed. Yet adherence to chronic asthma medication in longer-term RCTs has been shown to drop dramatically over time, for instance from ∼78% at 3 months to 50% by 2 years in one paediatric study (fig. 3) [20]. This contrasts markedly with the typical adherence rates of 75–125% seen in cRCTs [21–25]. Thus short-term classical trials miss the potential to investigate long-term adherence effects on overall treatment outcomes. Mapping cRCT adherence to routine care management assumes 100% compliance to interventions. It seems logical to expect that comparable products with variable adherence rates are likely to have variable asthma outcomes; if products have differential adherence rates in real-life, this may change conclusions.

Figure 3.

Long-term adherence to ICS. Short-term cRCTs will miss the interaction between real-life adherence and treatment outcomes. Reproduced from [20] with permission from the publisher.

Real-life observational database studies and UK government-funded pragmatic trials, also suggest that the modality/route of administration of medication can have an effect on medication persistence. A Canadian observational asthma study compared the persistence with therapy among new leukotriene receptor antagonist (LTRA) patients (n=2200) and new users of ICS (n=2200) [26]. Patients were matched for key clinical features of disease severity to minimise potential confounding by indication. Compliance was found to be significantly better among LTRAs than for ICSs; indeed, if in both groups all medications filled were taken at the prescribed dose, the annual percentage of days on therapy for LTRA users would have been twice that for ICS users (38% versus 19%; p<0.0001). The authors concluded that the data indicate a far from optimal persistence to both LTRAs and ICSs in patients with asthma, but that the relatively superior persistence to LTRAs might result in better effectiveness.

These data bore out in ELEVATE, two 2-year pRCTs designed to investigate the real-life effectiveness of 1) LTRA therapy compared with ICS for patients initiating maintenance asthma therapy and 2) LTRA add-on compared with LABA add-on therapy in patients with uncontrolled asthma despite maintenance ICS therapy [27].

ELEVATE’s pragmatic design differed from that of a cRCT by using broad inclusion criteria and evaluating effectiveness outcomes over an initial 2-month period, but also over a longer-term 2-year period (fig. 2c). The primary endpoint was asthma-related quality of life, a patient-oriented measure of effectiveness, assessed by the Mini Asthma Quality of Life Questionnaire (MiniAQLQ). At 2 months, initiation of LTRAs was equivalent to the guideline-recommended reference alternative treatment strategies in terms of the MiniAQLQ. At 2 years, true equivalence was not shown but there were no significant differences between LTRAs and the alternative treatment strategy for primary or secondary outcome measures. Interestingly, adherence rates were higher for patients randomised to receive LTRA, median 65% adherence compared with 41% for ICS patients. These data suggest a potential preference for oral LTRA therapy over ICS therapy, driving better outcomes for the LTRA patients than may have been anticipated based on evidence from the registration cRCTs alone. The ELEVATE authors concluded that their findings suggest a need for caution when extrapolating results from cRCTs to the broad population of patients with asthma who are treated in community settings and that clinical decisions should take into account data from both cRCTs and pRCTs.

As mentioned earlier, the inherent nature of cRCT design can also positively bias treatment outcomes by insisting on effective inhaler technique throughout the study period. The cRCT optimises the power of the study to evaluate the efficacy of the dispensed therapy but, in the real world, handling errors are common across all devices. Data from 3654 patients receiving the implementing Helping Asthma in Real Life (iHARP) clinical service, found that 85–90% of patients, across all types of asthma inhalers make at least one potentially serious handling error and ∼70% multiple potentially serious errors [28]. Real-life device studies tell us that patients have a range of abilities to use their inhalers correctly (as do their physicians), receive infrequent inhaler (and less-than-perfect) training and may not be fully adherent to their medication. This picture is markedly different to the cRCT environment of perfect inhaler technique, perfect technique training and retraining by expert, specialist staff and daily diary use to promote adherence.

Real-life populations: broader inclusion criteria

Studies that include a more heterogeneous asthma population, one that better reflects that population routinely treated in the community, provide opportunity to explore the interaction between comorbidities, lifestyle factors, patient characteristics and asthma treatment outcomes (fig. 2d).

Rhinitis

Several factors support more than a coincidental association between asthma and rhinitis; they share a similar epidemiology, common triggers, a similar pattern of inflammation and nasal challenge results in asthmatic inflammation, and vice versa.

Yet despite the likely association and high incidence of comorbid asthma and rhinitis, patients with rhinitis are often excluded from cRCTs of asthma therapies [29, 30]. The Clinical Outcomes with Montelukast as a Partner Agent to Corticosteroid Therapy (COMPACT) trial, however, included a broader asthma population, and examined whether asthma patients with comorbid allergic rhinitis responded differently to budesonide plus montelukast than patients without comorbid allergic rhinitis in terms of asthma control (lung function) [31]. In the subgroup of asthma patients with allergic rhinitis, a combined treatment approach that included montelukast and budesonide was found to provide significantly greater reductions in airflow obstruction than were achieved by doubling the dose of budesonide (fig. 4) [32]. The results suggest a treatment approach that targets the airway inflammation common to both diseases may be beneficial for the large proportion of asthma patients who also suffer from allergic rhinitis. They also illustrate the difference in treatment outcomes that real-life features of asthma management (i.e. presence of comorbid conditions) may affect, and the potential limitations in assuming cRCT results hold true across all patients and patient subgroups managed in routine care.

Figure 4.

The COMPACT study demonstrated the difference in outcomes associated with management of only lower airways inflammation (budesonide) compared with systemic (upper and lower airways) inflammation management (montelukast) in asthmatic patients without (a) and with (b) rhinitis. Reproduced from [31, 32] with permission from the publishers.

Lifestyle: smoking

Smoking rates among patients with asthma are similar to those of the general population, yet smoking is another feature of real-life known to interact with asthma outcomes [12]. Active smoking is associated with worsening asthma symptoms, accelerated decline in lung function, and impaired response to corticosteroids and so patients with a current smoking status are routinely excluded from asthma cRCTs [33–35]. The exclusion of smokers aims to minimise a potential source of bias and to evaluate a more direct relationship between the trial drug and its outcomes. Yet it also results in a sparse evidence base to guide clinicians in their asthma management decisions for the large proportion of asthma patients who continue to smoke.

More recently, a number of more pragmatic trials and observational studies have attempted to address this gap in the evidence base by taking a more naturalistic approach to patient selection and including patients with some degree of cigarette exposure.

To explore the interaction of a current smoking status on ICS treatment, patients with mild asthma (n=95) were recruited to a multi-centre, double-blind, parallel-group study to compare the differential outcomes of inhaled beclomethasone dipropionate (BDP) at doses of 400 or 2000 µg in smokers and nonsmokers. Patients in both the smoking and nonsmoking arms were randomised to receive either 400 or 2000 µg BDP and the change in their morning peak expiratory flow (PEF) was measured over a 12-week outcome period. After 12 weeks of inhaled BDP therapy, there was a considerable difference in changes in morning PEF between smokers and nonsmokers. Interestingly, this difference was less pronounced in the 2000 µg BDP arm than in the 400 µg BDP arm (fig. 5). In other words, the magnitude of treatment effect on morning PEF was much greater in nonsmokers than in smokers at the 400 µg dosage, while it was similar at the high dosage. Similarly, while there was a significantly higher exacerbation rate among smokers compared with nonsmokers in the 400 µg BDP treatment arm (6 versus 1, respectively), no significant difference was seen in exacerbation rates in patients receiving the high 2000 µg BDP daily dose (1 versus 2, smokers versus non-smokers, respectively). These data not only build on the results of studies showing diminished ICS efficacy in mild smoking asthma patients treated on low-dose ICS, but also suggest the disparity between treatment response in smokers and non-smokers appears to be reduced at higher ICS doses and may support consideration of higher dose ICS use in asthma patients with a current smoking status [34, 36].

Figure 5.

Mean (95% CI) difference between non-smokers and smokers with asthma, Suggesting alternatives to higher-dose ICS may be required. *: p≤0.01 for smokers versus non-smokers. Reproduced from [36] with permission from the publisher.

Another treatment option with potential utility in smoking asthmatics is leukotriene modifiers [37]. The largest double-blind asthma cRCT involving asthmatic smokers conducted to date (albeit, a population of relatively mild asthma patients with limited smoking history) randomised to montelukast (10 mg once daily; n=347), medium-dose fluticasone (250 µg twice daily; n=336), and placebo (n=336), respectively. Over the 6-month outcome period, patients in both the fluticasone and montelukast treatment arms demonstrated significantly more days of asthma control than placebo and also significant improvements on the secondary endpoint of change from baseline in mean daytime symptom score were observed, compared with placebo, but no significant differences between montelukast and fluticasone. Smoking history and exposure appeared to play a role in response to therapy: patients who had a smoking history of <11 pack-years tended to show more benefit from fluticasone while those with a smoking history of >11 pack-years tended to show greater numerical benefit (in terms of asthma control) with montelukast. Overall, the study authors concluded that the data support consideration of montelukast as a new treatment option, an alternative to high-dose ICS, for asthma patients who cannot manage to stop smoking.

Lifestyle: overweight and obesity

The incidence of both asthma and obesity has increased across the developed nations in recent years [38, 39]. Indeed, incidence of asthma has been positively associated with obesity and evidence suggests that obesity may be associated with increased asthma severity [40]. To explore the potential interaction of body mass index (BMI) and asthma treatment outcomes, Peters-Golden et al. [41] performed a post hoc analysis of four registered placebo-controlled trials studies that randomised over 3000 adult patients with moderate asthma to treatment with placebo, montelukast or ICS (beclomethasone). Patients were categorised as: BMI normal (<25.0 kg⋅m⁻²; 52% of patients), overweight (25–29.9 kg⋅m⁻²; 32%) and obese (≥30.0 kg⋅m⁻²; 16%) categories. Treatment groups were balanced for BMI, demographic characteristics and parameters of asthma control. The placebo response for the primary end point, asthma control days, was generally lower with increasing BMI. This held true across the secondary endpoints: forced expiratory volume in 1 s (FEV₁), reliever use and nocturnal awakening. Similarly, ICS response decreased with increasing BMI, yet the response to leukotriene antagonist therapy remained stable across all three BMI categories. The authors concluded that their post hoc findings not only suggest BMI may influence the natural history of asthma control, but also that the effect may be class specific and that future prospective studies should explicitly evaluate the potential differential influence of BMI on active agents.

Comorbidities: long-term therapeutic profiles

Studies that take a more pragmatic approach to patient selection and harness long-term observational data can also be used to explore the adverse events profile of chronic asthma therapies.

Using data from the Quebec health insurance databases, a new-user cohort study of respiratory patients (treated during 1990–2005) were followed until 2007, or until diabetes onset to evaluate any potential association of high-dose ICS and diabetes risk. A nested case–control analysis was used to estimate the rate ratios of diabetes onset and progression associated with current ICS use, adjusted for age, sex, respiratory disease severity and co-morbidity.

The cohort included 388  584 patients, of whom 30  167 had diabetes onset during 5.5 years of follow-up. Current use of ICS was associated with a 34% increase in the rate of diabetes and rate of diabetes progression (RR 1.34, 95% CI 1.17–1.53). Moreover, the risk appeared to be dose related.

Again, these data demonstrated not only the value of more naturalistic study designs in terms of offering evaluation of longer-term outcome periods in an affordable, feasible way, but also the importance of considering the patient and their comorbidities rather than an isolated disease when selecting the optimum intervention.

Using real-life evidence to improve asthma management

As discussed, high-quality pRCTs and observational studies are already generating useful insights into the interactions between features of real-life asthma management and treatment outcomes. The next step is to improve understanding of how the available data should be used to help guide and inform clinical decision-making, which require the development and application of quality standards for the design and appraisal of real-life studies.

To support this next step, the Respiratory Effectiveness Group (REG) has published a checklist of key quality criteria that researchers and reviewers should consider when designing and reviewing real-life studies [42]. A taskforce has since been established, involving REG’s leading respiratory and academic experts in real-life research, in collaboration with the EAACI, to build and develop quality appraisal tools that the taskforce will use to identify high-quality, published asthma comparative effectiveness studies that address gaps in the cRCT evidence base and associated guidelines [43]. Studies deemed to be of high quality by the taskforce will be interpreted to establish what additional data they contribute, seeking to avoid and clarify any potential discrepancies between cRCT and real-life study findings.

Although some clinicians and academics have raised concerns that real-life evidence sometimes conflicts with that from cRCTs, this perception is largely one of misassigning the research question that the respective studies were seeking to address. To return to guideline recommendations on the optimum second step in asthma treatment, ICS is recommended as the most effective preventer drug. As earlier mentioned, this statement would be better if it were qualified that ICS is the most effective preventer drug in the populations and clinical environments provided by the trials: i.e. in short-term use for those with perfect inhaler technique, substantial lung function impairment, 15% reversibility, if they do not smoke and have very good medication compliance. The real-life studies discussed in this paper that support consideration of LTRA therapy rather than ICS do not conflict with the classical trials, rather they relate to treatment options for smoking asthmatics, for asthma patients with comorbid rhinitis and for patients with suboptimal adherence to inhaled therapy who may benefit from an oral alternative (perhaps a substantial proportion of the asthma population).

Conclusions

Comprehensive assessment of therapeutic strategies requires evaluation of both their efficacy under optimal conditions (high internal validity) and effectiveness in real-life populations and situations (high external validity). Various study designs are required to answer different questions about appropriate and optimum use of available strategies and they should be viewed as complementary approaches to clinical research. Before integrating the results of these studies into guidelines and clinical practice, the first prerequisite is the description of the study characteristics in terms of ecology of care and studied population; both major determinants of the generalisability of results.

Many (if not most) of the decisions that clinicians face have not (and cannot) be adequately tested in classical trials in the appropriate practice context. The movement towards more real-life studies is a reaction to the unmet clinical challenges faced by clinicians every day.

Features of the real world interact with cRCT therapeutic efficacy to produce the clinical outcomes seen in the community. It is inappropriate to assume a one-size-fits-all approach and that cRCT findings and guideline recommendations can be effectively extrapolated to all patients managed in routine care. By excluding important real-life factors (extraneous variables), the internal validity of a trial and “robustness” of the data increase, but only for the increasingly narrow group of patients for whom the results are relevant. Few community-based patients are managed in the idealised settings of a cRCT; some patients have only one diagnosis but most have co-morbidities, and real-life co-morbidities and lifestyle factors often result in different responses to medication, responses that are not captured if such patients are routinely excluded from cRCTs.

What is required is a more careful, thoughtful approach to guideline development and clinical care to avoid a one-size-fits-all approach to recommendations. While meta-analyses of trials have their place, and may form the basis of some guideline recommendations, their conclusions need to be relevant and tailored to the population under consideration.