Advancing Outcome Measures for the New Era of Drug Development in Cystic Fibrosis

Abstract

The growing pipeline of candidate drugs for cystic fibrosis (CF) is challenging clinical trial research. There has been a shift from evaluating drugs aimed at treating the secondary manifestations of CF to evaluating drugs targeted toward the primary prevention of chronic lung disease. As CF is an orphan disease, there is a fundamental need to assess new therapies efficiently and accurately by mechanisms that best use the number of available patients. This need can be addressed with the continued advancement and refinement of CF outcome measures. We begin by presenting an overview of the outcome measures currently used in CF clinical studies, defined and categorized in terms of one of the three main classes of endpoints: clinical efficacy measures, surrogate endpoints, and biomarkers. To move forward efficiently, clinical research in CF is dependent on the development of new outcomes able to capture biologic and clinical response to novel therapeutic approaches. We conclude with a discussion of the criteria by which all new outcome measures should be evaluated. A systematic, rigorous approach to outcome measure development is needed to provide the tools necessary for evaluating new therapies and moving drugs out of the pipeline and into the CF clinic.

Cystic fibrosis (CF), a life-shortening, autosomal recessive disorder, affecting over 30,000 individuals in the United States and 60,000 worldwide, is caused by mutations in the gene encoding the CF transmembrane conductance regulator (CFTR) (1), and is characterized by pathophysiologic sequelae that reflect the reduction or complete absence of protein function (2). The protein regulates electrolyte and fluid content on the epithelial surfaces of multiple organs, including the lung, pancreas, liver, vas deferens, and sweat glands (3). In the lung, dysregulated chloride and sodium channel function leads to decreased airway surface liquid depth (4), which likely contributes to the common pathologic finding of dehydrated, inspissated mucus secretions, leading to plugging and dilatation of airway lumen. Although there are several hypotheses linking CFTR dysregulation to the chronic endobronchial bacterial infections and associated neutrophilic inflammation of CF lung disease (5), the factors leading to initial airway infection remain an active area of scientific investigation (6).

Since the discovery of the gene, the significant advances in scientific knowledge about the molecular basis of this disease have afforded many opportunities to develop new therapeutic approaches directed at both the basic defect and secondary consequences of CFTR dysfunction. Yet, as an orphan disease defined in the United States as affecting less than 200,000 individuals (7), translational research can be challenging due to the small number of patients available for study. This reality can slow enrollment in large, pivotal clinical trials, reduce the number of candidate therapeutics that can be tested, and discourage pharmaceutical development in this disease (8). To meet these challenges, the CF scientific community led by the Cystic Fibrosis Foundation (CFF) has initiated a multipronged approach to reduce barriers to the drug development process. First, the CFF has developed a drug discovery program focusing on high-throughput screening and combinatorial chemistry, which has resulted in the identification of several encouraging drug candidates that enhance CFTR intracellular trafficking (correctors) or CFTR ion channel function (potentiators) (9, 10), including one drug already in early-phase human trials. Second, a clinical trials network, the CFF Therapeutics Development Network (CFF TDN), has been established to assist industry in designing and implementing early-phase trials (11). Third, the CFF has initiated an extensive family education and advocacy program to encourage patient participation in the drug development and clinical research process. Fourth, the CFF has supported the ongoing development of CF outcome measures through several conferences and working groups, recognizing that the linchpin for successful therapeutic development and U.S. Food and Drug Administration (FDA) registration is the establishment of optimal outcome measures and study designs that efficiently demonstrate safety and efficacy.

In 1992, the CFF convened an initial consensus conference in collaboration with the FDA to better define outcome measures for the evaluation of new CF therapies (12). The summary of the conference provided an overview of the “state of the art” for pulmonary outcomes, and outlined priorities for the development of improved biological markers and clinical efficacy measures. Emphasis was placed on efforts to improve accuracy and validity of available measures, especially in children younger than 6 years of age (12). Since that conference, significant progress has been made in the identification of efficacious new therapies for CF. Most notably, two therapies have been FDA approved specifically for treatment of CF lung disease. In 1993, recombinant human DNase was approved as a daily administered mucolytic agent (13). In 1998, the FDA approved tobramycin solution for inhalation as maintenance therapy for patients colonized with Pseudomonas aeruginosa (14). Two additional pulmonary therapies have also reached phase 3 trials, which are currently in progress (15, 16). The CF community has also undertaken major efforts to evaluate the effectiveness of therapies approved for non-CF indications through randomized, placebo-controlled clinical trials. These trials include evaluation of ibuprofen (17), azithromycin (18–21), and hypertonic saline (22, 23). All of these studies demonstrated significant efficacy and represent major contributions to the intervention options available to treat secondary symptoms of CF lung disease in patients six years and older. They also represent a remarkable step forward in the conduct of well designed, adequately powered, multicenter clinical trials, which have overcome the limitations observed in early CF clinical research (24).

The past 15 years have provided further advances in CF outcome measures. The studies described above have established the use of pulmonary exacerbations and pulmonary function (e.g., FEV₁) as key efficacy measures able to capture clinically meaningful responses to therapeutic intervention. In addition to these more established outcomes, there have been significant advances in the use of infant pulmonary function testing (PFT) (25, 26) and chest computed tomography (CT) in CF (27), as well as the use of biological markers, such as nasal potential difference measurements (28, 29).

To define the current status and next steps in the development of several key outcome measures used for clinical research in CF, the CFF convened a workshop at the annual CFF TDN meeting in April 2006 in Seattle, Washington. The proceedings summarized in this article reflect the keynote presentation provided at this workshop. Subsequent articles included in this symposium summarize the discussions and recommendations of each of the outcome measures–specific working groups: (1) pulmonary exacerbations and patient-reported outcomes (PROs); (2) outcome measures for young children, including infant pulmonary function and CT; (3) inflammatory markers; (4) CFTR functional measures, including nasal potential difference and sweat chloride; and (5) mucociliary clearance.

This article begins with an overview of current CF outcome measures, which are categorized according to a well established framework that distinguishes between three main types of outcomes: clinical efficacy measures, surrogate endpoints, and biomarkers. In addition, we will review the fundamental characteristics for which all outcome measures should be evaluated to define their role in the drug development process. In doing so, we highlight the challenges faced in developing new therapies that are intended to prevent the onset of pulmonary disease rather than symptoms and secondary consequences.

CLINICAL EFFICACY MEASURES

The FDA defines a clinical efficacy measure as “a characteristic or variable that reflects how a patient feels, functions, or survives” (Table 1) (30). The ultimate endpoint in a life-shortening disorder such as CF is survival, but there are other outcome measures that are considered intermediate clinical efficacy measures. Intermediate endpoints are clinical efficacy measures that are not the ultimate endpoint, but are nonetheless of real clinical benefit and reflect how the patient is feeling or functioning (31). Clinical efficacy endpoints are the gold standard for definitive clinical trials seeking FDA approval. Examples of common outcome measures considered to be clinical efficacy measures in CF include pulmonary exacerbations and their treatment (e.g., intravenous antibiotics) and hospitalizations. These intermediate endpoints, in addition to demonstrated safety, led to the FDA approvals of recombinant human DNase (13) and tobramycin solution for inhalation (14).

TABLE 1.

OVERVIEW OF CYSTIC FIBROSIS OUTCOME MEASURES

Outcome Measures	Description
Clinical efficacy measures
Definition	A characteristic or variable that reflects how a patient feels, functions, or survives
Examples	Pulmonary exacerbations, intravenous antibiotic usage, hospitalizations, anthropometric measures (height and weight), PROs, quality of life
Surrogate endpoints
Definition	A laboratory measurement or physical sign that is used in therapeutic trials as a substitute for a clinically meaningful endpoint that is a direct measure of how a patient feels, functions, or survives and is expected to predict the effect of therapy.
Examples	FEV1, CT, infant pulmonary function
Biomarkers
Definition	A characteristic that is objectively measured and evaluated as an indicator of normal biologic process, pathogenic process, or pharmacologic response to a therapeutic intervention.
Examples	Bacterial density, inflammatory markers, CFTR functional measures (nasal potential difference, sweat chloride, airway surface liquid height), mucociliary clearance

Definition of abbreviations: CFTR = cystic fibrosis transmembrane conductance regulator; CT = computed tomography; PROs = patient-reported outcomes.

CF pulmonary exacerbations continue to be a key clinical efficacy measure in definitive clinical trials, and are generally defined as a compilation of patient signs and symptoms that often result in the need for aggressive treatment, including the use of intravenous antibiotics that may require hospitalization. There is no standard definition, however, of a pulmonary exacerbation, and the large multicenter clinical trials that have been conducted over the past 15 years have used many variations of physician-derived definitions (13, 14, 32–34). In a recent guidance by the FDA, PROs are described as the most meaningful approach for collecting sign and symptom data, because these represent direct measures of how a patient feels and functions (35). The CFF is thus providing support for the development of PROs and standardization of the exacerbation definition, as well as the continued development of a CF-specific quality of life questionnaire that has been successfully used to capture patient-perceived improvement in response to therapy (36). These outcomes are further discussed in the article in this symposium by Goss and Quittner (pp. 378–386). Ultimately, patient-reported evidence of improved symptoms will not be sufficient on its own to approve the clinical efficacy of a new therapy under FDA guidelines, but it may be a necessary body of evidence to support the findings of indirect measures of clinical improvement, such as decreased hospitalization rate or improved pulmonary function.

The determination of whether an outcome measure captures tangible benefit to the patient is notably subjective. In the case of drug development, the acceptance of an outcome measure as a true clinical efficacy measure is highly dependent on whether the measure may be used as a primary endpoint included in the submission of a New Drug Application. One example of this is improvement in pulmonary function (e.g., FEV₁), which is considered by many in the CF community to be a direct measure of a patient's increased ability to function. Improvement in FEV₁ has been used in the registration trials for recombinant human DNase (13) and tobramycin solution for inhalation (14) as a key efficacy measure. However, FDA approval in both instances required data on additional clinical efficacy measures, such as intravenous antibiotic use and hospitalization, to confirm tangible benefit to the patient. Further discussion on the use of FEV₁, including its use as a surrogate for survival, is included in the next section discussing surrogate endpoints.

It is likely that the most commonly accepted clinical efficacy measures, including pulmonary exacerbations, will become less sensitive to effective therapeutic interventions as the overall health and survival of the patients improve (e.g., the CF community becomes a “victim of its own success”). In addition, there will be limitations in the ability to demonstrate improvements in clinical efficacy measures in the younger patient population for whom chronic disease has not yet been established. For example, the rate of exacerbations, as defined by Rabin and colleagues (33), was estimated to be 42% among patients of all ages over a 6-month observation period. However, the rate of exacerbations was only 32% among patients between 6 and 12 years of age, and 23% among patients less than 6 years of age. These rates directly impact the sample sizes needed to detect clinically meaningful treatment effect sizes between an intervention and a control group. Lower event rates translate into larger sample sizes needed to detect smaller, yet still meaningful, effect sizes. Clinical efficacy measures, such as pulmonary exacerbations and hospitalization or intravenous antibiotic use, will therefore continue to be more optimally evaluated in patient populations with moderate to severe lung disease.

For younger patient populations, clinical efficacy measures, such as linear growth and weight gain, may be preferable for evaluating new therapies. In the Wisconsin study evaluating the impact of newborn screening on long-term clinical outcomes (37), linear growth and weight gain in the critical first years of life were key parameters that demonstrated the importance of early nutritional intervention and enzyme replacement in the health and development of children with CF. Linear growth and weight gain have been useful long-term measures of both clinical efficacy and safety for several therapeutic interventions. As a safety parameter, linear growth played a key role in defining the therapeutic index for long-term administration of oral glucocorticoids (38). Although significant improvement in pulmonary function was demonstrated at 24 and 48 months in patients receiving oral prednisone every other day as compared with the placebo arm, a decline in linear growth velocity was demonstrated by 24 months. Furthermore, the highest-dose treatment arm in the trial had been stopped early because of a similar finding of decline in growth velocity after 6 months of therapy (38). Even more concerning, a long-term follow-up study employing the CFF national registry evaluated linear growth in the treatment groups 6–7 years after cessation of prednisone therapy, and found that catch-up growth did not occur until 2 years after treatment was discontinued (39). In addition, z scores for height remained significantly lower in males, averaging 4 cm less in the treated group as compared with the placebo group. This difference was not seen in females. Based upon these troublesome findings, long-term, oral, alternate-day prednisone is not recommended in growing children with CF (6).

Height and weight velocities were also used as primary efficacy measures for several recombinant human growth hormone trials, demonstrating improved growth in children with CF whose height and weight were less than or equal to the 25th percentile (40–42). In addition, the phase 3 efficacy trials of inhaled tobramycin demonstrated improved weight gain in the treatment group (43). It is not surprising that height is an important clinical efficacy measure, as it positively correlates with pulmonary function (44) and survival (45). It is likely that linear growth will be more useful as a long-term rather than a short-term outcome measure, and statistical considerations must be given to other confounding effects on growth, such as infection with P. aeruginosa, baseline lung function, and diabetes mellitus (12).

As we move forward with outcome measures development in CF, clinical efficacy measures are of utmost importance. New therapies that prevent lung disease progression will likely not only need to be evaluated in young patient populations for whom chronic lung disease has not yet been established, but it may also take several years of evaluation of these therapies to determine their true impact on clinical efficacy. It is this setting for which the use of surrogate endpoints may be more feasible for evaluating new therapies than established clinical efficacy measures.

SURROGATE ENDPOINTS

The FDA defines surrogate endpoints as “a laboratory measurement or physical sign that is used in therapeutic trials as a substitute for a clinically meaningful endpoint that is a direct measure of how a patient feels, functions, or survives and is expected to predict the effect of therapy” (Table 1) (31). Surrogate endpoints are generally used as a substitute for true clinical efficacy measures when the clinical benefit may not be detectable in trials of reasonable cost, duration, or size, or in proof-of-concept or safety trials (30, 46–52).

FDA regulations state that a surrogate endpoint is considered to be “reasonably likely” to predict clinical benefit and, therefore, useable for drug approval if there is evidence based on epidemiologic, therapeutic, pathophysiologic, or other data supporting the association of the surrogate with the clinical benefit (51). Such an endpoint would not be considered to be validated unless there was evidence that the effect of the drug under study on the surrogate endpoint predicts the drug's effect on the clinical endpoint of interest (51). In CF, both FEV₁ at a given time point and the rate of decline in FEV₁ can be considered surrogate endpoints because they are well established predictors of survival (53–61). Phase 3 trials previously noted in this review have demonstrated significant improvements in FEV₁ in response to therapy over a 6-month time period (13, 14, 18, 20). By contrast, there is only one example in the CF literature of a therapeutic trial involving twice-daily high-dose ibuprofen that has demonstrated an effect on slowing the rate of FEV₁ decline (17). As opposed to clinical trials evaluating acute improvement in FEV₁, clinical trials that assess change in the rate of pulmonary function decline require either a much larger sample size or longer duration of follow-up (62). The ibuprofen trial, for example, required 4 years of follow up, which is often not practical in the CF disease setting. Nonetheless, the body of epidemiologic evidence that supports the importance of pulmonary function as a marker of long-term disease severity and survival provides community support for the use of FEV₁ as a surrogate measure of clinical efficacy in definitive trials. However, validation of this use needs to be furthered by directly correlating improvement in FEV₁ resulting from specific treatments to longer term clinical benefit, including survival. Studies evaluating the long-term outcomes of patients in successful therapeutic trials can be facilitated with the use of the CFF national registry, a large observational database that includes approximately 80% of the United States population of patients with CF (63).

The need for surrogate endpoints has increased with the growing pipeline of novel CF treatments that are intended to have the potential to correct the underlying genetic defects of the disease. Surrogate measures could be critical for achieving approval of these new therapies within reasonable time frames as compared with using traditional clinical efficacy measures, which could take years. There are important advances in the CF research setting that are beginning to position select outcome measures as potential surrogate endpoints. For example, CT, which is discussed in the article in this symposium by Davis and colleagues (pp. 418–430), has been proposed as a candidate surrogate that is able to demonstrate early injury to the lung before observable clinical signs and symptoms (27, 64–67). There will continue to be a need for developing surrogate endpoints that are more sensitive indicators of response to therapy in the early disease state.

One of the major challenges for developing validated surrogate outcome measures has been a lack of consensus on specific guidelines for the validation process (30). Fortunately, recent attention has been directed to better outlining the key elements of the validation approach. One key requirement is a “bridging study” that includes the surrogate endpoint and the clinical endpoint in the same trial (47). This enables quantification (with error) of the effect of the surrogate needed to produce an effect on the clinical outcome measure. Although it is possible to obtain drug approval based on nonvalidated surrogate endpoints via the FDA's “Accelerated Approval” regulations established in the early 1990s (51), the FDA has expressed its strong preference for validated surrogate endpoints when clinical endpoints are not feasible. The validation of a surrogate endpoint must be specific to both the disease and class of interventions, such that validation of a surrogate endpoint in CF for one class of interventions (e.g., antimicrobial therapy) may still need to be validated for other classes of drugs (e.g., antiinflammatory therapy). This requirement suggests the need for several bridging studies to achieve the ideal rigor of validation for a surrogate endpoint.

As one can easily surmise, the validation process is quite difficult, even with ample patient populations. In the orphan disease setting of CF, validation according to the ideal rigor will be difficult and, in some instances, impossible. For example, infant PFT, which is discussed in this symposium (Davis and colleagues), is a relatively new clinical outcome measure that enables the assessment of lung function in patients with CF less than 3-years of age. Although this measure will be useful for evaluating short-term improvements in pulmonary function, it will be more challenging to validate this outcome measure as a surrogate endpoint, because it will take a large study of many years duration to show a relationship of infant PFT with longer term clinical outcomes, including standard PFT and survival.

A recent example of FDA approval of a drug based on a nonvalidated endpoint is fabrazyme, for use in the population of patients affected with Fabry disease, which affects approximately 5,000 individuals worldwide (68, 69). The approval was based on statistically significant improvements in the clearance of a compound called globotriaosylceramide (GL)-3 from the kidney in the active group relative to the placebo group. This improved clearance was expected to predict longer term stabilization of kidney function and, hence, clinical benefit. In this situation, there was supportive natural history data to correlate GL-3 clearance with kidney function and clinical outcome, but there was no prior bridging study quantifying the amount of GL-3 clearance needed to improve clinical outcome. The approval of fabrazyme was, therefore, contingent on postmarketing studies evaluating the long-term outcomes of patients on this drug. This example provides an opportunity in the CF setting to consider the use of specific biological markers of the disease as surrogate endpoints that can enable more rapid approval of novel therapies able to correct or modify the primary defect of CF. The development of outcomes able to sensitively measure CFTR function are critical in this setting, and, although approval may be possible using such surrogates, long-term follow up studies are a necessity for evaluating the longer term clinical benefit and safety.

Although the opportunity exists to approve drugs based on a nonvalidated surrogate endpoint, there is no precedence in the CF setting to date. Furthermore, controversy still exists among the FDA and its consultants regarding the potential for approving drugs based on nonvalidated surrogate endpoints that could confer no actual clinical benefit, and possibly even be harmful (48, 52). Despite this controversy, there is a common understanding of the need to build efficiency in the drug approval process for the orphan disease setting. As CF researchers, our role should then be to identify potential surrogates and collaborate with the FDA regarding what data are needed for establishing the validity of these endpoints to predict clinical benefit. One very important resource for identifying candidate surrogate endpoints is among the vast number of CF biomarkers that comprise the third main class of CF outcome measures.

BIOMARKERS

Biomarkers are defined as “a characteristic that is objectively measured and evaluated as an indicator of normal biologic process, pathogenic process, or pharmacologic responses to a therapeutic intervention” (Table 1) (70). Often, the terms biomarker and surrogate are used interchangeably, but, according to the framework described in this article, surrogate status is reserved for those biomarkers that are intended as a substitute for a true clinical efficacy measure. By definition, all biomarkers should demonstrate biological activity. A subset of these will be demonstrated to be correlates of clinical efficacy, and a subset of these which are able to further predict clinical benefit will be candidate surrogate endpoints. Importantly, biomarkers do not need to be proven surrogate endpoints to play a critical role in drug development (71). As measures of biological activity, biomarkers enable early proof-of-concept studies that can help screen potential drug candidates and identify therapeutic targets. As correlates for clinical efficacy, biomarkers provide the impetus for larger studies evaluating clinical efficacy measures. Although biomarkers are critical in the decision-making process in early phase 1 and 2 trials, they are also important in later phase trials for increasing our knowledge of mechanism of action and enabling identification of treatment responders (49).

There are numerous biomarkers used in the CF clinical research setting, many of which are highlighted in the subsequent articles in this issue. These include a vast panel of markers of inflammation (Sagel and colleagues, this symposium, pp. 406–417), mucociliary clearance measures (Donaldson and colleagues, this symposium, pp. 399–405), and CFTR functional biomarkers, including nasal potential difference and sweat chloride (Rowe and colleagues, this symposium, pp. 387–398). In contrast to these relatively new biomarkers, one of the more established sets of biomarkers in CF includes microbiological parameters, and, in particular, those relating to P. aeruginosa. There have been several natural history studies demonstrating that the presence of P. aeruginosa is associated with poorer clinical outcome (53, 72–80). Studies have also demonstrated that the transition to more virulent mucoid strains correlates with poorer clinical outcomes (77, 81). Furthermore, the phase 3 study of tobramycin solution for inhalation demonstrated significant reductions in P. aeruginosa bacterial density in addition to improvements in FEV₁ and reduction in exacerbation outcomes within the same cohort of patients (14). These data have provided support for the use of bacterial density in the early evaluation of other antibiotics targeting P. aeruginosa.

As our knowledge of the basic genetic defects of CF continues to increase, the need for biomarkers able to capture information regarding the ability of new therapies to impact on CFTR dysfunction will also continue to grow. As discussed in subsequent papers in this issue (Rowe and colleagues), biomarkers, such as nasal potential difference and sweat chloride, are leading candidates for assessing drugs aimed at altering CFTR function. Alternative markers of CFTR function, including airway surface liquid height and other bioelectric parameters, are also being pursued aggressively among the U.K. CF Gene Therapy Consortium (82).

One of the major challenges presented with the use of biomarkers is whether there are sufficient data to provide the confidence needed for making key critical drug development decisions related to dose selection and proof of concept for later phase trials. Fortunately, significant efforts have been put forth to maximize the use of existing biomarker data. In particular, the CFF is leading efforts toward the establishment of specimen and data banks that will facilitate evaluation of new biomarkers for years to come. These banks enable the use of data from multiple clinical trials, which will be important for evaluating biomarkers, both with larger numbers than can often be obtained in a single study and among a broader patient population. As an example, a retrospective study has recently been completed using the CFF TDN data bank to combine data from four completed CF clinical trials to investigate associations between pulmonary function and sputum biomarkers of infection and inflammation (83). These data provided a more diverse and larger CF population with which to explore these associations than has been examined in previously published, small, single-center studies (84, 85). This study is an example of the exciting potential for data and specimen banks to enable the validation of biomarkers as correlates of disease severity.

Although establishing biomarkers as correlates of disease severity and clinical outcome is desirable, nearly all biomarkers must first be validated as reliable measures of biological activity. Described in the subsequent section is a systematic and consistent process for the development of not only biomarkers, but all new outcome measures in CF. Evaluating CF outcome measures against key characteristics that define a usable outcome in a clinical study will ultimately establish their optimal role in the drug development process.

KEY PRINCIPLES FOR THE ONGOING EVALUATION OF CF OUTCOME MEASURES

Outcome measures development should be viewed as a continuous process that must be defined uniquely for each outcome measure. The extent of rigor in the development of an outcome measure is dependent on its intended usage within the drug development process (30, 86, 87), and differs between clinical efficacy measures, surrogate endpoints, and biomarkers. Even established outcome measures can be further developed with the ongoing accumulation of new data. To advance CF outcome measures, it is important that the CF research community accepts a consistent and rigorous approach to outcome measurement development. We thus highlight the key characteristics for which all CF outcome measures should be evaluated (30, 71, 87, 88): (1) clinical and biological relevance; (2) sensitivity and specificity to treatment effects; (3) reproducibility; and (4) feasibility.

Clinical and Biological Relevance

Defining the physiologic or pathogenic process that the outcome measure is proposed to capture is of primary importance. Although technology is often an impetus for the development of some biological markers, motivation should first be based on clinical and biological relevance. Within the context of drug development, this relevance needs to be further defined in relation to each class of drugs to be evaluated. Key questions that should be addressed include the following:

• Is the outcome measure specific to a process occurring either early or late in the disease process?

• Is the outcome measure in the causal pathway of the drug?

• How is the outcome measure expected to change in response to the drug?

• How long will it take to see an expected change in response to the drug?

These questions require knowledge of not only the mechanism of the disease process, but also the mechanism of action of the drug, and represent the scientific rationale that is the basis for all clinical studies required in the drug development process. The ongoing accumulation of data from each study enables us to better understand and measure these mechanistic processes and to better refine answers to these questions. These questions will be of key focus for subsequent articles in this issue, which will define the biological relevance of outcome measures, such as inflammatory markers, CFTR functional measures, and mucociliary clearance in early-phase drug development.

Sensitivity and Specificity to Treatment Effects

The ideal outcome measure should be sensitive (i.e., demonstrate meaningful changes in response to a successful therapy). Furthermore, changes in the outcome measure should be specific to the mechanistic changes in the disease process resulting from therapy. Evaluating outcome measures with respect to their sensitivity and specificity is quite complex. There are two requirements for this process: (1) sensitivity can only be fully assessed with the use of a therapy that is successful at modifying the disease process; and (2) specificity can only be fully assessed if the relevant mechanisms of action of the therapy are known. For example, P. aeruginosa bacterial density is an established biomarker, which has been shown to be sensitive for demonstrating clinically meaningful reductions in response to tobramycin solution for inhalation, an antipseudomonal antibiotic (14). There are also supportive data that suggests that reductions in bacterial density within a patient in response to antimicrobial therapy are temporally related to an improvement in lung function (89). In addition, it is well documented in vitro that bacterial killing in CF sputum occurs in a dose-responsive fashion consistent with the mechanism of action of the drug rather than unanticipated mechanisms (90, 91). Thus, reduction in P. aeruginosa density is a sensitive and specific outcome measure for antimicrobial treatments in CF.

Unfortunately, infrequent “successes” in drug development result in few opportunities to establish the sensitivity of many outcome measures. It is critical that forethought is given to include outcomes of particular interest in studies throughout the clinical development process so that such opportunities are not lost. In complex disease settings such as CF, the challenges of fully establishing the specificity of an outcome measure in relation to a specific mechanism of action must be recognized (52). Acceptance of an outcome measures must, therefore, allow flexibility with regard to these criteria, especially for early biomarkers, where it may be difficult to define these attributes. Sensitivity and specificity are of greater importance for later phase drug development, where accepted clinical efficacy measures or validated surrogate endpoints are necessary.

Reproducibility

The reproducibility of an outcome measure is also referred to as its precision, and refers not only to inherent physiologic variation of the measure within a patient over time, but also the measurement process itself. The degree of reproducibility of the measurement directly impacts the sensitivity and specificity of the outcome measure, and, thus, it is desirable to establish reproducibility very early in the development process. Studies evaluating the reproducibility of an outcome should focus on the following questions: (1) Are measurements repeatable within a patient using the same standardized process? Studies addressing this question are performed using repeated measurements from patients at a single center to minimize measurement error and focus on the inherent biologic variation of the measure. (2) How robust is the repeatability of the outcome measure to differences in the measurement process? Studies addressing this question ideally would evaluate the concordance of measurements collected by different sites, laboratories, technicians, or physicians. The goal is to develop a standardized measurement collection process demonstrating consistent and acceptable variability across sites.

In the CF literature, there are several studies that have characterized the reproducibility of outcome measures, such as induced sputum inflammatory markers (92, 93) and sweat chloride (94). In this symposium, Rowe and colleagues (pp. 387–398) discuss the extensive efforts of the CFF TDN to improve reproducibility and decrease variance in the nasal potential difference measurement for multicenter studies. As recognized in these studies, reproducibility is heavily influenced by the measurement process, and efforts to minimize measurement error should be a major focus in the development of all outcome measures. A recent clinical trial evaluated the use of centralized spirometry procedures, which were able to reduce variability in spirometry measures both within and across sites, ultimately reducing measurement error and improving the statistical power to detect differences between treatment groups (95). Minimizing measurement error cannot only be accomplished at the time the measurement is made, but it can also be achieved using novel analytic approaches (96). For instance, statistical approaches for estimating changes in FEV₁ have been pursued using a repeated measures modeling approach that incorporates information about the expected treatment effect into the model (97). This modeling approach provides a flexible estimation framework that improves statistical power and decreases sample sizes as compared with traditional analytic approaches.

Feasibility

Establishing the feasibility of a clinical outcome measure is a multifaceted process that should evaluate not only the financial requirements of the measurement process, but also ethical considerations and patient burden. This evaluation will differ according to the stage of drug development and patient population for which the outcome measure will be used (8). For early-phase studies, it may be reasonable to implement less feasible measurements in terms of cost and patient burden than in later phase studies. For example, the use of nasal potential difference measurements in a clinical study introduces challenges with respect to limiting the study to specialized centers and finding patients who are willing to undergo the lengthy procedure. However, this outcome measure can offer very informative data in the early evaluation of therapies that could modify CFTR function, and alternative outcomes for measuring this biological activity are limited. Feasibility in terms of patient burden and ethical considerations will also differ according to the patient population, as clearly evidenced by differences in drug development for pediatric versus adult patient populations.

CONCLUSIONS

The overview of CF outcome measures presented in this article illustrates the breadth of outcome measures currently available for evaluation in CF clinical studies. However, even for established outcome measures, there is still room for continued refinement and improvement. Moving forward, an organized and focused approach to outcome measure development will provide valuable efficiencies in CF drug development, but it will be necessary to reach community consensus regarding both the intended use of each outcome and the rigor of development that should be completed to support this usage. The advancement of clinical outcome measures to support the new era of CF drug development will ultimately provide the definitive safety and efficacy data needed to move drugs out of the pipeline and into the clinics. The reader will see in the subsequent five articles in this symposium the significant steps that CF investigators worldwide are taking to improve and develop biomarkers, surrogate endpoints, and clinical efficacy measures as outcome measures for CF clinical studies.