Patient-reported Outcomes in Cystic Fibrosis

Abstract

Over the past 20 years, there has been tremendous progress in the area of patient-reported outcomes (PROs). A PRO instrument is defined as any measure of a patient's health status that is elicited directly from the patient and assesses how the patient “feels or functions with respect to his or her health condition.” The advances seen in clinical research regarding PROs has been mirrored in research in cystic fibrosis (CF). A large number of instruments have been used for both therapeutic and nontherapeutic clinical research for many chronic conditions. This review will summarize a history of the development of PROs and how PROs are viewed by the U.S. Food and Drug Administration. We will then review the current state of the art of patient-reported outcomes in CF, specifically addressing the evaluation of different PRO instruments in terms of their reliability and validity. Finally, we will delineate further areas for development of PROs in CF. We believe that the future of CF research will incorporate a more diverse selection of PRO outcome measures; these outcome measures ultimately may be incorporated into clinical care to standardize symptom assessment and provide information regarding the need for specific clinical interventions to improve the quality of care delivered to these patients.

Over the past 20 years, tremendous progress has been made in defining and measuring patient-reported outcomes (PROs), with growing recognition of their importance in health outcomes research (1, 2). A PRO instrument is defined as any measure of a patient's health status that is elicited directly from the patient and assesses how the patient “feels or functions with respect to his or her health condition” (3). This may include observable events, behaviors, or feelings (e.g., ability to walk quickly, lack of appetite, expressions of anger), or unobservable outcomes that are known only to the patient (e.g., perceptions of pain, feelings of depression). PROs have been used for many years as successful clinical efficacy measures for therapeutic trials because they directly assess tangible benefit to the patient.

PROs range from single-item symptom ratings to complex, multidimensional, health-related quality-of-life (HRQOL) measures. Regardless of the structure of the instrument, to use a PRO as a primary or secondary endpoint in a clinical trial it must meet rigorous psychometric criteria, which include a well-defined conceptual framework and strong evidence of reliability and validity (4). Although generic measures of HRQOL, such as the Short Form (SF)-36 and the Quality of Well-Being Scale (5, 6), were initially the most widely used instruments, a substantial body of literature now suggests that disease-specific measures are more sensitive to change and provide information that is more relevant for clinical interventions (7, 8). To date, a majority of PRO measures have been developed for adults, but a recent review of Medline citations from 1990 to 2001 indicated that pediatric measurement studies, which represented only 20% of published studies from 1990 through 1998, now account for more than half of the published literature from 1999 to 2001 (2).

Efforts to develop reliable and valid PROs, such as HRQOL instruments, have been very successful, resulting in the availability of numerous well-validated measures and a growing base of empirical and clinical evidence. Recently, the U.S. Food and Drug Administration (FDA) has formally recognized their importance and clinical utility by releasing a new Guidance for Industry on PROs (3). The cystic fibrosis (CF) community has clearly taken note of this trend and has moved toward improved PROs through an initiative by the Cystic Fibrosis Foundation in the late 1990s to develop an English translation of a CF-specific quality-of-life instrument (the Cystic Fibrosis Questionnaire [CFQ]) (9). Other PROs have been less developed in CF, specifically symptom scores. Given that patient-reported symptoms play a critical role in the clinical assessment of an acute change in pulmonary health status, termed a “pulmonary exacerbation” (PE), one could envision the incorporation of a PRO into the PE definition. Currently the definition of a PE incorporates a constellation of patient symptomology, laboratory data, and physical findings (examples provided in Table 1) to denote a decline in respiratory status usually resulting in a therapeutic intervention.

TABLE 1.

DIAGNOSTIC CRITERIA OF A PULMONARY EXACERBATION^*

Fuchs and colleagues, Pulmozyme:

“Exacerbation of respiratory symptoms”: a patient treated with parenteral antibiotics for any 4 of the following 12 signs or symptoms:

–Change in sputum

–New or increased hemoptysis

–Increased cough

–Increased dyspnea

–Malaise, fatigue, or lethargy

–Temperature above 38°C

–Anorexia or weight loss

–Sinus pain or tenderness

–Change in sinus discharge

–Change in physical examination of the chest

–Decrease in pulmonary function by 10% or more from a previously recorded value

–Radiographic changes indicative of pulmonary infection

Ramsey and colleagues, inhaled tobramycin:

Pulmonary exacerbation indicated by at least two of the following seven symptoms during the study:

–Fever (oral temperature >38°C)

–More frequent coughing (increase of 50%)

–Increased sputum volume (increase of 50%)

–Loss of appetite

–Weight loss of at least 1 kg

–Absence from school or work (at least 3 or preceding 7 days) due to illness

–Symptoms of upper RTI

These symptoms had to have been associated with at least one of the following three additional criteria:

–Decrease in FVC of at least 10%

–An increase in respiratory rate of at least 10 breaths/min

–A peripheral blood neutrophil count of ⩾15,000/mm3

Definition of abbreviation: RTI = respiratory tract infection.

^*From diagnostic criteria used by Fuchs and colleagues (29) and Ramsey and colleagues (59).

Improving survival in CF has been the primary focus of CF researchers, with median survival now reported to be 36.8 years (95% confidence interval, 27.5–31.6) in 2005 in the United States (10); however, of equal importance is to find management interventions and therapies that improve patients' symptoms and function. Advancing the application and interpretation of these tools is critical to our evaluation of potential new therapies to treat this disease, which may ultimately improve long-term outcome and survival. This review will summarize a history of the development of PROs, how PROs are viewed by the FDA, and the current state of the art of patient-reported outcomes in CF, comparing different PRO instruments in terms of their psychometric validation, with special attention to future needs for development. These PRO instruments are divided into three categories: HRQOL measures, respiratory symptom scores (RSS), and a compilation of symptoms and signs connoting a PE. Although there is overlap among these three types of measures (e.g., HRQOL measures may include a respiratory symptom scale) and common methodologies used in the validation process, there are differences in their composition and purpose. HRQOL measures tend to have longer recall intervals (weeks to months) and focus on the impact of symptoms on a patient's functioning and perception of quality of life. In contrast, RSS focus on the symptoms over a shorter time period (days), and PEs are defined as a subset of RSS, focused on acute deterioration of symptoms that leads to a therapeutic intervention.

HISTORY AND BACKGROUND OF PROs

The development of PRO instruments, such as HRQOL measures, has a long history dating back to the World Health Organization's broadened definition of health as “a state of physical, mental, and social well-being and not merely the absence of disease” (11). This statement laid the groundwork for conceptualizing health as multidimensional and served to focus attention away from the biomedical model, which emphasized physiological indices of health, toward a biopsychosocial model that considered the patient's ability to function in his or her daily life (12). There is also a long tradition of using PROs to assess pain—a case in which only the patient can judge the amount and intensity of pain he or she is experiencing (13). More recently, efforts to improve the quality of care for patients, particularly those with chronic illnesses, have led to a more “collaborative” approach to health care and a heightened awareness that the patient's perception of treatment benefit is an important consideration for product development, FDA approval, and adherence (14, 15). Thus, PRO instruments are important clinical efficacy endpoints as described by Mayer-Hamblett and colleagues in this symposium (pp. 370–377).

PRO measures are used for several different purposes: (1) as primary or secondary outcomes in clinical trials; (2) to evaluate new pharmaceutical, behavioral, and surgical interventions or to make comparisons between existing treatments; (3) to describe the effects of an illness on patients' daily functioning; (4) to analyze the costs and benefits of medical interventions; and (5) to aid in clinical decision making (2, 4, 16–18).

SUMMARY OF THE FDA GUIDELINES FOR PROs

As mentioned earlier, the FDA has recently released a Guidance document, developed over the last 10 years, designed to guide investigators and industry on the appropriate development and use of PROs in medical product development. The FDA will now accept adequately developed PRO measures as primary or secondary endpoints in clinical trials if they are appropriate for the disease, product, and indication.

The FDA Guidance document outlines a number of criteria that should be considered in evaluating and validating a PRO instrument for use as a clinical efficacy endpoint for in a clinical trial. These criteria include a conceptual framework underlying the instrument, selection of relevant concepts that should be assessed for the particular disease and product being tested, and establishment of the instrument's reliability, validity, and clinical sensitivity using well-developed psychometric methods. If a PRO instrument is being developed, the Guidance has outlined four major steps: (1) identification of the concepts and domains that are important to patients, which are then included in the conceptual framework; (2) creation of the instrument including the items, recall period, and response scales; (3) evaluation of the psychometric properties of the measure; and (4) modification of the instrument as it is applied in research and practice settings. The FDA encourages investigators and pharmaceutical companies to consult with them early and often, beginning in phase 2 trials, to determine if the PRO is appropriate to measure the purported claims.

INSTRUMENT SELECTION

Given that the process of instrument development is complex, time-consuming, and expensive, the first step in selecting a PRO instrument is to evaluate existing measures. Are there measures available that will meet the criteria for your study or clinical trial? The conceptual framework and psychometric properties of the instrument should be considered, as well as the age groups for which the instrument was developed (19). For example, if your patient population includes children and adolescents, it is important to determine whether a pediatric version of the instrument has been developed. Several sources of information may be accessed in your search, including literature databases (e.g., PubMed, PsychINFO), instrument review resources on the Internet (e.g., the American Thoracic Society Quality of Life Resource, http://www.atsqol.org/; and sites that provide reviews of a variety of instruments, e.g., Health and Psychosocial Instruments). Two instrument review resources specializing in PROs are available online: (1) QOLID (Quality of Life Instrument Database), which is supported by the Mapi Research Institute, contains descriptions of 533 instruments, 370 review copies of original measures, 385 translations, 135 use manuals, and descriptions of 80 databases; and (2) OLGA (Online Guide to Quality-of-Life Assessment; http://www.olga-qol.com/), which is a for-profit organization and provides mini-manuals that describe the content, scoring, and measurement properties for most major quality-of-life measures. Both of these websites are updated regularly with new empirical studies.

CHARACTERISTICS OF A VALID PRO

Development of a new instrument begins with identification of the key concepts that are represented in the conceptual framework. In the case of chronic pulmonary conditions, such as CF and chronic obstructive pulmonary disease (COPD), this will likely include respiratory and physical symptoms (20). Typically, the conceptual framework and specific domains of the PRO are identified through extensive reviews of the literature, previous generic and disease-specific instruments, and focus groups with health care professionals and patients. After generating these concepts, semistructured interviews or additional focus groups are conducted to create potential items.

Next, these items are pilot-tested with patients and caregivers for their relevance and acceptability. Item reduction procedures are then used to identify a list of critical items based on patient/caregiver responses using factor analytic or correlational procedures. The initial psychometric evaluation consists of several tests of reliability, construct validity, evaluation of the measure's sensitivity to change after an intervention, and finally, guidelines for interpretability, which will include the minimal clinically important difference (MCID) (21). Reliability assesses measurement error; reliability of an instrument describes whether the instrument consistently and reproducibly measures what it purports to measure. The primary components of reliability are internal consistency (measured by the intercorrelation of the items using Cronbach's α), test–retest reliability (repeatability of the score over a short time interval), and split-half reliability (evaluation of items in each half of the instrument). Construct validity (or internal and external validity) consists of several components: (1) content or face validity (content matches the construct definition), (2) convergent and divergent validity (measures evaluating similar constructs should be correlated and measures that assess dissimilar constructs should be uncorrelated), (3) discriminant validity (the measure differentiates patients who are expected to differ—by age or disease severity or some other variable), and (4) predictive validity (the instrument should predict changes in health status or other variables that are conceptually related). After establishing reliability and validity, the measure must demonstrate clinical sensitivity (responsiveness to change in the predicted direction) (see Table 2).

TABLE 2.

PSYCHOMETRIC PROPERTIES OF INSTRUMENTS

Measurement Property	Test	What Is Assessed
Reliability	Internal consistency	Whether the items in a domain are intercorrelated, as evidenced by an internal consistency statistic (e.g., coefficient α)
	Test–retest	Stability of scores over time when no change has occurred in the measured concept
	Interrater reliability	Agreement between two or more different raters or interviewers
Construct validity	Content or face validity	Extent to which items and response options are relevant and measure the domain or concept of interest
	Convergent validity	Significant relationship found between PRO scales and related measures
	Divergent validity	PRO instrument differentiates patients who differ by disease severity, age, sex, race, ethnicity, etc.
	Predictive validity	PRO scales predict changes in health status or other relevant variables
Sensitivity	Calculations of effects size and standard error of measurement, among others	Ability of PRO to detect change in predicted direction or no change when patient is stable
Interpretability	MCID or the MID	Smallest difference that can be reliably detected by patients and has clinical meaning. Can be determined using anchor- or distribution-based methods, or a combination of approaches

Definition of abbreviations: MCID = minimal clinically important difference; MID = minimal important difference; PRO = patient-reported outcome.

INTERPRETATION OF PROs

Once an instrument has demonstrated reliability, validity, and clinical sensitivity, the next step is to quantify how much change on the measure is clinically meaningful. This facilitates interpretation of the scores and establishes parameters for estimating the magnitude of change, or effect size, reported in clinical trials. This change has been referred to as the MCID or minimal important difference (MID) score. For example, several standardized 0- to 100-point PRO measures, such as the SF-36 and the PedsQL (Pediatric Quality of Life Inventory), have identified a change of 5 points as the MCID (22, 23).

Several different strategies have been developed to estimate the MCID, falling into two distinct categories: anchor- and distribution-based methods (21, 24, 25). Anchor-based methods, which are preferred by the FDA, ask patients to report the amount of change they have experienced in a particular domain on a visual analog scale (VAS) (from +7 to −7) and to complete the PRO for which the MCID is being calculated. The goal is to identify the smallest difference patients can detect in a particular area of functioning. Subsequently, a change on the VAS that is considered minimal (e.g., 2–3 points) is mapped onto numeric changes on the PRO.

In contrast, distribution-based methods rely on statistical procedures to identify the MCID. One widely accepted statistical approach is to calculate an effect size to determine whether the change in individual or group responses is clinically meaningful. Using this method, the MCID will correspond to a “small effect” (0.20) (26). Two other commonly used statistics to estimate the MCID are 0.5 of the standard deviation of the outcome scale and one standard error of measurement (27). Although there is a great deal of debate about which statistic is best, in practice they often converge around a similar value. Ideally, both anchor- and distribution-based methods should be used in identifying the MCID. The results from the different methods should be considered together in making a final determination of the MCID; this approach is referred to as triangulation (28). It also is important to recognize that patient characteristics, such as age, disease severity, and current health status, can influence the MCID (21).

Finally, and perhaps most importantly, an appropriate endpoint should be selected for the trial. PROs are not appropriate in all cases (e.g., diagnosis of infection requiring antibiotics), but if a PRO is chosen as a primary or secondary endpoint, it must be tightly linked to the expected benefits of the drug or intervention. Consideration of the timing and frequency of administration of the tool is critical and directly impacts the power to detect changes in the PRO, with more frequent administrations increasing power. The recall period of the instrument should also be considered and integrated into this decision, with shorter time intervals demonstrating fewer recall and memory biases; the duration of recall in these instruments varies from no recall period to up to 1 month. Analyses of PRO endpoints and the handling of PRO data should be included in the trial's statistical analysis plan. Standard statistical issues, such as power, handling of missing data, and analytical methodology, should be addressed a priori in the plan for PRO data just as they are for any other endpoint. Collecting data on the PRO in phase 2 will provide important information on the variability and performance of the measure, which should be helpful in powering the phase 3 trials.

STATE OF THE ART OF PRO INSTRUMENTS THAT MEASURE HRQOL IN CF

Arguably the most validated PRO instruments in CF research are those that measure HRQOL. The body of literature regarding HRQOL has increased dramatically in the last 10 years in CF research. Recent PE in patients with CF (using criteria developed by Fuchs and colleagues [29]) has been shown to have a profound negative effect on HRQOL (assessed with the Child Health Questionnaire and the SF-36) (30); a CF PE has also been associated with worse symptoms and health perception and reduced function and vitality (CFQ) (31). Given that PE is an important and often frequent clinical event in the natural history of CF lung disease, HRQOL is clearly an important endpoint. To date, two different disease-specific HRQOL instruments have been developed specifically for CF, the CFQ-revised (CFQ-R) and the Cystic Fibrosis Quality of Life (CFQoL) questionnaire (20, 32). Both of these instruments are considered valid instruments with demonstrated reliability, internal validity, and sensitivity (20, 31, 32). Data regarding the minimal change associated with treatment with inhaled tobramycin are being collected to estimate the MCID for the CFQ-R and will be useful for future therapeutic trials involving antimicrobials and possibly other therapies. These instruments together with a host of non–CF-specific HRQOL measures have been used in CF clinical research.

Generic and disease-specific measures have been used in nontherapeutic trials; the majority of these studies did not use a disease-specific HRQOL measure. A number of studies have focused on severe disease and lung transplantation in patients with CF (33–37). HRQOL instruments have also been used to assess associations between HRQOL and clinical variables, including sex (Child Health Questionnaire [CHQ] and CFQoL) (38–40) and family dynamics (adolescent and parents) (CHQ, adolescent and parent version; CFQ-R) (41, 42). Additional studies have examined the relationships between HRQOL and treatment burden, nutritional status, and the effects of CF PE on sleep (CHQ, CFQoL, CFQ-R) (31, 43, 44). Several studies have used HRQOL instruments to assess the site of treatment for PE (inpatient or outpatient); one used the CHQ–Parent Form (PF-50) and the Medical Outcomes Study SF-36 (45), and the other study used the Chronic Respiratory Questionnaire (CRQ) (46). The effects of exercise programs for patients with CF were evaluated using the Quality of Well-Being Scale (QWB) and the CFQ-R to assess the impact of an in-hospital exercise program in patients with CF (47, 48). Although this list of applications of PROs in CF is not comprehensive, it shows the diversity of both the studies and instruments used in the literature.

Instruments used in the context of clinical therapeutic trials in CF have also been diverse; however, more recently, one instrument has come to dominate studies performed in the United States, Canada, and Australia—the CFQ-R. Studies using the CFQ-R include randomized placebo-controlled studies of hypertonic saline, macrolides, growth hormone, and a novel pancreatic enzyme (49–55). A number of ongoing trials have included the CFQ-R (clinical trials: NCT00112359, NCT00128492, NCT00357279). Other randomized controlled studies not using the CFQ-R include two studies comparing hypertonic saline and dornase α (QWB) (56, 57) and one study evaluating the role of dornase α before or after chest physiotherapy (QWB) (58). Earlier well-known studies in CF did not have a specific HRQOL instrument used as an outcome measure (29, 59). The study assessed data from the intermittent nebulized tobramycin trial (59), and used a nonvalidated global rate-of-change questionnaire, in which patients (or parents) were asked to assess whether their condition was worse, unchanged, or better (60). The study concluded that the use of intermittent nebulized tobramycin was associated with improved global ratings of change as noted by the patients, parents, and physicians.

Clearly, the use of HRQOL instruments has increased significantly in CF clinical research. Their use was systematically reviewed and results were reported in 2005 (61). The authors reported a total of 16 randomized controlled trials in CF using HRQOL instruments; they concluded that the scales used in HRQOL instruments were often not described and interpretation of the results focused on purely statistical significance, ignoring clinical relevance. Thus, further work is needed to standardize the use of HRQOL in CF clinical research. We believe the field of HRQOL instruments has come a long way since this review—as evidenced by the number of ongoing studies and recent publications since this article was published. Some areas of ongoing research include refining our knowledge of the responsiveness of disease-specific HRQOL instruments to various treatments and interventions.

STATE OF THE ART OF INSTRUMENTS THAT MEASURE SYMPTOMS IN CF

Respiratory

To date, no specific patient-derived RSS for patients with CF exists. In the past, several investigators have used nonvalidated symptom scores to assess the impact of intravenous antibiotics or other interventions on CF symptoms. A table of selected symptom instruments and commonly used definitions of a PE used in CF clinical research are shown in Table 3. Signs are those patient phenomena that can be observed by a physician or care provider; symptoms are those patient-reported phenomena that are reported but not necessarily observed by anyone other than the patient. Note that some clinical phenomena can be both a sign and a symptom (i.e., a patient-reported cough that is also observed by the physician).

TABLE 3.

OVERVIEW OF SELECTED INSTRUMENTS THAT ASSESS SIGNS AND SYMPTOMS IN CYSTIC FIBROSIS

	Smith	RSSQ	CFQ-R	SGRQ
Signs and symptoms (new or increased)
Pulmonary signs and symptoms
Increased dypsnea w/exertion		X		X
Decreased exercise tolerance	X	X	X	X
Increased work of breath	X		X
Cough	X	X	X	X
Day cough			X
Night cough			X
Wet or congested cough
Chest congestion		X	X
Frequency of cough		X	X
Cough up mucus			X	X
Wheezing			X	X
Hemoptysis/coughing up blood		X
Sputum volume	X	X	X
Change in sputum appearance	X	X
Change in sputum color			X
Change in sputum consistency
Increased respiratory rate	X
Decreased lung function	X
Upper respiratory tract symptoms
Sore throat/runny nose
Sinus pain/tenderness		X
Change in sinus discharge		X
Constitutional and GI signs and symptoms
Malaise/fatigue/lethargy		X	X
Abdominal pain			X
Decreased appetite	X	X	X
Fever (> 38° C)		X
Weight change	X	X	X
Function
Absenteeism		X	X	X
Sports/lifting/running/walking	X		X	X
Feeling
Tired/mad/grouchy/worried/sad			X	X
Psychosocial			X	X

Definition of abbreviations: CFQ-R = Cystic Fibrosis Questionnaire–Revised; GI = gastrointestinal; RSSQ = Respiratory and Systemic Symptoms Questionnaire; SGRQ = St. George's Respiratory Questionnaire; Smith = Smith and colleagues' acute change score (62).

Smith and colleagues created an acute change score; this score was devised to standardize the physician assessment of patients during hospitalizations for CF PE (62). The score included physician assessment of activity level, cough, appetite, chest exam, weight, and respiratory effort. The score values were arbitrarily set (the scores ranged from 6 to 30), and the authors did not attempt to assess interobserver variability or reproducibility of the score. This scoring system was later used to assess the impact of using a β-lactam alone versus β-lactam plus aminoglycoside to treat a CF PE (63). All patients in the study had improved acute symptom scores during the treatment period regardless of treatment regimen; although the authors found statistically significant differences in the scale, they noted a significant degree of variability in the symptom score. Since the initial development of this system, rigorous methods of scoring and assessment have become more widespread. The primary limitations of this score are its lack of formal validation and its use solely by physicians to assess patients' symptoms. A second symptom score was developed and used in a clinical trial involving a vaccine against respiratory syncytial virus, the Acute Respiratory Illness Checklist. It was used as a symptom score to identify patients with lower respiratory tract infections, thus capturing a wider spectrum of CF exacerbations in study participants. This score was not designed as an acute-change instrument and is not a patient-centered instrument.

Disease-specific quality-of-life instruments often have symptom domains that can be used to derive a symptom subscore. One commonly used respiratory disease–specific HRQOL instrument with a symptom score is the St.George's Respiratory Questionnaire (using the U.S. English translation) (64–67). This instrument was developed for patients with COPD and assesses symptoms in the form of cough, sputum production, wheeze, and breathlessness. It also assesses activity level and the impact on daily life. This instrument has been found to be valid, reliable, and responsive in a population of patients with COPD, asthma, and bronchiectasis. The questionnaire has 76 items and three domains (symptoms, activity, and impact). It has not been used extensively in CF and is quite lengthy. The CRQ is a quality-of-life instrument developed for patients with COPD and has four domains (dyspnea, fatigue, mastery, emotion) (68). One of its drawbacks is that it must be interviewer administered. Unfortunately, low internal consistency was demonstrated for the dyspnea dimension in a study assessing an adapted and shortened version of the CRQ with 45 adult patients with CF (69). Another HRQOL instrument with a symptom scale is the CFQ-R, previously described above (20, 48, 70, 71). In addition to nine quality-of-life domains and an overall health perception scale, the instrument includes three symptom scales, with one scale focusing on respiratory symptoms. The symptom scales, however, are not developed to assess acute change in respiratory symptoms but changes in symptoms over the past 2 weeks. However, the respiratory symptom scale is currently being used to assess response to inhaled antibiotics and other products in development.

A number of other instruments that specifically assess the symptom of dyspnea have been studied in the CF population. These include the Borg scale, the Modified Medical Research Council Scale (MRC), the use of a VAS to assess dyspnea, a 15-count breathlessness score, the Phonation test, and the University of California at San Diego (UCSD) Shortness of Breath Questionnaire (SOBQ) (72–75). These scores have been developed to assess dyspnea after exercise and dyspnea related to activities of daily living. The earliest tool developed was the MRC. This is a 5-point scale that was later applied to patients with chronic bronchitis and emphysema, and more recently CF (76). This instrument has good content and construct validity, good test–retest properties, and discriminates well between disease states and shuttle walk distance (77). It has been used extensively in COPD. In a recent study in CF, the MRC correlated poorly with FEV₁ percent predicted (n = 14) and showed large interindividual variation, but was associated with maximal work achieved on bicycle exercise capacity testing (78). The MRC was used to assess the subjective degree of dyspnea during daily living. In this same study, the Borg scale was linearly related to the maximum voluntary ventilation. The Borg scale is a 0- to 10+-point scale to measure perceived dyspnea on exertion. The instrument has been shown to have acceptable content and criterion validity and test–retest reliability, and discriminates well, but only assesses a limited number of symptoms. The Borg scale has been used in three different studies of patients with CF and is one of the most widely used scales to quantify perceived dyspnea (73, 74, 78).

The VAS is a 10-cm anchor-ended line that was used to quantify pain and shortness of breath and has been used widely in CF (75, 79). The instrument has been shown to have both adequate content and criterion validity, and good discriminant validity, and test–retest and intra-/interrater reliability. The 15-count breathlessness score was developed to aid in the determination of breathlessness in children (73). To perform this test, children take a deep breath and then count out loud to 15; the number of breaths taken to complete the count is the score. Because of concern that children counted at different speeds, this test has been modified to use a digital timer (75). These same authors also devised a phonation test in which children were asked to sustain a maximum phonation of a syllable (“Ahhh”) while the time of phonation was measured in seconds.

The UCSD SOBQ is much more detailed than the previously mentioned measures of dyspnea (74). It involves an assessment of shortness of breath on a 6-point scale during 21 different activities of daily living and three questions related to limitation of activity and fear of the consequences of dyspnea, for a total of 24 questions. Scores are obtained by summing the answers to all of the questions and they range from 0 to 120. The instrument had excellent internal consistency (α = 0.96) and was negatively correlated with physiologic markers of disease severity [FVC, FEV₁, MIP (maximal inspiratory pressure), Dl_CO], exercise tolerance (timed six-minute-walk), and quality of life (Quality of Well-Being scale). Nine patients with CF (17% of the total patients evaluated) were included in the initial revision of this questionnaire. No further reports have been published about the use of this instrument in patients with CF.

A few symptom scales, other than for dyspnea, exist for asthma and COPD. These scales could conceivably be applied to patients with CF; however, these patient populations differ quite substantially in age, symptom profile, and natural history. The Breathlessness, Cough, and Sputum Scale (BCSS) is an example of a symptom score for COPD (80, 81). Designed as a daily diary, the BCSS is a patient-reported outcome measure that asks patients to rate the severity of the three symptoms, each on a 5-point scale; higher scores indicate more severe symptoms. Item scores are summed to yield a total score. The Clinical COPD Questionnaire is another example (82). This instrument was developed to measure symptom and functional state in daily clinical practice. The most recent symptom questionnaire is the Chronic Bronchitis Symptoms Assessment Scale; this symptom questionnaire uses 16 items to capture symptoms in patients with COPD (83). The instrument demonstrated strong internal reliability (overall Cronbach's α of 0.91), as well as statistically significant and moderate to strong associations with spirometry.

It is not known how these instruments developed for either asthma or COPD will perform in the CF population. Patients with CF do have many of the same symptoms as patients with COPD; however, their perception of these symptoms may differ greatly. A CF-specific symptom questionnaire is clearly needed to ensure that one is accurately capturing symptoms. To show responsivity to changes in symptoms, an instrument must accurately reflect those symptoms. Thus, instruments established in one disease (COPD) may not translate at all to CF. Given the lack of a validated RSS for CF, our group (C.H.G.) has been funded by the CF Foundation to create such an instrument, following steps outlined in the FDA Guidance document, and has consulted with FDA personnel, CF clinicians, and clinical research leaders. The development of this instrument started with audiotaping 30 qualitative interviews of children with CF and their parents and adults with CF. The audiotapes were subsequently transcribed and coded; currently a web-based draft instrument is being field tested in 60 children and adults with CF. The goal of the project is to have a simple instrument that is sensitive to acute changes in pulmonary symptoms. The instrument may have applications both in clinical trials and clinical care.

Pain

Pain assessment is a key component of understanding changes in pulmonary status in CF and has been measured using a Likert-type rating scale for pain frequency and the validated Faces Pain Scale for assessing pain intensity (84). The studies reported that a majority of the children and adolescents reported pain, and that pain often responded to nonpharmacologic treatment of pain. In a more recent study by the same group, pain (assessed with a retrospective interview) was associated with decreased HRQOL as measured by several domains on the CFQ-R (85). Four additional studies have evaluated pain in patients with CF (86–89). Two studies focused on pain in the setting of severe and end-stage disease (86, 87), whereas the other focused on musculoskeletal pain (88). One study focused solely on adults (89). There is still no consensus on specific instruments that should be used to assess pain in CF.

Depressive Symptoms

Depressive symptoms are often a component of HRQOL assessments in a chronic respiratory disease, such as CF. Although these symptoms are not directly related to the respiratory system, they may affect the reporting of other symptoms. Depressive symptoms have been assessed primarily in observational studies of various CF populations. Researchers have used the Beck Youth Inventories to study depression, anxiety, anger, disruptive behavior, and self-concept in children with CF (90). In addition, the Hospital Anxiety and Depression Scale, the Mood, Feelings Questionnaire, and the Self-Rating Depression Scale (ZUNG), and the Beck Depression Inventory have been used in a number of clinical settings, including pre– and post–lung transplantation, to evaluate the extent of depression in patients with CF (36, 37, 91–93). None of these studies involved a longitudinal assessment of these symptoms in CF. We (C.H.G.) have an ongoing longitudinal study in patients with CF using the Center for Epidemiologic Studies Depression Scale (CES-D). In this patient population, 23 subjects (30%) had a CES-D score of 16 or greater (a cutpoint suggestive of depression) (94). In recognition of the growing importance of understanding depressive symptoms in CF, the CF Foundation has just funded a national epidemiological study of depressive and anxious symptoms in patients with CF ages 12 through adulthood and parent caregivers of children with CF ages birth to 18 years. Of note, in a recent 48-week study of hypertonic saline in Australia, patients with CF had fewer exacerbations and hospitalizations and thus reported improvements on several scales of the CFQ-R. They, however, reported increased treatment burden on the CFQ-R subscale (51). Treatment burden and depressive symptoms may become increasingly important clinical outcome measures as more complex and time-consuming therapeutic regimens are added to the current complex care regimens facing patients with CF on a daily basis.

ROLE OF PROs IN DEFINING AN ACUTE PE

As noted above, acute PE is an important outcome measure in CF, with an increasing annual frequency as the population ages [Figure 1; U.S. CF Foundation Patient Registry data (103)]. Similar findings were reported by the Epidemiologic Study of Cystic Fibrosis (95). Increased PE rate has also been associated with decreased 2- and 5-year survival rates (96, 97). Despite calls for a consensus diagnosis of PEs by a CF Outcomes Group, which convened in 1992, and in a more recent editorial, no consensus diagnostic criteria exist (98, 99). Commonly used clinical diagnostic criteria can be found in the CF Foundation Clinical Practice Guidelines (100). Definitions derived from empirical data have been used in major clinical trials evaluating new therapies in CF; however they have not been formally validated (Table 1) (29, 59). These definitions have typically combined patient symptoms, laboratory data, and clinician evaluation of the patient. These definitions of a PE have revolved around the clinician's decision to treat a constellation of symptoms. An outcome that is defined by a decision to treat is problematic for several reasons. First, these definitions have not included the patient's perception of his or her symptoms. Second, treatments change over time and therefore will change the sensitivity of an outcome measure (PE). Third, regional differences in treatment thresholds will add significant variability to such an outcome measure.

Figure 1.

Bar graph showing the proportion of patients with cystic fibrosis (CF) by age group in the U.S. CF Registry population in 2004 who experience no exacerbations during the year, one exacerbation during the year, two exacerbations during the year, or three or more exacerbations during the year. Pulmonary exacerbations were treated in the hospital or at home with a course of intravenous antibiotics. Overall, 65% of patients did not have an acute exacerbation in 2004. Reprinted by permission from Reference 103. Copyrighted 2005 by the Cystic Fibrosis Foundation.

Additional work by Rosenfeld and colleagues, employing data from a large phase 3 trial of inhaled tobramycin (59), used a multivariate modeling approach to create an algorithm to identify participants with a PE (101). Using this approach, physician-assessed symptoms (rather than either physical signs or laboratory parameters such as pulmonary function measures) were the most important variables associated with a PE.

A recent instrument called the Respiratory and Systemic Symptoms Questionnaire has been created in an effort to standardize the identification of CF-related PEs, including mild events not necessitating intravenous antibiotics, for clinical trials. Like the definitions developed by Fuchs and colleagues (29) and Ramsey and coworkers (59), this instrument was not developed as a PRO. Despite all of this work, no consensus has been reached regarding the diagnostic criteria for a PE and no patient-reported outcome has been included in this definition. Integrating PROs with objective data in defining an acute PE may enhance the utility of this outcome measure and provide clinical researchers with the ability to identify milder events. Such an approach to defining an acute PE would potentially be more in line with the FDA's position on clinical trial endpoints and outcome measures.

CONCLUSIONS

The use of PROs in clinical trials has increased dramatically over the past decade, and with the publication of the FDA Guidelines on Patient-reported Outcomes, this trend is likely to continue. As mentioned earlier, if adequately developed and validated, PROs can be used as either primary or secondary outcomes in clinical trials to evaluate the benefits of new treatments or compare the efficacy of existing treatments from the patient's perspective. The emphasis on patient reports of health outcomes reflects both the national movement toward quality improvement in health care, with its philosophy of establishing “partnerships” with patients (102), and the application of the science of measurement to the standardization of patients' reports of symptoms and responses to treatment. To move the science of PRO development forward in CF clinical research forward, three major advances will be needed. First, a better understanding of the responsiveness of existing instruments to new medications or interventions is the next critical step. Second, development of new instruments that are sensitive to smaller and earlier, but important, changes in symptoms should be a priority. This increased sensitivity is required as overall health status and lung function improvements are seen in sequential birth cohorts of patients with CF. Third, investigators should explore integrating PROs into a common and universally accepted definition of a PE. These important steps will broaden the scope of outcomes measures that are available to clinical researchers. Clearly, these outcome measures are important for assessing existing therapeutic strategies as well as for new therapies that address the symptoms caused by CF. However, ultimately, if an agent is discovered that cures the disease, PROs will need to be very sensitive to the prevention of symptom development rather than to changes in symptom severity.