Pulmonary Outcome Prediction (POP) Tools for Cystic Fibrosis Patients

Summary

Rationale

Loss of lung function in patients with cystic fibrosis (CF) is associated with increased mortality and varies between individuals and over time. Predicting this decline could improve patient management.

Objectives

Develop simple pulmonary outcome prediction (POP) tools to estimate lung function at age 6 in patients aged 2–5 years (POP_2–5) and lung function change over a 4 year period in patients aged 6–17 years (POP_6–17).

Methods

Analyses were conducted using patients from the Epidemiologic Study of CF (ESCF). To be included in any analysis, patients had to have one year of clinical history recorded in ESCF prior to a clinically stable routine Index Clinic Visit (ICV). In addition to this criterion, for the POP_2–5 tool patients had to be between 2 and 5 years old at ICV and have a second clinically stable visit with spirometric measures at age 6. For the POP_6–17 tool, patients had to be between the ages of 6 and 17 years old at an ICV that included spirometric measures and had to have a second clinically stable visit with spirometric measures from 3 to 5 years after ICV. All patients enrolled in ESCF who met these inclusion criteria were studied. POP_2–5 and POP_6–17 populations were further divided into development groups (with ICV before 1/1/1998) and validation groups (with ICV after that date). Development groups were used to model forced expiratory volume in 1 s (FEV₁) percent predicted at age 6 years (for POP_2–5) and annualized FEV₁% predicted change from ICV to the second visit (for POP_6–17) by multivariable linear regression using age, sex, weight-for-age percentile, cough, sputum production, clubbing, crackles, wheeze, sinusitis, number of exacerbations requiring intravenous antibiotics in the past year, elevated liver enzymes, pancreatic enzyme use, and respiratory tract culture status, plus height-for-age percentile (POP_2–5) and index FEV₁ (POP_6–17). Integer-based POP_2–5 and POP_6–17 tools created from selected variables were evaluated by Pearson correlation and then prospectively validated with separate data collected later from ESCF patients with ICV after 1/1/1998.

Main Results

POP_2–5 and POP_6–17 development groups included 2,709 and 6,113 patients and validation groups included 3,458 and 7,086 patients, respectively. Variables retained were weight-for-age percentile, clubbing, crackles, wheeze, number of exacerbations, and Pseudomonas aeruginosa culture status (both tools), daily cough (POP_2–5), and age, sex, and index FEV₁% predicted (POP_6–17). Correlation coefficients for POP_2–5 and POP_6–17 tools prospectively applied to validation groups were +0.32 and +0.37, respectively.

Conclusions

These simple integer-based POP algorithms employ variables available at clinic visits and can be used to predict the probability of different future pulmonary outcomes for individual patients and patient populations.

Introduction

Several algorithms have been described to assist clinicians in assessing the lung disease status of individuals with cystic fibrosis (CF). Algorithms such as Chrispin-Norman,¹ Brasfield,² Wisconsin,³ Northern,⁴ and Age-Based Severity⁵ produce scores based upon chest radiographs, while Shwachman-Kulczycki⁶ and NIH⁷ algorithms employ radiographic, pulmonary function test (PFT), and clinical symptom data. These scoring systems primarily provide insight into lung disease stage, although some were developed to predict near-term mortality.⁷ Today, no single scoring system is routinely used across a broad segment of the CF clinical community, which may be a consequence of increased CF survival. For instance, the practical value of assessing the probability of near-term mortality for all patients by NIH score⁷ decreases correspondingly as the fraction of patients likely to experience near-term mortality approaches zero.

Today, there is a greater emphasis on the use of pulmonary function, and particularly forced expiratory volume in 1 s (FEV₁) and the fraction of FEV₁ compared to a reference population (FEV₁% predicted), than on radiographic scoring tools for assessing CF lung disease stage.⁸ Although Rosanthal⁹ suggested that reliance on FEV₁ as an indicator of health status is questionable, the measure remains an influential driver for treatment of pulmonary exacerbations¹⁰ and is central to the regulatory approval of CF respiratory therapies.¹¹

This attention to pulmonary function may explain why clinical use of respiratory therapies is associated with a patient’s disease status as measured by FEV₁.^12.13 Thus the potential to estimate an individual’s probability of FEV₁ change over the next few years can provide clinical guidance for the use of interventions intended to preserve lung function in patients most at risk, and thus most likely to benefit.

Unfortunately, variability of FEV₁ over short time intervals prohibits accurate estimation of an individual’s current rate of decline, and several years of data collection are required to obtain reasonable estimates of past decline rates.¹⁴ Additionally, past decline rates are not predictive of future decline rates. For example, young patients who have lost little or no FEV₁ are at the greatest risk of decline in the subsequent 4 years of their lives.¹⁵ In CF patients under age 6 years, traditional spirometry is difficult and may be unreliable, although it is clear that lung disease begins early in life.¹⁶ Although a variety of procedures have been described to assess pulmonary function in children under 6, no single method is widely available or uniformly employed by a majority of CF clinicians.¹⁷

A recently published method using data from the Epidemiologic Study of Cystic Fibrosis (ESCF) allows clinicians to estimate the annualized change in FEV₁ over the next 4 years in individual CF patients aged 6–17 years using a multivariate equation that weights the contribution of various risk factors (e.g., weight for age, presence of physical findings, infection status).¹⁵ The method produces an estimate of annualized future change in FEV₁% predicted by adding parameter estimates associated with categories of 11 predictive variables to an overall estimated change rate, with separate parameter estimates and overall rates provided for 3 age groups (6–8 years, 9 – 12 years, and 13 – 17 years). Although this helps to identify CF patients at high risk for lung function decline, the method may not be practical to use in the clinical setting and is not applicable to children less than 6 years old. We therefore have extended this work by developing a simple clinical scoring tool to estimate annualized change in FEV₁% predicted over the near future for CF patients aged 6–17 years that we call POP_6–17 (“pulmonary outcome prediction”). We used a similar approach to develop a second POP tool (POP_2–5) to estimate FEV₁% predicted at age 6 years for CF patients aged 2–5 years.

Methods

Data were obtained from ESCF, an encounter-based, longitudinal, multicenter, prospective observational study of North American CF patients conducted from 1994 to 2005.¹⁸ Written informed consent was obtained according to research policies at participating institutions. Baseline information including birth date and sex were recorded when the patient enrolled in the study. Pulmonary function test (PFT) results, height, weight, clinical signs and symptoms, pulmonary exacerbations requiring intravenous (IV) antibiotic therapy, and respiratory tract culture results were recorded at each patient visit. FEV₁ values were converted to percent predicted (FEV₁ % predicted) using the reference equations of Wang et al.¹⁹ for females through age 15 and males through age 17 and Hankinson et al.²⁰ at older ages. Detailed explanations of how these variables were captured in the ESCF have been provided previously.^15,18

POP scoring algorithms were constructed for patients aged 2–5 years (POP_2–5) and 6–17 years (POP_6–17). To be included in these analyses, patients had to have a year of clinical history recorded in ESCF prior to a clinically stable routine clinic visit (Index Clinic Visit [ICV]). In addition to this criterion, for the POP_2–5 tool, patients had to be between 2 and 5 years old at ICV and have a second clinically stable visit with spirometric measures at age 6. For the POP_6–17 tool, patients had to be between the ages of 6 and 17 years old at an ICV that included spirometric measures and had to have a second clinically stable visit with spirometric measures from 3 to 5 years after ICV. All patients enrolled in ESCF who met these criteria were included in the study. The two populations were further divided into development (with ICV before 1/1/1998) and validation (with ICV after that date) groups. The POP_2–5 outcome was FEV₁% predicted at age 6 (FEV₁6yr) obtained at a time of clinical stability during the calendar year between the patients’ 6th and 7th birthdays. The POP_6–17 outcome was the annualized rate of FEV₁% predicted change (FEV₁change) estimated by subtracting FEV₁% predicted at the ICV from FEV₁% predicted at a stable follow-up PFT (within 4 ± 1 years) and dividing by the elapsed years between measures.

POP_2–5 and POP_6–17 scoring algorithms were constructed from their respective development groups by iterative regression modeling with variables available at the ICV for each development group using SAS version 9.1 (SAS Institute, Inc., Cary, NC). Patient age, sex, weight-for-age percentile, daily cough, daily sputum production, clubbing, crackles, wheeze, sinusitis, number of exacerbations treated with IV antibiotics in the past year, elevated liver enzymes, pancreatic enzyme use, and respiratory tract culture status plus height-for-age percentile (for patients aged 2–5 years) and FEV₁% predicted at the ICV (for patients aged 6–17 years) were initially included as risk variables based on prior studies.^15,21 Parameter and least squares means estimates from initial regressions were used to empirically determine which variables and interactions were to be retained in subsequent iterations and whether combinations of categories or variable levels were indicated. Model reduction proceeded iteratively with sensitivity analyses used to explore alternate parameterizations. Once each model had been trimmed to a smaller number of predictive variables, integer values were assigned to each variable based on parameter estimates. This tentative score was then treated as a single variable and candidate variables were tested to see if they added significant explanatory power, in which case they were added to the score (or the integer value assigned to the variables was changed). Through this trial and error process, integer values for each variable were adjusted until resulting POP scores (obtained by summing all integers assigned to each variable present in a given patient) provided the best fit to the desired relationship between the POP score and the pulmonary outcome it was intended to predict. For the final POP_2–5 algorithm, each score unit was intended to approximate a 2% predicted deviation of FEV₁ from 100% predicted. Therefore, more negative POP_2–5 scores were intended to predict greater negative deviations from 100% predicted (i.e., reduced pulmonary function). For the final POP_6–17 algorithm, each score unit was intended to represent an FEV₁change of approximately 0.5% predicted per year, with a more negative POP_6–17 score intended to predict a greater annualized rate of decline in FEV₁% predicted over the next 4 years.

POP_2–5 and POP_6–17 algorithms were then prospectively validated by assessing associations between POP scores and actual FEV₁6yr (2- to 5-year-olds) or FEV₁change (6- to 17-year-olds) with separate patient data that had not been included in POP tool development (validation groups) using Pearson correlation coefficients and regressions. The purpose of validating POP tools with separate patient data was to reduce possible inflation of Type I error during trial and error assignment of integers to variable categories during tool construction. This is a common approach when developing predictive models when there are many potential predictors. For the POP_2–5 and POP_6–17 validation groups, Pearson correlation coefficients greater than or equal to 0.06 and 0.04, respectively, are statistically significant at P < 0.001 due to the large sample sizes in this study. In addition, c-indices for concordance between each possible pair of unequal POP scores and observed outcomes for validation groups were determined,²² as well as the concordance between POP scores with differences of 2 or more points and observed outcomes. The predictive power of the POP_6–17 scoring algorithm was compared to the previously published multivariate method developed for this patient population.¹⁵ The outcomes FEV₁6yr and FEV₁change and their associated Z-score values were recalculated using the reference equations of Stanojevic and colleagues.²³ Associations of POP scores with recalculated outcomes were assessed using Pearson correlation coefficients and regression.

Results

Development

Overall, 34,488 patients have participated in ESCF. All ESCF patients meeting a priori inclusion criteria (age range, enrollment period, and stable pulmonary function measures at prescribed times) were included in the current analyses. Inclusion criteria were such that a single patient could be included in multiple groups (at different ages). 1,344 patients were included in both the development and validation groups of the POP_2–5 tool (because they had eligible ICV both before and after January 1, 1998) and 4,449 patients were included in both the development and validation groups of the POP_6–17 tool. An additional 1,064 patients were included in both the development group for the POP_2–5 tool and the validation group for the POP_6–17 tool. All other patients (N=5652) were only included in one group. The POP_2–5 development and validation groups contained 2,709 and 3,458 patients aged 2–5 years with mean ages of 3.4 (SD 1.2) and 3.3 (SD 1.2) years, respectively. The POP_6–17 development and validation groups contained 6,113 and 7,086 patients aged 6–17 years with mean ages of 11.1 (SD 3.3) and 11.2 (SD 3.3) years, respectively. Mean FEV₁ at the ICV for POP_6–17 development and validation groups were 84.6% predicted (SD 21.8) and 85.6% predicted (SD 21.3), respectively, with similar FEV₁ distributions (data not shown). Distributions of other predictive variables for all populations are shown in Table 1. The median number of clinic visits included in estimation of IV exacerbations in the year prior to the ICV (Table 1) was 4 for both development groups and the POP_6–17 validation group and 5 for the POP_2–5 validation group, with a range from 1 to 30 visits. Of those patients with a history of respiratory tract culture in the year prior to ICV, a median of 2 cultures (range 1 to 23) were used to compile microbiology variables (Table 1).

Table 1.

Patient Characteristics at the Index Clinic Visit

	2–5 years old				6–17 years old
	Development n = 2709a,b		Validation n = 3458a		Development n = 6113c		Validation n = 7086b,c
Variable	n	%	n	%	n	%	n	%
Female sex	1314	49	1696	49	2890	47	3343	47
Weight-for-age percentile
<5th	305	11	357	10	1070	18	1157	16
5th to <10th	210	8	264	8	603	10	675	10
10th to <25th	556	21	661	19	1332	22	1520	21
25th to <50th	747	28	961	28	1490	24	1819	26
≥50th	891	33	1215	35	1618	26	1915	27
Cough
Occasional	1157	43	1492	43	2468	40	2894	41
Daily	730	27	897	26	2673	44	2953	42
Sputum production
Occasional	558	21	696	20	2016	33	2445	35
Daily	191	7	247	7	1514	25	1610	23
Clubbing	674	25	776	22	3394	56	3773	53
Crackles	156	6	168	5	934	15	953	13
Wheeze	97	4	92	3	297	5	282	4
Sinusitis	156	6	177	5	480	8	560	8
Number of IV exacerbations in past year
0	2087	77	2691	78	4327	71	5129	72
1	440	16	558	16	1141	19	1232	17
2	121	4	136	4	356	6	413	6
3	34	1	46	1	151	2	167	2
≥4	27	1	27	1	138	2	145	2
Elevated liver function tests	40	1	72	2	106	2	166	2
Pancreatic enzymes	2601	96	3306	96	5851	96	6748	95
Microbiology in past year
No culture	870	32	856	25	1667	27	1645	23
P. aeruginosa +	642	24	865	25	2715	44	3216	45
S. aureus +	684	25	1138	33	2021	33	2841	40
H. influenzae +	438	16	619	18	898	15	1013	14
Microbiology at index clinic visit
No culture	1761	65	1988	57	3657	60	4127	58
P. aeruginosa +	247	9	326	9	1291	21	1419	20
S. aureus +	267	10	565	16	1054	17	1402	20
H. influenzae +	155	6	225	7	339	6	387	5

^a- 1,344 patients included in both groups

^b- 1,064 patients included in both groups

^c- 4,449 patients included in both groups

Predictive variables retained in the final POP_2–5 and POP_6–17 scoring algorithms included weight-for-age percentile, clubbing, crackles, wheeze, number of exacerbations treated with IV antibiotics in the past year, and Pseudomonas aeruginosa culture history. In addition, daily cough remained as a variable in the POP_2–5 algorithm, and sex, age, and FEV₁% predicted at the ICV remained as variables in the POP_6–17 algorithm. Predictive variables and their associated integer values were incorporated into card format to allow convenience and rapid calculation of an individual’s POP score during a clinic visit (Fig. 1).

Figure 1. POP_2–5 and POP _6–17 score cards.

Left panel, POP_2–5 score card for patients with CF aged 2–5 years. Right panel, POP_6–17 score card for patients with CF aged 6–17 years. Predictive variables are arrayed vertically within boxes on each card. Possible values for each variable (e.g., present versus absent) are arrayed horizontally and the corresponding integer values are in boxes directly below. Single integers corresponding to the variable status at the index clinic visit are recorded for each variable in the empty boxes on the right. Integers are summed to derive the POP score at the bottom.

Validation

FEV₁6yr values for the POP_2–5 validation group averaged 96.7% predicted (median 98.2, SD 18.9), with 31.9% of FEV₁6yr values below 90% predicted (Fig. 2A). The relationship between POP_2–5 scores and FEV₁6yr for the validation group approximated target values, with a POP_2–5 score of 0 equal to 100.2% predicted FEV₁ (target: 100% predicted) and each POP_2–5 score unit equal to 1.94% predicted FEV₁ (target: 2% predicted). The median POP_2–5 score for the validation group was −1 (mean −1.8, SD 3.1), with 11.8% of patients having a POP_2–5 score of −6 or lower (Fig. 3A). The Pearson correlation coefficient between the POP_2–5 score and FEV₁6yr was 0.32 and the c-index was 0.602. FEV₁change for the POP_6–17 validation group averaged −1.86% predicted/yr (median −1.77, SD 4.25) (Fig. 2B). More than one-third (37.0%) of patients had an annualized rate of FEV₁ decline that exceeded 3% predicted/yr, and a comparable fraction (36.7%) had annualized declines of less than 1% predicted/yr (Fig. 2B). The median POP_6–17 score for the validation group was −4 (mean −4.09, SD 3.15) (Fig. 3B). A POP_6–17 score of 0 represented a 0.19% predicted/yr FEV₁ change (target 0.0% predicted/yr) and each POP_6–17 score unit equaled 0.50% predicted/yr FEV₁ change (target 0.5% predicted/yr). The Pearson correlation coefficient between the POP_6–17 score and FEV₁change was 0.37 and the c-index was 0.631.

Figure 2. Distribution of outcomes for 2- to 5-year-old and 6- to 17-year-old populations.

Panel A: distribution of POP_2–5 validation group patients by their FEV₁% predicted at age 6 years (FEV₁6yr). Panel B: distribution of POP_6–17 validation group patients by their annualized rate of FEV₁% predicted change (FEV₁change).

Figure 3. Distributions of POP scores and median outcomes by POP scores.

Panel A: distribution of POP_2–5 scores for 3,458 patients in the validation group. Panel B: distribution of POP_6–17 scores for 7,086 patients in the validation group. Panel C: median FEV₁% predicted at age 6 years (FEV₁6yr) values calculated using the reference equations of Wang et al.¹⁹ and Hankinson et al.²⁰ (closed circles) and Stanojevic et al.²³ (open circles) for patients in Panel A by POP_2–5 score. Bars indicate first and fourth quartile boundaries. Panel D: median annualized FEV₁% predicted change (FEV₁change) values calculated using the reference equations of Wang et al.¹⁹ and Hankinson et al.²⁰ (closed circles) and Stanojevic et al.²³ (open circles) for patients in Panel B by POP_6–17 score. Bars indicate first and fourth quartile boundaries.

The associations between POP scores and FEV₁% predicted outcomes recalculated using the Stanojevic et al. method ²³ were similar to those observed using the Wang et al. and Hankinson et al. reference equations (Fig. 3C and 3D). The regression of the recalculated FEV₁6yr as a function of the POP_2–5 score had an intercept of 100.8% predicted FEV₁ and a slope of 1.81% predicted/unit. The Pearson correlation coefficient was 0.29. Regression of the recalculated FEV₁slope as a function of the POP_6–17 score had an intercept of −0.23% predicted/yr and a change of 0.36% predicted/yr-unit score. The Pearson correlation coefficient was 0.28. Association between POP scores and pulmonary function endpoints recalculated as Stanojevic et al. FEV₁ Z-scores (FEV₁ Z-score at age 6 for POP_2–5 and annualized change in FEV₁ Z-score for POP_6–17) had Pearson correlation coefficients of 0.29 and 0.24, respectively.

Discussion

Despite consistent incremental improvements in predicted median survival over past decades, CF remains a life-shortening disease in which loss of lung function is the primary driver of mortality.⁸ For better or worse,⁹ clinicians have come to rely heavily on FEV₁ as a measure of disease stage and aggressiveness,²⁴ need for immediate intervention,¹⁰ and therapeutic efficacy.¹¹ However, a high variability in estimated future disease progression based on FEV₁ measurements creates a challenge for the CF clinician. Ideally, patients at high risk for rapid lung function decline in the near future should be candidates for more-intensive intervention, while the management of patients predicted to be more stable should be maintained as is. In this way, maintenance of the population pulmonary function can be achieved at a lower level of health care utilization and treatment burden. Estimating a patient’s actual (current) rate of lung function decline is an inherently retrospective process, leaving no opportunity for preemptive or preventative intervention.¹⁴ When clinicians rely solely on current FEV₁ as a predictor of FEV₁ decline over the next few years, therapeutic intervention tends to be more aggressive after FEV₁ decline.¹³ This management strategy fails to prevent initial decline and likely contributes to the observation that children and adolescents with the best lung function are at highest risk for rapid decline.¹⁵ Previous analyses of the association of risk factors in 3-year-old children with poorer FEV₁ at age 6²¹ and use of risk factors in 6- to 17-year-old children to estimate future rate of FEV₁ decline¹⁵ have been recognized by clinicians but have not proven simple enough to find their way into routine clinical practice. We have therefore developed two simple POP (pulmonary outcome prediction) tools using an approach previously employed in other contexts^25–27 that can be used in the clinic to estimate future lung function in children and adolescents with CF (Fig. 1). The POP_2–5 tool (for CF patients aged 2–5 years) predicts FEV₁% predicted at age 6 (FEV₁6yr), extending earlier analyses showing that certain risk factors (e.g., low weight-for-age percentile) at age 3 are associated with reduced FEV₁6yr.²¹ Possible POP_2–5 scores range from −18 to +2, with more negative scores representing worse predicted outcomes (i.e., lower FEV₁% predicted at age 6). The POP_6–17 tool (for patients aged 6–17 years) predicts annualized change in FEV₁% predicted (FEV₁change) over the next 4 years. Possible POP_6–17 scores range from −16 to +9, with more negative scores representing worse predicted outcomes (i.e., higher annualized rates of FEV₁% predicted decline over the next 4 years). The POP_6–17 tool is an extension of a previous analysis that produced a linear regression equation for estimating future rate of FEV₁ decline based on risk factors.¹⁵

Generation of integer-based predictive tools includes a trial and error assignment of integer values to parameter estimates has the potential to introduce additional Type I error by including variables that are not actually predictive but have been included by chance. For this reason, the POP tools were modeled using patient data from a development group and then prospectively tested on a separate set of patient data (the validation group).

These POP tools are by no means the first integer-based scoring tools to be described for assessing the clinical status of CF patients. Important predecessors include radiographic scoring tools such as the Chrispin-Norman,¹ Brasfield,² Wisconsin,³ Northern,⁴ and Age-Based Severity⁵ scores and mixed radiographic/symptom-based tools such as the Shwachman-Kulczycki⁶ and NIH⁷ scores. These scoring systems have been used largely as disease staging (i.e., comparative) rather than predictive tools, although some have been used to assess risks of near-term mortality.⁷ No single one of these scores is in wide use today, although all have value for staging CF lung disease and following disease progression. Recently, lung disease staging using high-resolution computerized tomography (HRCT) has gained momentum,^28–30 but has yet to be widely incorporated into routine clinical practice.

Our interest in developing these tools was to provide a method for clinicians to estimate the FEV₁ of their patients in the near future, and most importantly, to identify those patients whom epidemiologic experience suggests have an increased probability of an unfavorable future FEV₁ relative to current clinical impression. An optimal CF pulmonary outcome prediction tool should have reasonable precision while retaining enough simplicity to allow routine use in the clinic. To achieve these characteristics, a balance must be struck between the increased precision achieved through inclusion of many possible predictive variables and the reduced complexity achieved by elimination or combination of as many variables as possible. Further, how a variable is defined can also affect tool utility. Specifically, variables that are easily accessible in the clinic should make the tool more practical. For example, we chose “exacerbations treated with IV antibiotics in the past year” rather than defining an exacerbation using a complex algorithm composed of signs and symptoms. An unambiguous record of IV treatments should be readily accessible in a patient’s chart at a clinic visit and there is no need to reconstruct signs and symptoms to determine whether episodes during the past year met clinical criteria for defining an exacerbation. Finally, because the POP tools have been developed and validated with data collected from CF caregivers at over 100 sites, inter-observer variability has been accounted for within the variance of each tool.

Clinicians should not conclude that the variables included in our POP tools are the only ones that might be relevant or useful for predicting FEV₁ in the future. Rather, these combined variables provide reasonable precision to predict relative probabilities of poor FEV₁6yr and FEV₁change outcomes. For instance, it is well-established that CFTR mutation type(s) influence CF disease course, ^31,32 although outcomes are known to vary among patients with identical CFTR genotypes.³³ The variable “use of pancreatic enzymes,” which can be considered a surrogate for the presence of two “severe” CFTR mutant alleles, was included as a candidate variable during modeling. However, it did not survive backward elimination during development of either POP tool, presumably because remaining variables adequately captured risks associated with different CFTR mutations. Interestingly, when we compared our POP_6–17 tool with our more complex regression equation that included eleven potentially predictive variables, including use of pancreatic enzymes,¹⁵ this simpler tool exhibited better precision in predicting FEV₁change for the validation group (Pearson correlation coefficient of 0.37 for the current method versus 0.26 for the previous method). Addition of CF radiographic assessment scores into POP tools might have increased tool precision. However, it would have been problematic to include a radiographic assessment as no single score is widely used by CF physicians today (for instance, less than a quarter of patients enrolled in ESCF had at least one Brasfield chest roentgenogram score² recorded).

For any POP score value, there is a distribution of observed patient outcomes (Fig. 3C and 3D), with some patients having outcomes worse than would be predicted by the POP score and others with outcomes better. The c-indices for the POP_2–5 and POP_6–17 scores were 0.602 and 0.631, respectively. The c-index is the probability of concordance between a POP score and the outcome it predicts.²² A pair of patients with different POP scores is considered concordant if the patient with the lower POP score also experiences a worse outcome. The c-index is the fraction of all possible pairs with unequal POP scores in which the patient with the lower POP score had a worse outcome. Each pair of patients with unequal POP scores and equal outcomes is assigned a concordance of 0.5. A c-index of 0.5 would indicate a random predictive value of the POP tool; a c-index of 1.0 would be perfectly concordant. A difference of 1 POP tool point between two patients predicts a difference in outcome of only 2 points of FEV₁ % predicted (2% predicted FEV₁ at age 6 for POP_2–5 and 0.5% predicted per year over 4 years for POP_6–17), a difference that is unlikely to be considered clinically compelling. Concordance for all pairs of patients with POP score differences of at least 5 units (predicting at least a 10 point difference in FEV₁% predicted) was 0.692 for POP_2–5 and 0.731 for POP_6–17. Concordance increased to 0.822 and 0.828, respectively, for POP score differences of 10 or more units. Thus, differences of a few POP points between patients or groups of patients are not particularly meaningful, while larger differences are quite likely to correctly predict differences in FEV₁ outcomes.

Variability in outcomes likely reflects differences in the underlying biological severity of CF lung disease within the population, the extent to which supportive care has been influenced by recognition of risk factors and individual clinician practice patterns, and inter-observer variability in the assessment of clinical signs and symptoms in ESCF. Although a given POP score does not unambiguously predict a patient’s outcome, and individuals with scores that differ by only one or two points may not have markedly different risks (Fig. 4), the scores can be useful to categorize individuals into groups with substantially different risks. For example, a patient with a POP_2–5 score of −7 or lower was more likely than not (52%) to have an FEV₁ below 90% predicted at age 6, as were >75% of patients with POP_2–5 scores of −10 or lower, whereas only one in three patients with POP_2–5 scores of −2 experienced a similar outcome, with lower probabilities for patients with higher POP_2–5 scores (Fig. 4A). Similarly, a patient with a POP_6–17 score of 0 or above had only a 17.5% risk of an annualized FEV₁ decline of 2.75 % predicted/year or greater, whereas those scoring between −1 and −5 had double the risk (36.1%) and those scoring −6 or below had triple the risk (52.1%) (Fig. 4B).

Figure 4. Probabilities of pulmonary outcomes by POP scores.

Panel A: probability that a patient will experience an FEV₁ of ≥90% predicted (left) or <90% predicted (right) at age 6 years as a function of their POP_2–5 score. Panel B: probability that a patient will experience an annualized decline in FEV₁ of <1% predicted/yr (left) or >2.75 % predicted/yr (right) as a function of their POP_6–17 score.

Although there are a number of approaches and commercial devices for the assessment of lung function in CF patients too young for traditional spirometry,¹⁷ no single method has achieved widespread use in the clinics participating in ESCF. For this reason, a single measure of this type was not available in the ESCF and could not be incorporated into the POP_2–5 tool. As they become available, these measures could be assessed for their ability to improve pulmonary outcome prediction for young children with CF. For instance, the multiple-breath inert gas washout (MBW) technique³⁴ has been shown to be more sensitive than spirometry for characterizing early lung disease in children with CF.^16,17,35 MBW at age 6 could be substituted for FEV₁% predicted and/or incorporated as a predictive variable as one of several factors to consider in a future POP_2–5 tool.

There are well-recognized precedents for developing outcome tools to assist in the management of chronic diseases, including a tool to estimate 5-year survivorship in CF.^36,37 However, managing CF lung disease progression with predictive tools presents unique challenges. For example, the status of several predictive variables incorporated into cardiovascular health management tools (e.g., blood pressure, blood cholesterol levels, smoking status) can be improved by behavioral and/or pharmacologic intervention. Further, these interventions have been demonstrated to improve outcomes and have become cornerstones of standard disease management. In the case of CF POP tools, there may be interventions that are widely believed to affect the status of predictive variables such as crackles or cough (e.g., antibiotic therapy), but objective evidence is lacking. Similarly, recent CF therapeutic guidelines recommend some chronic therapies based on the results of clinical trials of relatively short duration, but the impact of their long-term use has not been evaluated. ³⁸ The small size of the CF population and extended course of CF lung disease make prospective characterization of potentially disease-modifying therapies problematic and increase the importance of retrospective analyses of large CF patient registries.^39,40 Although improvements in predicted survival over the past decade have been accompanied by an associated reduction in clinical signs and symptoms, to date there is only suggestive evidence that these changes can be attributed to more aggressive intervention.⁴¹ Despite these caveats, the potential benefit to individual CF patients arising from an ability to predict pulmonary outcomes is clear. Recognizing a high potential for a poor outcome in the near future should motivate clinicians to intensify management.

Recently, Stanojevic et al. published an alternative method for calculating FEV₁% predicted in children and proposed use of FEV₁ Z-scores for assessing pulmonary function loss.²³ This approach has the strength of accounting for the impacts of age, height, and sex on coefficients of variation, but the method is limited to non-Hispanic white populations, and the reference equations are not yet widely used. When the POP tools reported here were assessed for their ability to predict outcomes recalculated using the Stanojevic et al. reference equations and also when expressed as FEV₁ Z-scores, they retained their predictive power.

In conclusion, we have developed two tools—POP_2–5 and POP_6–17—that reduce the complexity associated with estimating the risks of poor future pulmonary outcomes in children with CF aged 2–17 years. Although clinicians may already recognize that certain predictive variables (e.g., crackles, cough, and low weight for age) are cause for concern, the POP tools weight the contribution of each variable for pulmonary outcome. Identifying patients within their clinics who share relatively higher risks or lower risks for poor pulmonary outcomes may help clinicians standardize patient management and could lead to more rigorous intervention for previously unrecognized higher-risk patients. Additionally, the ability to segregate patients into different groups with respect to probability of poor future lung function may prove useful for clinical researchers interested in studying higher-risk populations. Finally, these simple tools may assist clinicians in communicating the relative probability of future lung disease progression and outcomes to patients with CF and their families.