Classification of malnutrition in cystic fibrosis: implications for evaluating and benchmarking clinical practice performance

Abstract

Background

In 2005, the Cystic Fibrosis Foundation (CFF) revised the nutrition classification guidelines to eliminate the use of percentage of ideal body weight (%IBW) to define “nutritional failure”; the CFF also recommended that children with cystic fibrosis maintain a body mass index percentile (BMIp) ≥ 50th.

Objective

We assessed the effect of the 2005 CFF nutrition classification guidelines on evaluating the performance of nutritional care practices.

Design

Data from 14 702 children reported to the 2002 CFF Patient Registry were analyzed to compare malnutrition rates in 113 cystic fibrosis centers in the United States. Nutritional failure was defined according to the 2002 CFF criteria—ie, height < 5th percentile, %IBW < 90%, or BMIp < 10th. “Below BMI goal” was defined according to the 2005 CFF criterion, ie BMIp < 50th.

Results

Eliminating %IBW resulted in a 6% reduction (from 33% to 27%) in the nutritional failure rate in the United States. The use of BMIp < 50th led to the classification of 57% of children as below the BMI goal. Misclassification of nutritional failure according to %IBW ranged from 1% to 16% among 113 centers and was greater in the centers with a larger proportion of tall patients. After the elimination of %IBW, one-third of centers changed to a different tertile ranking for nutritional failure rates (kappa = 0.50, moderate-to-poor agreement). More than half the centers changed to a different tertile ranking, from nutritional failure to below BMI goal (kappa = 0.22, poor agreement).

Conclusion

Eliminating misclassification by %IBW and implementing the new BMI goal led to profound and unequal changes in malnutrition rates across cystic fibrosis centers.

INTRODUCTION

Nutritional status has a significant effect on pulmonary disease progression and survival in patients with cystic fibrosis (CF) (1, 2) Since 1992, the US Cystic Fibrosis Foundation (CFF) has published clinical practice guidelines to classify nutritional status (3–5). The European Consensus Committee also published similar recommendations (6). All of these guidelines recommend simple anthropometric indicators to classify nutritional status, indicators that are also used in other chronic diseases and in public health settings (7–13). Specifically, height percentile (HTp) < 5th was used to identify short stature, and either percentage of ideal body weight (%IBW) < 90%, weight-for-height percentile (WHp) < 10th for ages 0–2 y, or body mass index percentile (BMIp) < 10th for ages 2–20 y was used to identify underweight (4–6).

The designation of these nutritional indicators as measures of clinical outcomes has had a profound effect on CF clinical practices. They are used by CF health care providers not only to make treatment decisions for individual patients but also to judge the relative effectiveness of the clinical practice in each CFF-accredited CF center (14, 15) This latter use is by means of a single measure of “nutritional failure”—defined by the criterion of HTp < 5th, %IBW < 90%, WHp < 10th (age < 2 y), or BMIp <10th (age ≥ 2 y)—that is used in center-specific reports to compare a center to its national peers. Centers that have the fewest patients with nutritional failure are felt to represent best clinical practice and are often benchmarked by other centers. Many quality-improvement (QI) projects are also designed by using nutritional failure as the outcome to evaluate program performance (14).

However, studies and clinical applications have shown that the single measure—nutritional failure—adopted by the CFF is problematic. The use of %IBW is methodologically flawed because it misclassifies underweight in short and tall children (16–18). In addition, the calculation of %IBW is time-consuming, and there is no readily available clinical resource to track its progress over time, which makes longitudinal monitoring difficult (19). Furthermore, the cutoff of 10th percentile for BMI to indicate underweight was chosen arbitrarily (17).

The CFF recognized the above problems and recommended the discontinuation of the use of %IBW and the application of a new BMI goal (ie, BMIp ≥ 50th), on the basis of its association with better lung function, as shown from an analysis of registry data in 2005 (20). Many CF centers have begun incorporating these new guidelines into routine CF care. However, it is unclear how the new nutrition classification system would change center-specific malnutrition rates; many presume such a change would affect all CF centers equally (ie, relative rankings would remain the same using the 2002 and 2005 guidelines), but that assumption has not been validated by formal analysis. In addition, many centers implemented QI projects to improve nutritional failure rates but then recently transitioned into using the new BMI goal of ≥50th percentile as the outcome measure, without knowing the effect of this transition. In evaluations of program effectiveness, to separate the effect of a change in classification system from that of the implementation of the QI project, it is crucial to understand how the old and new nutrition classification systems differ. Therefore, the present study was conducted to quantify the effect of eliminating %IBW and applying the new BMI goal of ≥50th percentile on evaluating the performance of nutritional care practices at the national and center levels.

MATERIALS AND METHODS

Study design and source of data

The CFF Patient Registry documents the diagnosis and annual follow-up evaluations of CF patients who are seen at accredited CF centers in the United States, as described in detail elsewhere (21). CFF Patient Registry data reported in the year 2002 were used. Of the 23 106 patients documented in the registry, 14 968 were children <20 y old. Of these, 0.1% (18 observations) had missing height or weight, and another 1.7% (248 observations) had height, weight, or BMI values that deviated >4 SD from the reference mean. These height and weight values likely were outliers resulting from measurement or recording errors, and they were excluded. The remaining 14 702 patients were included in the analyses presented in Table 1. Of the analyses comparing CF centers, those with <50 patients (75 centers totaling 1534 patients—ie, 10.4% of 14 702) were excluded (14).

TABLE 1.

Prevalence of malnutrition at the national level in children with cystic fibrosis¹

	Proportion of patients (n = 14 702)	Comments
	%
2002 CFF definition of nutritional failure
HTp < 5th or BMIp < 10th or %IBW < 902	33.0	Malnutrition rate overestimated because of misclassification by %IBW
Possible categories of nutritional failure
Category 1: HTp > 5th, BMIp >10th, %IBW < 90	6.2	Normal (misclassification of underweight by %IBW)
Category 2: HTp > 5th, BMIp < 10th, %IBW > 90	2.7	Underweight (misclassification of normal weight by %IBW)
Category 3: HTp < 5th, BMIp < 10th, %IBW > 90	2.5	Underweight and short stature (misclassification of normal weight by %IBW)
Category 4: HTp < 5th, BMIp > 10th, %IBW < 90	0.0	Short stature (misclassification of underweight by %IBW)
Category 5: HTp > 5th, BMIp < 10th, %IBW < 90	7.6	Underweight
Category 6: HTp < 5th, BMIp > 10th, %IBW > 90	13.2	Short stature
Category 7: HTp < 5th, BMIp < 10th, %IBW < 90	0.9	Underweight and short stature
Corrected classification of nutritional failure3
Overall	26.8	Corrected nutritional failure after elimination of %IBW < 90
HTp > 5th, BMIp < 10th	10.3	Underweight
HTp < 5th, BMIp > 10th	13.2	Short stature
HTp < 5th, BMIp < 10th	3.4	Underweight and short stature
New (2005) definition of below BMI goal
BMIp < 50th	56.8	“Short stature” not identified

¹HTp, height percentile; BMIp, BMI percentile; %IBW, percentage of ideal body weight; CFF, Cystic Fibrosis Foundation.

²Weight-for-height percentile was used instead of BMIp for children <2 y old.

³Corrected classification refers to the elimination of %IBW as an indicator of underweight.

Approval for the conduct of this study was granted by the Health Sciences Institutional Review Board at the University of Wisconsin–Madison.

Determination of the rates of nutritional failure and below BMI goal

Age- and sex-specific HTp, WHp for ages <2 y, and BMIp for ages 2–20 y were calculated in a computerized program in SAS software (SAS Inc, Cary, NC) by using the Centers for Disease Control and Prevention reference values (22; Internet: http://www.cdc.gov/growthcharts). The %IBW was calculated according to the method defined by Moore et al (23) in a computerized program in SAS, described previously (16, 24), by using the Centers for Disease Control and Prevention reference values (22).

For children <2 y old, nutritional failure was defined by HTp <5th, %IBW <90%, or WHp <10th (4), and below BMI goal was defined by WHp <50th (20). For children ages 2–20 y, nutritional failure was defined by HTp <5th, %IBW <90%, or BMIp <10th (4), and below BMI goal was defined by BMIp <50th (20).

Statistical analysis

Paired t test was conducted to test the difference between uncorrected (ie, including %IBW <90% indicator) and corrected (ie, excluding %IBW <90% indicator) rates of nutritional failure. Comparisons of proportions were assessed by chi-square contingency table methods. Agreement between 2 criteria in identifying children with malnutrition was assessed by using the kappa statistic (25). A kappa value <0.4 was considered to be in poor agreement according to Landis and Koch (26). All statistical analyses were performed with the use of SAS software (version 9.1; SAS Institute Inc, Cary, NC).

RESULTS

Effect of eliminating %IBW < 90% on nutritional failure rate at the national level

As shown in Table 1, one-third of children with CF were classified as having nutritional failure according to the CFF definition based on the 2002 guidelines (4). Examination of all 7 possible categories of nutritional failure showed that 6.2% of children were in category 1—ie, HTp > 5th, BMIp > 10th, and %IBW < 90%. These children were classified as having nutritional failure despite having BMIp > 10th. Most of these children were tall (median HTp: 79th; 58% > 75th, 90% > 50th), which explains why their %IBW appeared worse than their BMIp and, hence, misclassified them as underweight (16–18). Therefore, this group of children should not be regarded as having nutritional failure.

Children in category 2 (Table 1) also had discrepant classifications between BMIp and %IBW criteria. These children were underweight according to BMIp < 10th, but their %IBW was > 90%. Again, this discrepancy can be explained by the children’s stature—in these cases, short stature (median HTp: 12th; 87% < 25th)—as detailed previously (16–18). This explanation also applies to children in category 3 (Table 1), all of who had height below the 5th percentile. In this case, %IBW misclassifies them as normal-weight (16–18). However, misclassification by %IBW for patients in categories 2 and 3 does not have an effect on nutritional failure rate, because children in these 2 categories had BMIp < 10th and therefore would be considered as having nutritional failure regardless of their %IBW status.

It should be mentioned that it is virtually impossible for children to be in category 4 (Table 1), because, in persons of very short stature, ideal weight calculated from the %IBW method is severely underestimated, which leads to an inflated %IBW that almost always looks better than BMIp. This theoretical argument is supported by the observation of 0% in category 4.

The elimination of %IBW < 90% as an indicator for underweight led to a 6.2% reduction, from 33.0% to 26.8%, in corrected nutritional failure rate at the national level (Table 1). More important, misclassification of nutritional failure due to %IBW (category 1, Table 1) occurred primarily in children of tall stature. This finding has a remarkable effect on ranking nutritional failure rates among centers, as described below.

Effect of eliminating %IBW indicator on ranking nutritional failure rates among centers

Misclassification by %IBW < 90% ranged from 1.2% to 15.6%, as shown in Table 2. The misclassification rate was significantly (P < 0.001) and positively associated with the proportion of tall patients (ie, > 75th percentile). Specifically, the 10 centers with the highest misclassification rates (all: >10%) had an average of 13.3% patients taller than the 75th percentile, whereas the 10 centers with the lowest misclassification rates (all: < 2%) had an average of 7.3% of tall patients. Eliminating misclassification by %IBW resulted in changes in the numerical rankings on nutritional failure rates for all except 6 centers. Moreover, rankings for one-third of centers changed to a different tertile: 19 centers shifted to a better tertile and 19 centers shifted to a worse tertile (kappa = 0.50, moderate-to-poor agreement). The largest changes in ranking were a decrease of 51 (improved) and an increase of 28 (worsened) (Table 2). The rankings of these 2 centers are shown by the dashed and dotted lines, respectively, that extend from Figure 1A to within Figure 1B. Figure 1A also shows that misclassification by %IBW (gray bars) was not equal among centers. Because the use of %IBW according to the 2002 CFF definition (4) can only misclassify patients as worse, centers with higher misclassification rates changed to better rankings, whereas centers with lower misclassification rates changed to worse rankings, on corrected nutritional failure rates. Rankings for the best 5 centers remained unchanged, whereas rankings for 3 of the worst 5 centers improved slightly, after elimination of %IBW (Figure 1A, B).

TABLE 2.

Nutritional failure rates in children with cystic fibrosis (CF) among 113 CF centers¹

	Value	Minimum	Median	Maximum
Center size, patients < 20 y old (n)	117 ± 632	50	93	385
Nutritional failure rate by the 2002 CFF criterion (%)3	33.1 ± 7.0	11.1	32.4	49.3
Misclassification rate by %IBW < 90 (%)	6.2 ± 2.9	1.2	5.9	15.6
Corrected nutritional failure rate (%)4,5	26.9 ± 6.4	9.3	26.6	42.0
Corrected ranking minus uncorrected ranking	0 ± 15	−51	1	28

¹CFF, Cystic Fibrosis Foundation; HTp, height percentile; BMIp, BMI percentile; %IBW, percentage of ideal body weight.

²x̄ ± SD (all such values).

³Criterion: HTp < 5th, BMIp < 10th, or %IBW < 90. Weight-for-height percentile was used instead of BMIp for children <2 y old.

⁴Correction: %IBW < 90 was eliminated as an indicator of underweight.

⁵Criterion: HTp < 5th or BMIp < 10th. Weight-for-height percentile was used instead of BMIp for children <2 y old.

FIGURE 1.

Comparison of cystic fibrosis centers according to malnutrition rates, ranked from lowest rate (ranked 1st) to highest rate (ranked 113th). HTp, height percentile; BMIp, BMI percentile; %IBW, percentage of ideal body weight. Data from the 113 cystic fibrosis centers that reported ≥50 patients to the 2002 Cystic Fibrosis Foundation Patient Registry are included. A: Centers by the uncorrected nutritional failure rates according to the 2002 guidelines, which included the %IBW criterion: ie, HTp <5^th, BMIp <10^th*, or %IBW <90. B: Eliminates misclassification rates by %IBW and re-orders centers by the corrected nutritional failure rates: ie, HTp <5th or BMIp <10^th*. C: Re-orders centers by the rate of below BMI goal according to the new (2005) guidelines: ie, BMIp < 50^th*. D: Adds HTp <5th as a stature indicator to refine below BMI goal and re-orders centers according to this adjustment: ie, HTP <5th or BMIp <50th*. *Weight-for-height percentile was used instead of BMIp for children <2 y old.

As an example of the systematic misclassification that can occur on the basis of height, Figure 1A, B also illustrates the change in the ranking after the elimination of %IBW for the CF center that had the highest percentage (18.0%, solid line) of tall children (ie, HTp > 75th percentile). Specifically, this center’s ranking changed from 65th to 26th after %IBW was eliminated.

Effect of changing from nutritional failure to below BMI goal as the outcome measure to rank nutritional care performance

At the national level, the use of BMIp < 50th percentile led to the classification of 56.8% of patients at below BMI goal, which more than doubled the corrected nutritional failure rate defined by HTp < 5th or BMIp < 10th (26.8%; Table 1). In addition, the change from using nutritional failure to using below BMI goal led to profound and unequal changes in the rankings when rates among centers are compared. For example, Figure 1B, C shows that only 1 of the 5 best centers on nutritional failure remained within the 5 best centers on below BMI goal. Most strikingly, rankings for 2 of the 5 best centers on nutritional failure declined to within the worst tertile on below BMI goal. Likewise, rankings for 2 of the 5 worst centers improved considerably to be within the middle tertile. The largest change in ranking is shown by the dash-dotted line from Figure 1B to within Figure 1C; this center changed from the worst tertile on nutritional failure to become the sixth best on below BMI goal. Overall, rankings for more than half of the centers changed to a different tertile: 29 centers shifted to a better tertile and 30 centers shifted to a worse tertile (kappa = 0.22, poor agreement). Differences among centers in the percentages of patients between the 10th and the 50th percentiles most likely explain these dramatic changes in rankings. An additional contributing factor to these changes in rankings is that an indicator for short stature is not included in defining below BMI goal (because the CFF Nutrition Classification Subcommittee is still revising the indicator for short stature). If the old HTp < 5^th value is added as an indicator to define below BMI goal, as illustrated in Figure 1D, the changes in rankings observed in the change from nutritional failure (Figure 1B) to below BMI goal (Figure 1C) for the 4 centers marked as hatched bars would be reduced substantially. It should be noted that, overall, 5.1% patients (range: 0 –12.5% among centers) had HTp < 5th but BMI ≥ 50th; these patients would be classified as normal when a stature indicator is not included to define below BMI goal.

DISCUSSION

In this report, we clearly showed the effect of changing the outcome measure used in establishing the clinical practice guidelines for CF nutritional care. At the national level, the elimination of misclassification by %IBW led to a 6% reduction in the prevalence of nutritional failure as defined by the 2002 CFF practice guidelines (4). More important, we showed that the reduction in nutritional failure rate after the elimination of %IBW does not occur uniformly across CF centers, primarily because the %IBW method selectively misclassifies tall patients as being underweight. Consequently, centers having proportionately more tall patients rank worse when %IBW misclassification is not corrected. The unequal reduction in nutritional failure rate led to profound changes in center rankings—ie, up to one-third of CF centers had ranking changes that were large enough to move the center to a different tertile.

Several implications can be drawn from the above findings. First, they show how change in a single indicator in defining the outcome measure can affect estimates of the prevalence of malnutrition; therefore, no single index should be used exclusively to define malnutrition. Second, they show how the use of an inappropriate indicator in defining the outcome measure can be misleading in judging center performance—eg, centers with more tall patients rank worse on nutritional status than centers with fewer tall patients. Third, they show that difficulties arise when centers rely on an inappropriate outcome measure to design and evaluate the success of QI programs. For example, many QI projects that target nutritional care are designed by setting a goal of reducing the proportion of children with nutritional failure, as described by numerous abstracts presented at the North American CF Conference between 2004 and 2007. To achieve this goal, patients classified as having nutritional failure are reviewed and prescribed individualized interventions to improve their height or weight status (or both) to reach HTp >5^th, %IBW >90%, and WHp or BMIp >10th. For tall patients with %IBW < 90% and BMIp > 50th, not only is it frustrating for to them and their families and CF care providers to be focused on improving nutritional status that is already adequate, but also it erroneously results in a QI program that is less likely to succeed.

The reason why %IBW selectively misclassifies tall patients as being underweight is described in detail in our previous reports (16, 17) and confirmed in a more recent study by Philips et al (18). The %IBW method assumes the ideal weight for a given age and stature to be the weight value that corresponds to the same percentile ranking as the child’s height-for-age percentile. This assumption is valid for children with average stature but not for tall or short children, because the amount of weight gain per increment of height gain is not constant as the stature deviates from the population average (18, 27). For example, a 10-y-old boy with an average height and weight (138 cm and 31.6 kg, respectively) would have a %IBW of 99% that agrees with his BMIp of 50th. On the other hand, a tall (147 cm) 10-y-old boy weighing 36.3 kg would have a very low %IBW (86%) but a normal BMIp (50th). Conversely, a short (130 cm) 10-y-old boy weighing 24.2 kg would have a near-normal %IBW (96%) but a very low BMIp (5th). These examples show how the %IBW method overestimates underweight in tall children and underestimates it in short children (16–18), which leads to misclassification of underweight in tall and short children. Although it could be argued that BMIp is not the gold standard to define the desirable weight for a given age and stature in children, it is superior to %IBW because it is directly derived from weight and height data that are representative of the US population (22). Two more recent studies also showed that BMIp was superior to height-for-age and weight-for-age for screening malnutrition in children with CF (28) and cancer (29).

Another important finding from the present study is that centers that had the lowest and highest nutritional failure rates did not maintain their rankings when the new BMIp cutoff of 50th was used to determine the rate of below BMI goal (Figure 1B, C). In other words, the rankings of many CF centers improved or declined simply because of the change in the nutrition classification system. This fact raises 3 serious concerns regarding the validity of using a single outcome measure to identify the “best” centers for benchmarking the performance of clinical practices. First, nutritional failure targets more severe underweight (ie, BMIp < 10th) and includes a criterion for short stature (which may be a result of chronic malnutrition or normal genetic variation), whereas below BMI goal focuses on identifying suboptimal weight (ie, BMIp < 50th) on the basis of its association with lung function (20). These 2 outcome measures are conceptually different, and it is possible that centers where care focuses on patients with severe malnutrition but which are less aggressive in monitoring patients with mild malnutrition rank better on nutritional failure than on below BMI goal.

Second, malnutrition in CF should be evaluated by other manifestations such as abnormal status of fat-soluble vitamins and essential fatty acids. In this context, neither the nutritional failure nor the below BMI goal outcome measure established by anthropometric variables should be used as the sole basis for making treatment decisions and identifying the best centers by which to benchmark nutritional practices. Third, center-specific QI and benchmarking efforts require careful monitoring of the processes of care that lead to improved outcomes in the cohort of patients who are targeted to benefit from these efforts, and ideally, improved outcomes should be defined by using multiple measures rather than a single measure of performance. Further evidence-based research and discussion are required before revised benchmark criteria can be recommended.

Although findings from this study generate concerns regarding the use of the 2002 and the 2005 nutrition classification guidelines for comparing nutritional care performance across CF centers, the CFF has made significant contributions to the quality and delivery of nutritional care for CF patients, contributions that have been vital to improving the health of these patients. The CFF continues to update its Nutrition Practice Guidelines on the basis of scientific evidence and ease of implementation in the clinical setting. The decision to eliminate %IBW was supported by unequivocal evidence. However, we showed that this change is associated with profound implications for evaluating and benchmarking nutritional practices. The decision on the new BMI indicator represents a change in concept, to broaden the screening of malnutrition, and an evidence-based 50th percentile cutoff value is based on its association with lung function (20). This approach represents a significant advance from the development of previous guidelines. Nevertheless, the use and the value of BMI percentiles in children, especially underweight children, are still evolving, and more research is needed to show the effectiveness of improving BMI on improving lung function, which can then be used to refine the BMI criterion in the future.

In conclusion, our findings showed that eliminating misclassification by %IBW and implementing the new BMI goal led to profound and unequal changes in apparent malnutrition rates across CF centers. Therefore, the importance of using appropriate, and multiple, indicators to define outcome measures cannot be overemphasized. The new (2005) nutrition classification system should be used in combination with other clinical measures; this is true whether the purpose is to improve care for individual patients, to reduce variation in CF healthcare practices, or to monitor trends in progress at the population level. Our study provides an excellent example in the field of CF, which is relevant to other pediatric chronic diseases (29), for evaluating health care practices in relation to clinical outcomes.