Not so implausible: impact of longitudinal assessment of implausible anthropometric measures on obesity prevalence and weight change in children and adolescents

Abstract

Purpose:

Implausible anthropometric measures are typically identified using population outlier definitions, which conflate implausible and extreme measures. We determined the impact of a longitudinal outlier approach on estimated prevalence of body mass index (BMI) categories and mean change in anthropometric measures in pediatric electronic health record data.

Methods:

We examined 996,131 observations from 147,375 children (10–18 years) in the ADVANCE Clinical Data Research Network, a national network of community health centers. Sex-stratified, mixed effects, linear spline regression modeled weight, height, and BMI as a function of age. Longitudinal outliers were defined as observations with studentized residual>|6|; population outliers were defined by Centers for Disease Control-defined z-score thresholds.

Results:

At least 99.7% of anthropometric measures were not extreme by longitudinal or population definitions (agreement≥0.995). Prevalence of BMI categories after excluding longitudinal or population outliers differed by <0.1%. Among children >85^th percentile at baseline, annual mean changes in anthropometric measures were larger in data that excluded longitudinal (girls: 1.28 inches, 12.45 pounds, 1.55 kg/m²; boys: 2.36, 14.65, 1.07) versus population outliers (girls: 0.53 inches, 8.07 pounds, 0.76 kg/m²; boys: 1.45, 11.25, 1.44).

Conclusions:

Longitudinal outlier methods may reduce underestimation of weight, height, and BMI change in children with elevated baseline values.

INTRODUCTION

Electronic health record (EHR) data is increasingly used for pediatric obesity research, surveillance, and screening [1]. Weight and height measured in clinical settings may contain more error than in research settings, but there is no standard method for identifying implausible measures among pediatric anthropometric EHR data [2].

A common approach for identifying implausible anthropometric measures in U.S. children identifies biologically implausible values relative to the Centers for Disease Control and Prevention (CDC) growth curves [3]. This approach assumes that extreme measures - those that exceed predefined z-score cutoffs based on a reference population - are biologically implausible and erroneous; we refer to these measures as population outliers. This assumption has been challenged in the context of increasing prevalence and severity of pediatric obesity in the U.S. [4]. Evidence based on children 2–4 years of age with high body mass index (BMI) values at two measurement occasions suggests that population outliers include valid values, underestimating obesity prevalence by 1% [5]. Moreover, population outlier definitions do not consider implausible change over time, and may truncate estimates of anthropometric change among the heaviest children as their weights increase beyond “biologically implausible” thresholds. Importantly, children who are racial/ethnic minorities or of low socioeconomic status have a higher burden of obesity relative to more affluent White children [6,7], and are thus particularly vulnerable to misclassification when population outlier methods are applied.

Recent work has proposed longitudinal methods for identifying implausible values in pediatric anthropometric data. In contrast to population outlier methods, longitudinal outlier methods are child-specific; they use repeated measures for each child to inform the validity of any given measure. However, these methods vary with regard to the requisite number of measures per child, the dependence on the validity of first observation, and the extent to which data from the full study population is incorporated. Methods include a jackknife residual method for growth data from birth to 24 months of age [8], conditional growth percentiles for birth to 6.5 years of age [9], and a data cleaning method for EHR data based on deviation from the weighted moving average for each child 1–21 years of age [1]. Welch and colleagues used mixed effects models to identify longitudinal outliers in clinical data on a predominately adult patient population [10], which accommodates the substantial variability in number and spacing of anthropometric measures, as well as in age of first observation found in clinical data.

In this study, we extend Welch’s approach to accommodate the nonlinear growth patterns typically observed in pediatric populations. We develop and test this method in 147,375 low-income children receiving care in community health centers in the 23 states participating in the ADVANCE Clinical Data Research Network. The extensive longitudinal measures and high burden of severe childhood obesity in our study population enabled examination of the extent to which exclusion of population outliers might bias the estimates of weight, height, and BMI change over time in the heaviest children as their weights increase beyond the population outlier cutpoints. Our objectives were to (a) demonstrate a mixed effects model-based method for identifying longitudinal implausible anthropometric measures in pediatric EHR data, (b) compare classification of and level of agreement between population outliers and longitudinal outliers and (c) determine the impact of outlier identification approach on the estimated values for (i) prevalence of BMI categories and (ii) change in anthropometric measures.

METHODS

Study population

We used data from the ADVANCE Clinical Data Research Network, a national network of Federally-Qualified Health Centers serving >4.3 million safety net patients across the U.S. [11]. ADVANCE is a multi-center collaborative led by OCHIN, Inc. (not an acronym) health information network in partnership with Health Choice Network (HCN) and Fenway Health. Together, these systems have outpatient clinical data from >130 community health centers with > 1,100 clinics. ADVANCE includes pediatric and adult patients with ≥1 office visit on or after 1/1/2012 across OCHIN, HCN, and Fenway Health networks, and their historical records, dating back as far as 2005.

This study examines anthropometric data for children 10–18 years of age from OCHIN and HCN networks. Fenway data were excluded because the network does not include pediatric clinics. We focused on middle childhood through adolescence as critical life stages in which obesity often develops; however, the proposed methodology could be adapted for other ages. EHR data were extracted from clinical encounters from 01/01/2012 through 12/31/2015 (354,614 children, 1,270,278 encounters). Initial data screening at the encounter-level excluded (1) 1,455 encounters with height ≤25 inches or ≥100 inches and (2) 123 encounters with weight of ≤5 or ≥1000 pounds. This initial exclusion of biologically impossible values was required to achieve model convergence; selected cutpoints were well beyond the 1st and 99th percentiles of the original distributions for weight (62 and 280 pounds) and height (51.7 and 73.3 inches), respectively. We then excluded 207,036 children with <3 remaining height, weight, and body mass index (BMI) measures. Our final analytic sample included 147,375 children randomly assigned to training or test datasets. 73,599 children were selected for the training dataset; the remaining 73,776 were retained for the test dataset. Mean weight, height, and BMI for included versus excluded children were generally within 0.1 z-scores (Appendix Table A.1.)

Study variables

Dependent variables included time-varying weight, height, and BMI. Weight and height were ascertained from clinical encounter records. BMI was calculated from weight and height measured on the same day. For descriptive analysis, BMI was converted to age- and sex-specific BMI z-scores and percentiles using the CDC 2000 growth curves [12], then classified into underweight (<5th percentile), normal weight (≥5th, <85th), overweight (≥85th, <95th), obesity (≥95th to <20% higher than the 95th percentile), and severe obesity (≥20% higher than the 95th percentile) [13].

Independent variables included time-varying age (continuous) and time-constant sex (male, female) and race/ethnicity (Asian, Native Hawaiian/Pacific Islander, Black/African, American Indian/Alaska Native, White, Multiple Race, Other/Unknown, and Hispanic). Sex, date of birth, and race/ethnicity were recorded by clinical staff and are non-missing for the majority of patients because most community health centers are required to report these data to the US Health Resources and Services Administration.

Statistical analysis

Weight, height, and BMI trajectories were modeled using linear spline mixed effects regression models. We built on previous work proposing similar methods to identify longitudinal outliers in clinical data on a predominately adult patient population, accounting for clustering of multiple observations per patient over time [10]. We extended this model to account for nonlinear growth in youth by incorporating piecewise regression over age [14]. Statistical analysis was conducted in SAS Version 9.4 (SAS Institute, Cary, NC). Each of the following steps were performed first in the training dataset (n=73,599), then repeated in the test dataset (n=73,776).

First, mixed effects regression modeled weight, height, and BMI as a function of age, specifying random intercept and slopes for subjects, and fixed effects for race/ethnicity. Analyses were stratified by sex to accommodate sex-specific growth patterns. Piecewise trajectories were fit by including spline terms for age; the number and placement of knots were determined by inflection points identified in loess plots, stratified by sex and race/ethnicity in order to address potential racial/ethnic differences in growth trajectories. Graphically-identified inflection points within the age range of the study (10–18 years) for weight, height, and BMI were 12.5 years for girls and 15 years for boys. Racial/ethnic groups had different intercepts, but similar inflection points; therefore final models were stratified by sex and included race/ethnicity as a covariate:

$$\begin{gathered} \text{Girls:}{Y}_{it}={\mathrm{\beta }}_0+{\mathrm{\beta }}_1*Age+{\mathrm{\beta }}_{2}Ag{e}_{-}after12.5+{\mathrm{\beta }}_x{X}_i+b_{0i}+b_{1t}Age+b_{2t}Ag{e}_{-}after12.5+{\epsilon }_{it} \\ \text{Boys:}{Y}_{it}={\mathrm{\beta }}_0+{\mathrm{\beta }}_1*Age+{\mathrm{\beta }}_{2}Ag{e}_{-}after15+{\mathrm{\beta }}_x{X}_i+b_{0i}+b_{1t}Age+b_{2t}Ag{e}_{-}after15+{\epsilon }_{it} \end{gathered}$$

Where Y_it denotes weight, height, or BMI for patient i at time t. Continuous Age was centered at ten years, with additional spline variable for years after age 12.5 and 15 for girls (Age_after12.5) and boys (Age_after15), respectively. X_i was race/ethnicity. b_0i enabled patient variability at age 10. b_1t enabled different patient-level growth trajectories from age 10–12.5 for girls and age 10–15 for boys. b_2t enabled different patient-level growth trajectories from age 12.5–18 for girls and age 15–18 for boys. Model coefficients are presented in Appendix Tables A.2-3.

Second, after fitting the sex-stratified longitudinal models, we calculated studentized residuals for each measurement, representing the difference between the measurement at any given encounter relative to the expected value from the child’s specific growth curve, standardized by their estimated standard error. Longitudinal encounter-level outliers were defined as measurements with studentized residuals >|6|. Population outliers were defined based on modified z-scores as described by the CDC: weight z-score <−5 or >8, height z-score <−5 or >4, BMI z-score <−4 or >8, relative to the CDC 2000 growth curves [15].

Third, we assessed agreement between population and longitudinal outliers by computing percent agreement and prevalence-adjusted bias-adjusted kappa (PABAK) statistics. PABAK was selected over Cohen’s Kappa, which can yield paradoxical results when the prevalence of the outcome is not evenly distributed across categories of interest [16].

Fourth, we determined the impact of outlier identification approach on prevalence of BMI categories and change in anthropometric measures. Within two analytic subsamples – one excluding population outliers, the other excluding longitudinal outliers – we calculated prevalence of BMI categories and mean change in weight, height, and BMI from the first to the last observed measurements $\left(\frac{LastMeasuremen{t}_i-FirstMeasuremen{t}_i}{Year{s}_i}\right)$.

RESULTS

Study characteristics were similar in the training and test samples (Table 1). Both were composed of 57% females and a large proportion of Hispanic (43%) and non-Hispanic white (36%) children. Among the first observed encounters for each child, mean age was 13.6 years; mean height, weight, and BMI z-scores were 0.1, 0.7, and 0.8, respectively. Among the last observed encounters for each child, mean age was 15.7 years, mean height z-score was smaller (−0.08), and mean weight and BMI remained elevated (mean z-score 0.7 and 0.7, respectively). Children had a median of 4 height and BMI measures and 5 weight measures, although number of measurements per child varied; 10% had only 3 measures, 10% had ≥10 measures.

Table 1.

Characteristics of children aged 10 to 18 years in the ADVANCE Weight Cohort^a

	Training Dataset	Test Dataset
Total (n)	73,599	73,776
Sex [n (%)]
Female	42,319 (57.5)	42,306 (57.3)
Male	31,280 (42.5)	31,470 (42.7)
Race/Ethnicity [n (%)]
American Indian or Alaskan Native	316 (0.4)	346 (0.5)
Asian	1,882 (2.3)	2,103 (2.9)
Black or African American	10,722 (14.6)	10,770 (14.6)
Multiple Race	939 (1.3)	940 (1.3)
Native Hawaiian or Other Pacific Islander	340 (0.5)	363 (0.5)
White	26,699 (36.3)	26,587 (36.0)
No Information/Unknown	1,154 (1.6)	1,157 (1.6)
Hispanic	31,547 (42.9)	31,510 (42.7)
First observed visit [mean (SD)]
Age (years)	13.58 (2.59)	13.57 (2.58)
Height z-score	0.07 (1.17)	0.07 (1.17)
Weight z-score	0.70 (1.38)	0.70 (1.39)
BMI z-score	0.75 (1.50)	0.76 (1.46)
Last observed visit [mean (SD)]b
Age (years)	15.73 (2.47)	15.72 (2.48)
Time since first visit (years)	2.15 (1.57)	2.15 (1.57)
Height z-score	−0.08 (1.12)	−0.08 (1.12)
Weight z-score	0.69 (1.38)	0.69 (1.39)
BMI z-score	0.74 (1.47)	0.74 (1.38)
Number of measures per child [median (10th, 90th Percentile)]
Height	4 (3, 10)	4 (3, 10)
Weight	5 (3, 13)	5 (3, 13)
BMI	4 (3, 10)	4 (3, 10)

BMI, body mass index; SD, standard deviation; HCN, Health Care Network

^aOCHIN and HCN patients who, between 1/1/2012 and 12/31/2015, had at least 3 height, weight, or BMI measures when they were 10–18 years of age, regardless of weight status

In mixed effects models (Appendix Table A.2-3), weight, height, and BMI increased at faster rate prior to the first inflection point (girls: 12.5 years; boys: 15 years), compared to after. Relative to white children, children of color were generally shorter and heavier at baseline, with the exception of Asian children who were shorter and lighter.

Overall, few outliers were identified using either longitudinal and population definitions in the test dataset; ≥99.7% of weight, height, and BMI measurements were not extreme by either definition in both girls and boys (Agreement and PABAK ≥0.995; Table 2). Among population outliers, the proportion also classified as longitudinal outliers varied. For example, most low weight and height population outliers were also classified as longitudinal outliers in girls (weight: 89.3%; height: 67.1%) and boys (weight: 88.2%; height: 74.6%). For low BMI, agreement was lower: few of the population outliers were longitudinally-defined outliers (6.8% in girls, 17.3% in boys). High population outliers were typically also longitudinal outliers in girls (weight: 100%; height: 84.6%; BMI: 83.7%), but less so in boys (weight: 18.4%; height: 50.6%; BMI: 48.8%).

Table 2.

Agreement between CDC population and proposed longitudinal outliers: Test dataset^a

Proposed longitudinal outliersb	CDC population outliersc [n (column %)]			Percent Agreement	Prevalence- Adjusted Bias- Adjusted Kappa
	Low	Not extreme	High
Girls
Weight
Low	67 (89.3)	282 (0.1)	0 (0.0)
Not extreme	8 (10.7)	300,463 (99.8)	0 (0.0)	0.9983	0.9974
High	0 (0.0)	235 (0.1)	33 (100)
Height
Low	155 (67.1)	274 (0.1)	0 (0.0)
Not extreme	76 (32.9)	189,129 (99.8)	10 (15.4)	0.9971	0.9956
High	0 (0.0)	199 (0.1)	55 (84.6)
BMI
Low	6 (6.8)	123 (0.1)	0 (0.0)
Not extreme	82 (93.2)	188,305 (99.9)	16 (16.3)	0.9987	0.998
High	0 (0.0)	27 (0.0)	82 (83.7)
Boys
Weight
Low	45 (88.2)	194 (0.1)	0 (0.0)
Not extreme	6 (11.8)	195,775 (99.8)	84 (81.6)	0.998	0.9969
High	0 (0.0)	117 (0.1)	19 (18.4)
Height
Low	50 (74.6)	268 (0.2)	0 (0.0)
Not extreme	17 (25.4)	140,040 (99.7)	39 (49.4)	0.9969	0.9954
High	0 (0.0)	110 (0.1)	40 (50.6)
BMI
Low	18 (17.3)	56 (0.0)	1 (12.0)
Not extreme	86 (82.7)	139,534 (99.9)	41 (50.0)	0.9986	0.9979
High	0 (0.0)	15 (0.0)	40 (48.8)

BMI, body mass index; CDC, Centers for Disease Control

^aChildren 10–18 years of age in the ADVANCE Weight Cohort (n=73,776)

^bStandardized residual >|6| from sex-stratified, linear spline, mixed effects regression

^cWeight z-score <−5 or >8, height z-score <−5 or >4, BMI z-score <−4 or >8, relative to the CDC 2000 growth curves

Exclusion of population or longitudinal BMI outliers had negligible impact on the prevalence of BMI categories; estimated prevalences were within 0.1 percentage point (Table 3). Prevalence estimates were also similar if height and weight outliers were omitted from calculated BMI values (as opposed to excluding BMI outliers).

Table 3.

BMI Category^a Prevalence (child-level): Test Dataset^b

	Girls		Boys
	CDC Population outlier approach	Proposed Longitudinal outlier approach	CDC Population outlier approach	Proposed Longitudinal outlier approach
Height and weight outliers excluded	n=33,259	n=33,155	n=25,860	n=25,782
Underweight	2.0	1.99	2.76	2.75
Normal BMI	51.6	51.67	52.36	52.43
Overweight	21.32	21.38	16.38	16.39
Obese	15.3	15.29	16.76	16.73
Severe obesity	9.78	9.67	11.73	11.7
BMI outliers excluded	n=33,285	n=33,264	n=25,868	n=25,874
Underweight	2.01	2.03	2.74	2.79
Normal BMI	51.57	51.57	52.36	52.32
Overweight	21.33	21.34	16.38	16.37
Obese	15.31	15.3	16.77	16.76
Severe obesity	9.79	9.77	11.75	11.76

BMI, body mass index index; CDC, Centers for Disease Control

Children classified based on last observed BMI measure, after omission of height and weight or BMI outliers, according to population or longitudinal outlier approach

^aBMI percentiles were classified according to Centers for Disease Control 2000 growth curves: underweight (<5th percentile), normal weight (5th to <85th), overweight (85th to <95th), obese (95th to <20% higher than the 95th percentile), and severe obese (≥20% higher than the 95th percentile).

^bChildren 10–18 years of age in the ADVANCE Weight Cohort (n=73,776). Children were classified based on last observed BMI measure, after omission of height and weight or BMI outliers, according to the population or longitudinal outlier approach.

Annualized mean change in height and BMI was similar upon removing longitudinal outliers, compared to removing population outliers (Table 4). However, among children who were >85^th percentile at baseline, mean changes in height, weight, and BMI were consistently and substantially higher in data screened for longitudinal outliers (girls: 1.24 inches, 12.39 pounds, 1.53 kg/m²; boys: 2.34, 14.08, 1.07), compared to population outliers (girls: 0.61 inches, 8.22 pounds, 0.75 kg/m²; boys: 1.53, 11.61, 0.48).

Table 4.

Annualized change in anthropometry (child level): Test dataset^a

	Girls		Boys
	CDC Population outlier approach	Proposed Longitudinal outlier approach	CDC Population outlier approach	Proposed Longitudinal outlier approach
Height changeb
Full sample	n=33,429	n=33,391	n=26,021	n=25,996
Mean (SD)	0.78 (3.05)	0.73 (6.60)	1.76 (5.00)	1.74 (7.95)
Median (10th, 90th percentile)	0.42 (−0.34,2.58)	0.40 (−0.42,2.64)	1.82 (0.00,3.35)	1.79 (0.00,3.43)
>85th percentile at baseline	n=5,802	n=4,662	n=5,518	n=4,717
Mean (SD)	0.61 (2.93)	1.24 (7.51)	1.53 (2.92)	2.34 (5.45)
Median (10th, 90th percentile)	0.52 (−0.44,2.46)	0.70 (−0.14,3.34)	1.67 (0.00,3.25)	2.36 (0.00,3.95)
Weight changeb
Full sample	n=42,226	n=42,164	n=31,363	n=31,338
Mean (SD)	7.98 (28.20)	8.21 (32.00)	10.87 (22.46)	10.43 (42.57)
Median (10th, 90th percentile)	6.81 (−4.30,21.56)	6.78 (−5.57,23.33)	10.59 (−0.63,23.66)	10.56 (−2.12,24.72)
>85th percentile at baseline	n=17,068	n=17,434	n=13,620	n=13,067
Mean (SD)	8.22 (37.87)	12.39 (38.09)	11.61 (26.18)	14.08 (39.10)
Median (10th, 90th percentile)	7.92 (−7.06,24.76)	10.73 (−5.14,30.23)	12.39 (−4.09,27.01)	15.08 (−2.48,30.09)
BMI changeb
Full sample	n=33,230	n=33,267	n=25,757	n=25,871
Mean (SD)	0.87 (5.64)	0.91 (8.74)	0.66 (5.69)	0.72 (7.94)
Median (10th, 90th percentile)	0.73 (−1.12,3.02)	0.74 (−1.43,3.37)	0.67 (−1.02,2.53)	0.67 (−1.28,2.88)
>85th percentile at baseline	n=15,153	n=15,475	n=12,115	n=11,661
Mean (SD)	0.75 (7.37)	1.53 (8.39)	0.48 (7.89)	1.07 (10.33)
Median (10th, 90th percentile)	0.80 (−1.71,3.41)	1.27 (−1.32,4.56)	0.69 (−1.78,2.83)	1.11 (−1.46,3.71)

BMI, body mass index index; CDC, Centers for Disease Control; SD, standard deviation

^aChildren 10–18 years of age in the ADVANCE Weight Cohort (n=73,776).

^bAnnualized change calculated as (last measure – first measurement)/number of years, after omission of height and weight or BMI outliers, according to the population or longitudinal outlier approach.

The comparison of longitudinal and population outliers with regard to prevalence, agreement, BMI category prevalence, and change over time were similar in the training and test datasets (Appendix Tables A.4-A.6).

DISCUSSION

In this study, we demonstrated a longitudinal method for assessing weight, height, and BMI outliers in a large population of low-income children. Overall, outliers were uncommon and agreement between population and longitudinal outlier definitions was high. The method used to identify outliers had negligible impact on the estimated prevalence of BMI categories. However, within children with elevated height, weight or BMI at baseline (>85^th percentile), exclusion of population outliers resulted in smaller estimates of mean change in weight, height, and BMI over time, compared to exclusion based on longitudinal outliers. This latter finding indicates that population-based outlier exclusion criteria may limit the accuracy of growth trajectory estimation for high-risk children, for whom growth tracking has high clinical relevance.

Despite concerns about error in weight and height in clinical data, <0.3% of pediatric weight, height, and BMI measurements in our analytic sample were classified as outliers by either definition. This finding is consistent with a recent review, in which the prevalence of implausible height, weight, or BMI values ranged from 0.03% to 4.5% using variable, largely cross-sectional definitions in large study populations. Yang and Hutcheon’s longitudinal outlier approach identified 0.3% of weight measures in young children as potential outliers in an intervention trial [9]. Among studies using EHR data, Daymont et al reported that 3.8% of weight and 4.5% of height measures were implausible; or 0.3% of weights and 1% of heights implausible for reasons other than values “carried forward” [1], which were not explicitly identified in our study. In previous work using EHR data, BMI among children 2–17 years was highly reliable over time (ICC 0.97) [17], and childhood obesity prevalence was consistent with NHANES [18]. In short, high quality anthropometric measurements are available from EHR data, even in the lower-resourced setting of safety net providers. This would be anticipated, given the clinical importance of accurate measurements for medication dosing and other clinical decisions.

Correspondingly, the outlier identification method had negligible impact on the estimated prevalence of BMI categories. This result contrasts with findings from Freedman et al., who found that inclusion of valid but extreme BMI measures resulted in a 1 percentage point difference in obesity prevalence in predominately low-income children 0–5 years of age [5]. The small impact of excluding potentially valid but extreme anthropometry measures on overall obesity prevalence in our EHR-based sample of older children is encouraging, particularly for studies that focus on overall obesity prevalence in cross-sectional populations.

Yet pediatric obesity research increasingly focuses on weight and BMI changes over time, temporal growth patterning, or, as severe obesity becomes more common, children in the upper ranges of weight and BMI. Indeed, we found that the estimates of mean change in weight, height, and BMI over time were smaller in analyses that excluded population outliers in children who were >85^th percentile at baseline. These findings suggest that longitudinal outlier approaches are valuable in analyses that focus on weight change among the highest risk children.

Strengths and limitations

This study should be interpreted with several limitations in mind. Similar to most prior studies, classification as a longitudinal or population outlier was not validated against a gold standard. Yet we contributed evidence about agreement between approaches and their impact on estimates of BMI classification prevalence and BMI change. While we did not test alternative thresholds for the longitudinal residual values, we recommend this as an important next step in future research. Third, linear splines for age adequately captured the growth trajectory in our study population from 10–18 years of age while remaining feasible in our large, complex database, but other functional forms should be explored in studies using different age ranges. Fourth, determining the extent to which our findings are generalizable to children of higher socioeconomic status requires replication in middle and higher income study populations. Other study populations may exhibit different patterns of health care utilization and, therefore, timing of measures; however, measurements were consistent with the well-child visit schedule observed in most clinical populations. Balancing these limitations is our feasible and reproducible approach for identifying implausible values in a large, multi-state study population of under-represented and vulnerable low-income children.

Recommendations for identifying potentially invalid weight, height, and BMI values in pediatric EHR data

We offer several recommendations for defining and excluding potentially implausible weight, height, and BMI values in pediatric EHR data. First, population-based definitions are well-defined and easy to implement in any research context. They are the primary option for cross-sectional studies; and likely sufficient for longitudinal studies that examine mean BMI or obesity prevalence at specific points in time and do not focus on children with weight or BMI above the 85^th percentile.

Second, our findings suggest that longitudinal approaches are important for studies that examine changes in weight or BMI over time or among children with elevated weight or BMI. The development of longitudinal approaches is an evolving area, with several proposed approaches [1,8,9]. This paper contributes a regression-based approach for identifying longitudinal anthropometric outliers in exceptionally large sample of older children with highly variable time points found in EHR data. Advancement of longitudinal methods requires more validation against a gold standard, such as expert clinician review as described by Daymont et al [1].

Third, our longitudinal approach can be applied to weight, height, and calculated BMI. We found that excluding extreme weight and heights from BMI calculations produced similar BMI category prevalences as excluding extreme calculated BMI. Yet we recommend outlier assessment of weight and height because it aligns with the underlying data error mechanism. At a minimum, the approach should be explicitly reported to support research reproducibility.

Conclusion

We describe a longitudinal method for identifying weight, height, or BMI outliers that can be easily implemented and reproduced in EHR-based pediatric study populations. In our low-income clinical study population, less than 0.3% of anthropometry measures were potential outliers and prevalence of BMI classifications were robust to outlier identification method. Yet our findings suggest that longitudinal outlier identification methods are needed for unbiased estimation of weight, height, and BMI change in children with elevated values at baseline.

Highlights.

• Improved methods for identifying anthropometric outliers in children are needed.

• Population outlier methods underestimate growth in high-risk children.

• Longitudinal outlier methods reduce underestimation of growth in high-risk children.

List of Abbreviations/Acronyms:

BMI

Body mass index

EHR

Electronic Health Record

PABAK

Prevalence-Adjusted Bias-Adjusted Kappa

Appendix A. Supplemental Tables

Table A1.

Characteristics of children aged 10 to 18 years in the ADVANCE Weight Cohort^a

	Analytic Sample	Excluded
Total (n)	147,375	207,242
Sex [%]
Female	57.42	51.60
Male	42.58	48.40
Race/Ethnicity [%]
American Indian or Alaskan Native	0.45	0.39
Asian	2.70	2.81
Black or African American	14.58	17.88
Multiple Race	1.28	1.10
Native Hawaiian or Other Pacific Islander	0.48	0.59
White	36.16	31.73
No Information/Unknown	1.57	3.30
Hispanic	42.79	42.19
First observed visit [median (10th, 90th Percentile)]
Age (years)	13.39 (10.27,17.25)	14.41 (10.56,18.23)
Height z-score	0.07 (−1.30,1.48)	0.03 (−1.33,1.45)
Weight z-score	0.55 (−0.95,2.46)	0.45 (−1.02,2.26)
BMI z-score	0.60 (−0.82,2.43)	0.47 (−0.93,2.21)
Last observed visit [median (10th, 90th Percentile)]
Age (years)	16.16 (12.04,18.73)	14.86 (11.08,18.47)
Time since first visit (years)	1.85 (0.37,4.37)	0.38 (0.04,1.94)
Height z-score	−0.09 (−1.44,1.31)	−0.01 (−1.37,1.41)
Weight z-score	0.54 (−0.97,2.48)	0.48 (−1.02,2.33)
BMI z-score	0.58 (−0.85,2.45)	0.50 (−0.92,2.28)
Number of measures per child [median (10th, 90th Percentile)]
Height	4 (3, 10)	1 (1, 2)
Weight	5 (3, 13)	1 (1, 2)
BMI	4 (3, 10)	1 (1, 2)

BMI, body mass index; SD, standard deviation; HCN, Health Care Network

^aOCHIN and HCN patients who, between 1/1/2012 and 12/31/2015, were 10–18 years of age

Table A.2.

Mixed model regression results: Test dataset^a

	Girls		Boys
	Full	Outliers Removed	Full	Outliers Removed
β (95% Confidence Interval)
Weight
Intercept	88.32 (87.70, 88.93)	87.97 (87.36, 88.58)	84.04 (83.31, 84.76)	83.94 (83.23, 84.65)
Age1b	14.85 (14.71, 15.00)	14.81 (14.67, 14.95)	14.11 (14.00, 14.22)	14.09 (13.99, 14.19)
Age2c	−9.58 (−9.77, −9.39)	−9.35 (−9.54, −9.17)	−7.84 (−8.07, −7.60)	−7.79 (−8.00, −7.57)
Asian	−18.59 (−20.71, −16.47)	−18.29 (−20.41, −16.17)	−9.89 (−12.39, −7.38)	−9.56 (−12.04, −7.07)
NHPI	8.78 (3.91, 13.65)	8.28 (3.41, 13.14)	15.55 (9.79, 21.3)	15.21 (9.52, 20.89)
Black	7.84 (6.80, 8.88)	7.43 (6.39, 8.47)	0.52 (−0.79, 1.83)	0.86 (−0.44, 2.16)
AI/AN	5.57 (0.81, 10.33)	5.25 (0.48, 10.02)	2.18 (−4.26, 8.62)	2.53 (−3.85, 8.91)
Multiple race	7.05 (4.15, 9.96)	6.55 (3.64, 9.46)	0.68 (−3.41, 4.76)	0.73 (−3.31, 4.76)
Other race	0.58 (−2.29, 3.45)	0.89 (−1.98, 3.77)	1.62 (−1.78, 5.01)	1.57 (−1.79, 4.94)
Hispanic	0.11 (−0.65, 0.87)	−0.01 (−0.78, 0.75)	5.69 (4.77, 6.61)	5.95 (5.03, 6.86)
Height
Intercept	56.39 (56.31, 56.46)	56.54 (56.47, 56.61)	55.44 (55.36, 55.52)	55.50 (55.43, 55.58)
Age1b	2.50 (2.48, 2.52)	2.48 (2.46, 2.5)	2.49 (2.48, 2.500)	2.48 (2.47, 2.49)
Age2c	−2.18 (−2.21, −2.16)	−2.16 (−2.18, −2.14)	−2.02 (−2.04, −2.00)	−2.01 (−2.03, −1.98)
Asian	−2.03 (−2.22, −1.85)	−2.05 (−2.23, −1.87)	−1.68 (−1.91, −1.45)	−1.66 (−1.90, −1.43)
NHPI	−0.75 (−1.16, −0.33)	−0.78 (−1.18, −0.37)	−0.06 (−0.60, 0.48)	0.00 (−0.55, 0.54)
Black	−0.01 (−0.09, 0.08)	0.01 (−0.08, 0.09)	−0.03 (−0.15, 0.09)	−0.01 (−0.13, 0.11)
AI/AN	0.11 (−0.29, 0.51)	0.07 (−0.32, 0.46)	0.18 (−0.43, 0.78)	0.15 (−0.46, 0.76)
Multiple race	0.26 (0.01, 0.50)	0.23 (−0.01, 0.47)	−0.08 (−0.45, 0.29)	−0.10 (−0.47, 0.28)
Other race	−0.19 (−0.44, 0.06)	−0.19 (−0.44, 0.06)	−0.45 (−0.77, −0.13)	−0.44 (−0.76, −0.11)
Hispanic	−1.45 (−1.52, −1.39)	−1.47 (−1.53, −1.40)	−0.92 (−1.00, −0.83)	−0.89 (−0.98, −0.81)
BMI
Intercept	19.71 (19.56, 19.86)	19.68 (19.56, 19.8)	19.86 (19.73, 20.00)	19.85 (19.72, 19.97)
Age1b	1.22 (1.16, 1.27)	1.16 (1.13, 1.18)	0.80 (0.78, 0.82)	0.80 (0.79, 0.82)
Age2c	−0.60 (−0.67, −0.53)	−0.47 (−0.51, −0.43)	−0.27 (−0.32, −0.21)	−0.22 (−0.26, −0.19)
Asian	−2.16 (−2.59, −1.72)	−2.08 (−2.48, −1.68)	−0.88 (−1.32, −0.44)	−0.95 (−1.38, −0.52)
NHPI	1.50 (0.52, 2.48)	1.33 (0.43, 2.23)	2.81 (1.80, 3.82)	2.83 (1.86, 3.81)
Black	1.49 (1.29, 1.7)	1.26 (1.07, 1.45)	−0.09 (−0.31, 0.13)	−0.1 (−0.32, 0.11)
AI/AN	0.99 (0.04, 1.94)	1.01 (0.13, 1.89)	0.10 (−1.01, 1.21)	0.12 (−0.97, 1.22)
Multiple race	1.22 (0.64, 1.81)	1.27 (0.73, 1.81)	0.32 (−0.37, 1.01)	0.24 (−0.44, 0.91)
Other race	−0.08 (−0.68, 0.52)	0.07 (−0.48, 0.62)	0.29 (−0.32, 0.90)	0.31 (−0.28, 0.90)
Hispanic	0.90 (0.74, 1.05)	0.82 (0.68, 0.97)	1.48 (1.32, 1.64)	1.43 (1.28, 1.58)

AI/AN, American Indian/Alaskan Native; BMI, body mass index; NHPI, Native Hawaiian or Pacific Islander

^aChildren 10–18 years of age in the ADVANCE Weight Cohort (n=73,776)

^bAge1 corresponds to age in years from age 10 to 12.5 for girls and from age 10 to 15 in boys.

^cAge2 corresponds to age in years from age 12.5 to 18 for girls and from age 15 to 18 in boys.

Table A.3.

Mixed model regression results: Training dataset^a

	Girls		Boys
	Full	Outliers Removed	Full	Outliers Removed
β (95% Confidence Interval)
Weight
Intercept	88.04 (87.43, 88.65)	87.85 (87.24, 88.46)	84.34 (83.62, 85.07)	83.96 (83.26, 84.67)
Age1b	14.73 (14.59, 14.87)	14.62 (14.49, 14.76)	14.03 (13.92, 14.14)	14.03 (13.93, 14.13)
Age2c	−9.33 (−9.51, −9.14)	−9.03 (−9.21, −8.84)	−7.58 (−7.81, −7.35)	−7.54 (−7.75, −7.32)
Asian	−15.93 (−18.17, −13.68)	−15.85 (−18.11, −13.58)	−12.26 (−14.86, −9.67)	−11.66 (−14.21, −9.1)
NHPI	10.16 (5.15, 15.17)	10.27 (5.24, 15.3)	11.88 (5.66, 18.1)	11.5 (5.38, 17.62)
Black	7.99 (6.95, 9.03)	7.64 (6.6, 8.68)	0.49 (−0.83, 1.81)	0.74 (−0.56, 2.04)
AI/AN	7.62 (2.74, 12.5)	7.42 (2.51, 12.33)	8.54 (1.28, 15.8)	8.66 (1.53, 15.8)
Multiple race	7.58 (4.65, 10.51)	7.37 (4.43, 10.32)	3.08 (−0.88, 7.04)	2.54 (−1.36, 6.44)
Other race	−1.78 (−4.63, 1.07)	−1.87 (−4.74, 1)	0.42 (−2.95, 3.79)	−0.56 (−3.89, 2.76)
Hispanic	0.02 (−0.74, 0.78)	−0.17 (−0.93, 0.6)	5.36 (4.44, 6.29)	5.94 (5.03, 6.85)
Height
Intercept	56.42 (56.35, 56.49)	56.54 (56.47, 56.61)	55.47 (55.4, 55.55)	55.55 (55.47, 55.63)
Age1b	2.5 (2.48, 2.53)	2.49 (2.47, 2.52)	2.49 (2.48, 2.5)	2.48 (2.47, 2.49)
Age2c	−2.19 (−2.21, −2.16)	−2.17 (−2.2, −2.15)	−2.02 (−2.05, −2)	−2 (−2.02, −1.98)
Asian	−1.98 (−2.18, −1.79)	−2.03 (−2.21, −1.84)	−1.92 (−2.16, −1.68)	−1.88 (−2.12, −1.64)
NHPI	−0.63 (−1.05, −0.21)	−0.68 (−1.1, −0.27)	−0.61 (−1.19, −0.03)	−0.47 (−1.06, 0.11)
Black	−0.05 (−0.13, 0.04)	0 (−0.09, 0.08)	−0.07 (−0.19, 0.05)	−0.06 (−0.18, 0.06)
AI/AN	0.07 (−0.34, 0.48)	0.06 (−0.35, 0.46)	−0.11 (−0.77, 0.55)	−0.15 (−0.81, 0.52)
Multiple race	0.04 (−0.21, 0.29)	0.02 (−0.23, 0.26)	−0.12 (−0.48, 0.24)	−0.13 (−0.5, 0.23)
Other race	−0.29 (−0.55, −0.04)	−0.34 (−0.59, −0.09)	−0.31 (−0.63, 0.01)	−0.28 (−0.6, 0.05)
Hispanic	−1.51 (−1.57, −1.44)	−1.52 (−1.58, −1.46)	−0.99 (−1.07, −0.9)	−0.98 (−1.06, −0.89)
BMI
Intercept	19.53 (19.38, 19.67)	19.56 (19.45, 19.68)	19.9 (19.77, 20.03)	19.88 (19.75, 20)
Age1b	1.23 (1.18, 1.28)	1.16 (1.13, 1.19)	0.77 (0.75, 0.8)	0.78 (0.76, 0.79)
Age2c	−0.59 (−0.65, −0.53)	−0.46 (−0.5, −0.42)	−0.15 (−0.21, −0.1)	−0.16 (−0.19, −0.12)
Asian	−1.78 (−2.24, −1.33)	−1.74 (−2.15, −1.32)	−1.15 (−1.59, −0.7)	−1.19 (−1.63, −0.75)
NHPI	2.13 (1.15, 3.12)	2.12 (1.21, 3.02)	2.6 (1.55, 3.65)	2.31 (1.26, 3.36)
Black	1.52 (1.32, 1.73)	1.22 (1.03, 1.4)	−0.06 (−0.28, 0.16)	−0.08 (−0.3, 0.13)
AI/AN	1.14 (0.16, 2.12)	1.2 (0.3, 2.1)	1.23 (0.01, 2.45)	1.31 (0.1, 2.51)
Multiple race	1.37 (0.78, 1.96)	1.29 (0.74, 1.83)	0.77 (0.11, 1.43)	0.81 (0.16, 1.47)
Other race	−0.18 (−0.78, 0.42)	−0.16 (−0.71, 0.4)	0.24 (−0.35, 0.82)	0.3 (−0.28, 0.88)
Hispanic	0.96 (0.81, 1.12)	0.88 (0.74, 1.02)	1.45 (1.3, 1.6)	1.43 (1.28, 1.59)

AI/AN, American Indian/Alaskan Native; BMI, body mass index; NHPI, Native Hawaiian or Pacific Islander

^aChildren 10–18 years of age in the ADVANCE Weight Cohort (n=73,599)

^bAge1 corresponds to age in years from age 10 to 12.5 for girls and from age 10 to 15 in boys.

^cAge2 corresponds to age in years from age 12.5 to 18 for girls and from age 15 to 18 in boys.

Table A.4.

Agreement between population and longitudinal outliers: Training dataset^a

Longitudinal outliersb	Population outliersc [n (column %)]			Percent Agreement	Prevalence-Adjusted Bias-Adjusted Kappa
	Low	Not extreme	High
Girls
Weight				0.9982	0.9974
Low	55 (87.3)	313 (0.1)	0 (0.0)
Not extreme	8 (12.7)	301,602 (99.8)	3 (7.3)
High	0 (0.0)	205 (0.1)	38 (92.7)
Height				0.997	0.9955
Low	150 (79.8)	288 (0.2)	0 (0.0)
Not extreme	38 (20.2)	189,824 (99.7)	47 (47.5)
High	0 (0.0)	198 (0.1)	52 (52.5)
BMI				0.9988	0.9983
Low	3 (4.2)	112 (0.1)	0 (0.0)
Not extreme	68 (95.8)	189,166 (99.9)	10 (9.4)
High	0 (0.0)	28 (0.0)	96 (90.6)
Boys
Weight				0.9975	0.9963
Low	34 (56.7)	208 (0.1)	0 (0.0)
Not extreme	26 (43.3)	192,484 (99.8)	96 (86.5)
High	0 (0.0)	151 (0.1)	15 (13.5)
Height				0.9968	0.9951
Low	64 (57.1)	228 (0.2)	0 (0.0)
Not extreme	48 (42.9)	137,612 (99.8)	57 (57.6)
High	0 (0.0)	116 (0.1)	42 (42.4)
BMI				0.9985	0.9977
Low	15 (17.9)	74 (0.1)	0 (0.0)
Not extreme	69 (82.1)	137,251 (99.9)	38 (46.9)
High	0 (0.0)	26 (0.0)	43 (53.1)

BMI, body mass index

^aChildren 10–18 years of age in the ADVANCE Weight Cohort (n=73,599)

^bStudentized residual >|6| from sex-stratified, linear spline, mixed effects regression

^cWeight z-score <−5 or >8, height z-score <−5 or >4, BMI z-score <−4 or >8, relative to the CDC 2000 growth curves

Table A.5.

BMI Category^a Prevalence (child-level): Training Dataset^b

	Girls		Boys
	Population outlier approach	Longitudinal outlier approach	Population outlier approach	Longitudinal outlier approach
Height and weight outliers excluded	n=33,345	n=33,214	n=25,666	n=25,585
Underweight	1.99	1.98	2.91	2.89
Normal BMI	51.82	51.90	52.21	52.26
Overweight	21.39	21.45	16.91	16.93
Obese	15.41	15.38	16.70	16.65
Severe obesity	9.40	9.30	11.28	11.27
BMI outliers excluded	n=33,364	n=33,342	n=25,686	n=25,683
Underweight	1.98	2.00	2.90	2.92
Normal BMI	51.80	51.80	52.19	52.16
Overweight	21.39	21.39	16.90	16.91
Obese	15.41	15.41	16.70	16.68
Severe obesity	9.41	9.40	11.31	11.32

BMI, body mass index

^aBMI percentiles were classified according to Centers for Disease Control 2000 growth curves: underweight (<5th percentile), normal weight (5th to <85th), overweight (85th to <95th), obese (95th to <20% higher than the 95th percentile), and severe obese (≥20% higher than the 95th percentile).

^bChildren 10–18 years of age in the ADVANCE Weight Cohort. Children were classified based on last observed BMI measure, after omission of height and weight or BMI outliers, according to the population or longitudinal outlier approach.

Table A.6.

Annualized change in anthropometry (child level): Training dataset^a

	Girls		Boys
	Population outlier approach	Longitudinal outlier approach	Population outlier approach	Longitudinal outlier approach
Height changeb
Full sample	n=33,492	n=33,439	n=25,798	n=25,769
Mean (SD)	0.78 (3.79)	0.71 (10.07)	1.71 (5.77)	1.65 (5.78)
Median (10th, 90th percentile)	0.43 (−0.34,2.57)	0.41 (−0.42,2.65)	1.80 (0.00,3.34)	1.77 (0.00,3.42)
>85th percentile at baseline	n=5,839	n=4,545	n=5,383	n=4,606
Mean (SD)	0.53 (5.92)	1.28 (6.74)	1.45 (5.66)	2.36 (7.32)
Median (10th,90th percentile)	0.53 (−0.44,2.38)	0.73 (−0.13,3.26)	1.65 (0.00,3.25)	2.39 (0.00,4.04)
Weight changeb
Full sample	n=42,254	n=42,194	n=31,195	n=31,166
Mean (SD)	8.07 (22.78)	8.34 (37.82)	10.71 (21.60)	10.93 (40.46)
Median (10th, 90th percentile)	6.91 (−4.27,21.47)	6.89 (−5.75,23.48)	10.50 (−0.22,23.18)	10.46 (−1.64,24.33)
>85th percentile at baseline	n=16,873	n=17,398	n=13,385	n=12,961
Mean (SD)	8.07 (27.95)	12.45 (44.52)	11.25 (24.09)	14.65 (50.88)
Median (10th,90th percentile)	7.89 (−7.12,24.35)	10.63 (−4.91,30.42)	12.12 (−3.64,26.61)	14.80 (−1.60,29.87)
BMI changeb
Full sample	n=33,310	n=33,345	n=25,558	n=25,682
Mean (SD)	0.89 (4.48)	0.90 (8.27)	0.68 (5.27)	0.63 (15.11)
Median (10th, 90th percentile)	0.72 (−1.06,2.99)	0.72 (−1.41,3.37)	0.66 (−1.01,2.50)	0.66 (−1.26,2.79)
>85th percentile at baseline	n=14,973	n=15,389	n=11,938	n=11,565
Mean (SD)	0.76 (4.97)	1.55 (10.51)	0.44 (6.58)	1.07 (10.25)
Median (10th, 90th percentile)	0.80 (−1.64,3.33)	1.24 (−1.40,4.46)	0.64 (−1.66,2.82)	1.09 (−1.44,3.67)

BMI, body mass index; SD, standard deviation

^aChildren 10–18 years of age in the ADVANCE Weight Cohort.

^bAnnualized change calculated as (last measure – first measurement)/number of years, after omission of height and weight or BMI outliers, according to the population or longitudinal outlier approach.