The Obesity Epidemic and Changes in Self-Report Biases in BMI

Abstract

Objective

To assess time trends in measurement error of BMI and the sensitivity/specificity of classifying weight status in the United States by analyzing the difference in BMI between self-reported and measured height and weight.

Design and Methods

Data from 18,394 respondents aged 20–89 years from the National Health and Nutrition Examination Survey (NHANES) from 1999 through 2008 were analyzed. Multiple linear regression and logistic regression models estimated trends in reporting bias and misclassification of weight status by BMI categories, sex, age, and racial/ethnic groups, adjusting for the sampling design.

Results

We find no evidence that there are time trends in the accuracy of self-report by BMI categories, sex, age, or racial/ethnic groups. The well-known downward bias in self-report has remained stable over the last decade; approximately one in six to seven obese individuals were misclassified as non-obese due to underestimation of BMI.

Conclusion

Increases in obesity rates based on self-reported height and weight are likely to reflect actual weight increases and are not inflated by changes in reporting accuracy.

Self-reported height and weight have been widely used in surveys to estimate body mass index (BMI). However, self-reported height and weight are subject to measurement error comprised of bias and random error (1). Social desirability bias and recall error are well known reasons for measurement error (2–4). Overall, measurement error results in underestimation of BMI that differs by weight status and sociodemographics (3,5,6).

Increased media coverage of obesity in the past decade may have changed people’s awareness of their weight and accuracy of self-report (7). If the relationship between self-report and actual BMI changes, trend estimates or comparisons of surveys at different time points will conflate true changes in BMI with changes in reporting accuracy. In particular, increased accuracy in self-report (e.g., due to more frequent monitoring of weight) would diminish the traditional downward bias in self-reported BMI and could lead to an inflated estimate of actual obesity trends. Two previous studies considered trends in measurement error among Americans, but did not include detailed trend analyses of recent years during which obesity became much more prominent in media and policy (5,8).

This paper investigates whether the relationship between self-report and objective measurement has changed in the United States between 1999 and 2008 by analyzing both bias and random error in the difference between actual BMI and BMI calculated from self-reported height and weight.

DESIGN AND METHODS

We analyzed combined data from the five continuous two-year surveys of National Health and Nutrition Examination Survey (NHANES) from 1999 through 2008, a multistage survey of a representative sample of the non-institutionalized U.S. civilian population. Information on self-reported height and weight was collected during an in-person interview followed by physical examinations shortly afterwards measuring height and weight. The five surveys collected data on a total of 24,693 respondents aged 20–89 years; study details are available elsewhere (9). The current analyses are limited to 18,394 adults aged 20–89 years self-classified as non-Hispanic Whites, non-Hispanic Blacks, or Mexican Americans, excluding 1,078 pregnant women and 3,066 respondents with either “flagged” or missing anthropometric measures. BMI with flag was significantly higher as expected (e.g., for wearing clothes), while self-report and measured BMI did not differ by the missingness status. For sensitivity analysis, we imputed missing values using the MI procedure in STATA, and the qualitative conclusions remained the same.

We considered the difference between the two BMI values (hereafter referred to as “BMI difference”) for each respondent by subtracting “self-reported BMI” (calculated from self-reported height and weight) from “measured BMI” (calculated from measured height and weight). We performed the same analysis separately for height and weight. We examined changes in the mean of the BMI difference (to measure systematic bias) and its variance (to measure accuracy). There were 376 subjects with a BMI difference more than 3 SD-range of the mean. A sensitivity analysis removing those outliers did not affect our conclusions.

Subgroup characteristics of interest included: BMI categories (“non-obese”, defined as a measured BMI <30, and “obese”, defined as a measured BMI≥30), sex, age groups (20–39 years, 40–59 years, and 60–89 years), racial/ethnic groups (non-Hispanic Whites, non-Hispanic Blacks, and Mexican Americans). Trends in BMI difference were examined through simple and multiple linear regressions; time was measured in survey year (standardized to 0 at year 1999–2000). Sensitivity and specificity of obesity based on self-reported BMI was obtained using measured BMI as the standard. Trends in misclassification were examined through logistic regression. All results were considered statistically significant at P<0.05 unless indicated otherwise. We performed all analyses in STATA version 12.0 (StataCorp LP, College Station, TX) and accounted for the sampling design.

RESULTS

Between 1999 and 2008, self-reported BMI consistently underestimated measured BMI by an average of 0.53 units. Subgroup analyses regressing BMI difference and the squared residual on survey year did not indicate significant linear time trends in the bias or the degree of random error for any of the subgroups (Table 1).

Table 1.

Mean and SD of BMI difference by survey year, BMI categories, and socio-demographic characteristics ^a

Characteristic	N	Mean and SD of BMI difference					Test for trends b
		1999–2000	2001–2002	2003–2004	2005–2006	2007–2008	p value
Overall	18 394	0.55	0.49	0.46	0.56	0.57	0.467
		(1.847)	(1.984)	(1.664)	(1.770)	(1.734)	0.132
Sex
Male	9 373	0.31	0.22	0.31	0.42	0.36	0.052
		(1.528)	(2.014)	(1.494)	(1.718)	(1.595)	0.635
Female	9 021	0.80	0.75	0.61	0.69	0.77	0.601
		(2.088)	(1.917)	(1.803)	(1.811)	(1.837)	0.082
Race/ethnicity
Non-Hispanic White	10 000	0.60	0.49	0.46	0.56	0.59	0.683
		(1.590)	(1.844)	(1.515)	(1.629)	(1.570)	0.426
Non-Hispanic Black	4 312	0.44	0.53	0.51	0.56	0.52	0.522
		(2.497)	(2.197)	(2.210)	(2.293)	(2.175)	0.239
Mexican American	4 082	0.25	0.36	0.34	0.51	0.42	0.115
		(2.975)	(2.888)	(2.064)	(2.157)	(2.283)	0.050
Age
20–39	6 232	0.46	0.39	0.43	0.42	0.48	0.745
		(1.813)	(2.331)	(1.703)	(1.821)	(1.801)	0.259
40–59	6 218	0.48	0.43	0.41	0.59	0.53	0.130
		(1.979)	(1.797)	(1.709)	(1.838)	(1.701)	0.261
60–89	5 944	0.85	0.81	0.62	0.74	0.82	0.511
		(1.645)	(1.513)	(1.486)	(1.505)	(1.641)	0.941
BMI categories
BMI < 30	12 110	0.25	0.18	0.15	0.21	0.21	0.688
		(1.475)	(1.685)	(1.248)	(1.301)	(1.287)	0.052
BMI ≥30	6 284	1.26	1.18	1.11	1.23	1.27	0.690
		(2.364)	(2.387)	(2.163)	(2.278)	(2.207)	0.322

^aStandard deviations (SD) are presented in parentheses. Observations are weighted to be nationally representative of the U.S. population.

^bTest results of trends in BMI difference are based on simple liner regression models regressing BMI difference on survey year. Test results of trends in the standard deviation are based on simple linear regression models regressing squared residuals on survey year.

We examined differential trends in the bias by the subgroups through multiple linear regression of BMI difference on survey year, BMI categories, sex, age and racial/ethnic groups, and their two-way interactions with survey year. Survey year or its interactions with each of the subgroup characteristics were not significantly associated with BMI difference, suggesting that there were no meaningful time trends or differential trends by the subgroups. We also assessed quadratic and cubic polynomial models with respect to survey year to test for non-linear trends and none of the coefficients were significant. BMI categories, sex, and age groups were independently associated with BMI difference. Underestimation of BMI was significantly greater among: obese individuals (compared with non-obese individuals) by 0.96 units (SE=0.066; p <0.001), females (compared with males) by 0.42 units (SE=0.048; p <0.001), non-Hispanic Whites (compared with Mexican Americans) by 0.22 units (SE=0.094; p=0.023) and individuals aged 60–89 years (compared with 20–39 years and 40–59 years) by 0.25 units and 0.34 units (SE=0.074; p=0.001 and SE=0.063; p <0.001).

We examined differential trends in the degree of random error through multiple regression of the squared residual (actual BMI difference minus predicted BMI difference from the model) on the same set of independent variables. Again, there were no significant trends, neither in the direct effect of year nor in interactions with any subgroup characteristics. Reporting accuracy varies across the subgroups (but does not change over time). The squared residual was significantly greater among: obese individuals (compared with non-obese individuals) by 3.00 squared-units (SE=0.593; p <0.001), and individuals aged 40–59 years (compared with 60–89 years) by 1.02 squared-units (SE=0.431; p=0.020) and non-Hispanic Black and Mexican Americans (compared with non-Hispanic Whites) by 5.13 squared-units (SE=1.727; p=0.003) and by 2.07 squared-units (SE=0.707; p=0.004), respectively.

We repeated the analysis using height and weight as outcome variables. We did not find changes over time in accuracy, or differential changes by subgroups (Tables 2 and 3).

Table 2.

Mean and SD of height difference (measured - self-reported height in centimeters) by survey year, BMI categories, and socio-demographic characteristics ^a

Characteristic	N	Survey year					Test for trends b
		1999–2000	2001–2002	2003–2004	2005–2006	2007–2008	p value
Overall	18,394	−0.87	−0.77	−0.88	−0.92	−0.96	0.189
		(3.036)	(3.111)	(2.691)	(2.663)	(2.789)	0.091
Sex
Male	9,373	−1.19	−1.05	−1.25	−1.28	−1.31	0.167
		(2.820)	(3.430)	(2.724)	(2.764)	(2.836)	0.430
Female	9,021	−0.56	−0.48	−0.53	−0.57	−0.62	0.451
		(3.205)	(2.728)	(2.610)	(2.510)	(2.700)	0.059
Race/ethnicity
Non-Hispanic White	10,000	−0.90	−0.74	−0.88	−0.93	−0.98	0.235
		(2.670)	(2.873)	(2.445)	(2.453)	(2.517)	0.246
Non-Hispanic Black	4,312	−0.73	−1.01	−0.92	−0.86	−0.83	0.969
		(3.764)	(3.412)	(3.167)	(2.888)	(3.098)	0.049
Mexican American	4,082	−0.75	−0.67	−0.81	−0.97	−0.97	0.189
		(5.041)	(4.690)	(3.955)	(4.017)	(4.196)	0.146
Age
20–39	6,232	−0.56	−0.52	−0.84	−0.75	−0.78	0.041
		(2.896)	(3.689)	(2.779)	(2.805)	(3.059)	0.549
40–59	6,218	−0.61	−0.48	−0.55	−0.67	−0.75	0.120
		(3.212)	(2.472)	(2.500)	(2.418)	(2.452)	0.060
60–89	5,944	−1.91	−1.86	−1.61	−1.73	−1.75	0.249
		(2.767)	(2.803)	(2.748)	(2.719)	(2.766)	0.840
BMI categories
BMI < 30	12,110	−0.77	−0.66	−0.77	−0.83	−0.82	0.318
		(3.028)	(3.291)	(2.614)	(2.616)	(2.681)	0.056
BMI >= 30	6,284	−1.12	−1.01	−1.11	−1.11	−1.24	0.280
		(3.041)	(2.650)	(2.831)	(2.740)	(2.966)	0.963

^aStandard deviations (SD) are presented in parentheses. Observations are weighted to be nationally representative of the U.S. population.

^bTest results of trends in height difference (measured - self-reported height) are based on simple liner regression models regressing height difference on survey year. Test results of trends in the standard deviation are based on simple linear regression models regressing squared residuals on survey year.

Table 3.

Mean and SD of weight difference (measured - self-reported weight in kilograms) by survey year, BMI categories, and socio-demographic characteristics ^a

Characteristic	N	Survey year					Test for trends b
		1999–2000	2001–2002	2003–2004	2005–2006	2007–2008	p value
Overall	18,394	0.76	0.65	0.44	0.70	0.70	0.855
		(4.215)	(4.445)	(4.010)	(4.619)	(4.236)	0.724
Sex
Male	9,373	−0.15	−0.28	−0.28	0.06	−0.15	0.358
		(3.671)	(4.500)	(3.923)	(4.959)	(4.085)	0.134
Female	9,021	1.66	1.57	1.14	1.33	1.52	0.228
		(4.515)	(4.191)	(3.971)	(4.165)	(4.220)	0.529
Race/ethnicity
Non-Hispanic White	10,000	0.83	0.69	0.45	0.71	0.75	0.710
		(3.819)	(4.239)	(3.640)	(4.408)	(3.959)	0.593
Non-Hispanic Black	4,312	0.58	0.44	0.53	0.74	0.65	0.537
		(6.057)	(5.359)	(5.727)	(5.841)	(5.558)	0.692
Mexican American	4,082	0.19	0.53	0.15	0.50	0.28	0.871
		(4.710)	(4.962)	(4.322)	(4.518)	(4.384)	0.309
Age
20–39	6,232	0.82	0.68	0.42	0.49	0.66	0.287
		(4.464)	(4.911)	(4.150)	(4.846)	(4.428)	0.870
40–59	6,218	0.77	0.67	0.57	1.01	0.76	0.427
		(4.160)	(4.415)	(4.249)	(4.914)	(4.361)	0.413
60–89	5,944	0.61	0.52	0.22	0.44	0.63	0.913
		(3.785)	(3.396)	(3.176)	(3.408)	(3.527)	0.584
BMI categories
BMI < 30	12,110	0.08	−0.04	−0.26	−0.13	−0.10	0.135
		(3.153)	(3.226)	(3.041)	(3.343)	(3.179)	0.584
BMI >= 30	6,284	2.32	2.18	1.90	2.27	2.24	0.953
		(5.704)	(6.108)	(5.216)	(6.073)	(5.432)	0.678

^aStandard deviations (SD) are presented in parentheses. Observations are weighted to be nationally representative of the U.S. population.

^bTest results of trends in weight difference (measured - self-reported weight) are based on simple liner regression models regressing weight difference on survey year. Test results of trends in the standard deviation are based on simple linear regression models regressing squared residuals on survey year.

We compared the sensitivity and specificity of obesity (BMI≥30) to examine the net impact of the measurement error on classification of relative weight status. Because of the downward bias in self-reported BMI, the sensitivity (i.e., the percentage of obese individuals correctly identified as obese by self-reported BMI) was consistently lower than the specificity (i.e., the percentage of non-obese individuals correctly identified as non-obese by self-reported BMI) as a whole and among any subgroups (full results not shown but available from the corresponding author upon request). An estimated 15.5% of obese individuals and 2.4% of non-obese individuals were misclassified as non-obese and obese, respectively, resulting in underestimation of obesity prevalence. Logistic regression analyses did not indicate significant time trends in misclassification (correctly vs. incorrectly classified) for any of the subgroups.

We tested for differential trends in misclassification by the subgroups through multiple logistic regression of the misclassification status on the aforementioned set of independent variables. Survey year or its interactions with each of the subgroup characteristics were not statistically significant. The odds of misclassification among obese individuals was 7.2 times (SE=1.15; p<0.001) that among non-obese individuals. Among Mexican Americans, the odds was 2.1 times (SE=0.03; p<0.001) that among non-Hispanic Whites. Other subgroup characteristics were not associated with misclassification.

DISCUSSION

The study examined whether the relationship between self-report and objective measurement of relative weight status changed between 1999 and 2008, a plausible scenario given the increased media coverage of obesity. There is no evidence that people’s responding behaviors have changed. We find no changes over time in accuracy of self-reported BMI and misclassification of weight status nor any differential changes by subgroups. The downward bias in classifying weight status has remained stable over the last decade; approximately one in six to seven obese individuals were misclassified as non-obese due to underestimation of BMI.

There could be a secondary effect on the accuracy of overall trends to the extent that the US population demographics change. The increasing proportion of Hispanics and older people (10) may add to misclassification of weight status, given the larger random error and the greater downward bias in self-report among those groups. If the population were to continue to gain weight (11) and to have the same level of bias, it could lead to underestimated trends when using self-report. Severe obesity has been increasing even more rapidly than moderate obesity (12), so we would expect that if there is any bias in using self-report for trend estimates, it would be in the direction of underestimating increases. Nevertheless, the lack of time trends in self-report BMI bias among obese individuals, despite the increase in severe obesity and expected increase in reporting bias, may suggest that differentials in self-report biases among obese individuals have diminished over the period. A study reports improved accuracy of self-report BMI among obese individuals between 1988–1994 and 2005–2008, using NHANES data, and relates it to increased acceptance of obesity status (8).

The study findings may be considered as a lower bound of bias and random error associated with self-reported anthropometric measures. In NHANES, self-report may be more accurate because the survey was conducted in-person and it was followed by an examination (3,13,14). Self-report biases may be larger in telephone surveys (e.g., the Behavioral Risk Factor Surveillance System), or in-person interviews without physical examinations (e.g., the National Health Interview Survey) (3,13,15).