Implications of Missing Income Data

We strongly support the recommendation of the recent article by Kim et al.¹ that health researchers pay heed to the strong social patterning of missing data often exhibited by key variables in epidemiologic studies. Yet, although we agree with the authors' advice that researchers should “at a minimum, carefully examine characteristics of respondents with missing income information,” we must strongly caution readers against their recommendation to “routinely includ[e] a separate income category of respondents with missing income information in all analyses.”

Contradicting this recommendation is extensive literature on missing data,^2–7 including two articles cited by Kim et al.^2,4 This research has shown that this “missing indicator” method will result in biased effect estimates under most conditions. In particular, bias occurs even when the missing data are missing completely at random (e.g., people missing data on income are a random sample from all income groups). If the data are missing at random (e.g., missingness on income depends only on variables that are observed), then multiple imputation or weighting techniques can be used to obtain valid effect estimates.⁶ If, however, the data are not missing at random (e.g., there are additional unobserved predictors of missingness), more complex models for non-ignorable nonresponse are required,⁷ and, ultimately, investigators would do well to adopt a sensitivity analysis framework. In all cases, researchers must give serious thought to data quality issues raised by the extent and patterning of missingness, the pathways leading to this missingness, and the implications for valid causal inference.

In summary, the problem of the social patterning of missing data that Kim et al. highlight is very real and troubling for epidemiologic research and underscores why epidemiologists cannot afford to ignore poverty and its impact on both health status and causal inference.⁸ To do the research right, we must use appropriate methods. In 1995, Greenland and Finkle² were alarmed to find that “the indicator method [was] widely perceived as a formally correct method of handling missing values.” Because this view continues to persist in some quarters, we must reiterate their recommendation that epidemiologists avoid the “potentially disastrous ad hoc” missing indicator approach.