We thank Wang et al. ( 1 ) for their continued interest in our previous work (2) and for identifying the small typographical error in the off-diagonal element of the Fisher's information matrix for the normal case and the I 22 element of the gamma case. We reviewed our simulation code by comparing the Fisher's information matrices therein to those in our original manuscript and found that our code was not reflective of the typographical error. We then compared our simulation code, the code provided by Wang et al., and simple approximations of second derivatives based on 2 small deltas and were pleased to confirm that our code and simulations were in fact correct ( 1 ). Thus, we have renewed confidence in the validity of our original simulations and that the simulation differences the authors discuss are well within simulation error. More importantly, a comparison with the Fisher's information matrix that contains the typographical errors shows a small impact on the variance of the area under the receiver operating characteristic curve. Nevertheless, we always favor science without mistakes, regardless of how small the mistake may be.
Although the code provided by Wang et al. ( 1 ) may be a useful tool for likelihood-based estimation of the area under the receiver operating characteristic curve from data with missing data below a limit of detection (LOD), more contemporary methodologies have been developed since our original publication 10 years ago that provide broader solutions to account for data below the LOD. Examples of these modern techniques include either using the machine-read values when available or utilizing multiple imputation bounded by the LOD when machine-read values are not available ( 3–5 ). These methods are more robust to distributional assumptions, are easier to implement with standard statistical packages, and are applicable to a variety of analyses beyond the receiver operating characteristic curve. Our focus continues to be on methods that have a large impact, as we invest our intellectual capital prudently to improve public health.
ACKNOWLEDGMENTS
This research was supported by the Intramural Research Program of the Eunice Kennedy Shriver National Institute of Child Health and Human Development, National Institutes of Health, Bethesda, Maryland.
Conflict of interest: none declared.
REFERENCES
- 1. Wang M , Li Z , Reeves B . Re: “Receiver operating characteristic curve inference from a sample with a limit of detection” [letter]. Am J Epidemiol . 2016. ; 184 ( 7 ): 552 – 553 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Perkins NJ , Schisterman EF , Vexler A . Receiver operating characteristic curve inference from a sample with a limit of detection . Am J Epidemiol . 2007. ; 165 ( 3 ): 325 – 333 . [DOI] [PubMed] [Google Scholar]
- 3. Whitcomb BW , Schisterman EF . Assays with lower detection limits: implications for epidemiological investigations . Paediatr Perinat Epidemiol . 2008. ; 22 ( 6 ): 597 – 602 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Chen H , Quandt SA , Grzywacz JG , et al. . A distribution-based multiple imputation method for handling bivariate pesticide data with values below the limit of detection . Environ Health Perspect . 2011. ; 119 ( 3 ): 351 – 356 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Royston P . Multiple imputation of missing values: further update of ice, with an emphasis on interval censoring . Stata J . 2007. ; 7 ( 4 ): 445 – 464 . [Google Scholar]