Abstract
Objective:
Approximately 1–2 of every 1000 American newborns has hearing loss identified by newborn screening. This study was designed to determine if infant hearing loss is more common in socioeconomically disadvantaged communities.
Study Design:
In this retrospective study we analyzed electronic medical record data using geostatistical models.
Setting:
Infants were residents of Durham County, NC, born in two hospitals of the Duke University Health System. This county includes the city of Durham and surrounding suburban and rural communities.
Subjects:
Subjects were hearing-screened newborns, born between 2005 and 2016, whose residential address was in Durham County, NC.
Methods:
This was a retrospective study using medical record data. We used Bayesian regression models with smoothing of coordinate date to identify both spatial and non-spatial predictors of infant hearing loss.
Results:
We identified 19,348 infants from Durham County, NC, of whom 675 had failed initial hearing screening and 191 had hearing loss confirmed on follow up. Hearing loss was significantly associated with minority race (OR 2.45, 95% CI 1.97 – 3.06), as well as lower gestational age and maternal sexually transmitted infections. We identified significant geographic heterogeneity, with a higher probability of hearing loss in poorer urban neighborhoods (local OR range 0.59–1.39). Neighborhood disadvantage was a significant predictor of hearing loss, as was high local seroprevalence of CMV among pregnant women.
Conclusions:
Urban, low income neighborhoods have a high prevalence of infant hearing loss compared with more affluent surrounding communities, particularly among minorities. This distribution may be attributable to congenital CMV infection.
Introduction
Hearing loss affects 1 to 2 of every 1000 American newborns, and has various causes, including genetic disorders and infections 1. Screening newborns for hearing loss in the newborn period has become the standard of care, allowing for early interventions that improve hearing, communication, and developmental outcomes, and facilitates the treatment of underlying disorders. Approximately 95% of infants in the United States are now screened in the neonatal period, and newborn hearing screening is mandated in most states.
Congenital cytomegalovirus infection (CMV) is the leading infectious cause of hearing loss in infants 2. CMV infections, including congenital CMV, are more common among socially disadvantaged groups and among non-white minorities. We recently conducted geospatial analyses demonstrating that CMV seropositivity, including among pregnant women, significantly clusters in socioeconomically disadvantaged urban neighborhoods with large minority populations 3–5.
We hypothesized that the spatial distribution of infant hearing loss would be heterogeneous, and that hearing loss would be more common in urban and low income communities, especially among minorities. This distribution would be similar to that of CMV, raising the hypothesis that CMV produces a higher burden of infant hearing loss in these communities. To investigate this hypothesis we have performed a geospatial analysis of more than 19,000 hearing-screened infants born in a single health system.
Materials and Methods
Design
This was a case-control study using electronic health records and geographic identifiers.
Research Ethics Oversight
This study was approved by the Institutional Review Board of the Duke University School of Medicine.
Patient Data
We identified all infants born at Duke University Hospital and Duke Regional Hospital (formerly Durham Regional Hospital) between 2005 and 2016. The longitude and latitude spatial coordinates were obtained for the mother’s residential address at the time of birth. Additional health record data included infant gender, race, ethnicity, birth weight, gestational age at birth, admission to the neonatal intensive care unit (NICU), the results of CMV testing (if done), and the results of both initial and (if done) follow-up hearing testing. We also obtained data on maternal parity and maternal history of sexually transmitted infections (STI). The dataset was then limited to those infants with residential coordinates in Durham County, NC.
Hearing testing was performed by automated auditory brainstem responses, then repeated while still in the hospital if the initial result yielded a “fail” in either or both ears. A second “fail” generated a referral to pediatric audiology for follow up testing after discharge, scheduled to be done within 4 weeks of birth unless the patient was in the NICU.
A hearing screening failure can be defined different ways: some infants with a failed initial hearing screen passed follow up testing and others did not have follow up testing documented at all. To understand if our models were sensitive to the case definition of failure, we defined a “failed” hearing screen in four different ways for the purposes of our modeling:
-
1)
Any infant who failed initial screening and follow up testing, OR who failed initial screening but did not have documented follow up, was classified as “fail.”
-
2)
Infants who failed both initial and follow up testing were classified as “fail.” Infants who failed initial testing but had no documented follow up were classified as “pass.”
-
3)
Infants who failed initial testing but had no documented follow up were excluded. All failures were infants who had documented failure on follow up testing.
-
4)
Infants who failed initial testing but did not have documented follow up were randomized to fail vs pass at a ratio reflecting the documented proportion of failure on confirmatory testing in our dataset (339 fail vs 970 pass, 26%, 95% CI 24–28%). This value was determined from our initial dataset (not limited to infants from Durham County).
Data on CMV testing during pregnancy were from a cohort of women screened for CMV antibodies as part of the Maternal-Fetal Medicine Units Network study of CMV-hyperimmune globulin for treatment of primary CMV infection during pregnancy. We have previously published spatial analyses of this cohort 3,4.
We did not determine which infants in our dataset were siblings, nor which were infants of women in the separate maternal CMV dataset. We were not able to identify infants with genetic syndromes or congenital anomalies that may affect hearing, so these conditions were not excluded from this study.
Statistical and Spatial Analysis
Infants whose residential coordinates fell in Durham County, NC, were retained for this study. The goal of our primary analysis was to determine how the odds of infant hearing loss varied geographically within Durham County, and to identify both individual and neighborhood-level variables associated with this spatial distribution. To accomplish this we fit hierarchical Bayesian spatial models 6 using the brms package 7 in the statistical programming language R (www.r-project.org). The brms package facilitates construction of Bayesian regression models, which are transferred to the sampling program Stan 8 for Markov chain Monte Carlo sampling of the posterior distribution. Other key R packages used for this study included bayesplot 9, ggplot2 10, and viridis 11 for visualization of modeling results.
We used binomial generalized additive models to investigate the relationship of infant hearing loss to geography. The dichotomous result of hearing testing was our outcome variable, and our primary predictor variable was a 2-dimensional smooth of each infant’s longitude-latitude coordinate pair using splines. Consequently we were able to model a locally-varying relationship between geography and the odds of hearing loss. Other relevant covariates were added to models as linear predictors. “Significant” covariates were those which did not contain 0 in the central 95% of their posterior probability distribution. We used non-informative default prior distributions in our models.
After fitting each model we predicted the probability of hearing loss onto a dense longitude-latitude grid covering the spatial extent of the study area. In addition to reporting local probability, we also computed local odds ratio by dividing local odds by the average odds of the entire dataset. Regions where there was a 95% probability that the local odds differed from global odds were circumscribed with contours. We compared models using an information criterion computed from a leave-one-out cross validation (LOOIC) 12. Posterior distributions of model parameters were visualized using the MCMC_intervals function in bayesplot.
We interpolated our model predictions using ArcGIS 10.5 (ESRI, Redlands, CA) to create a smooth, continuous prediction surface. These prediction surfaces display geographic heterogeneity in the local probability of hearing loss. We also expressed this heterogeneity as local odds ratio, which in this case refers to the ratio of local odds to overall odds of hearing loss. ArcGIS was used for map production, and Photoshop Creative Cloud (Adobe Systems, San Jose, CA) for layout and effects such as drop shadows.
Our adjusted models incorporated infant variables (gender, race, gestational age, birth weight, NICU admission) and maternal variables (parity and history of maternal STI). In our initial dataset of 47,274 infants race and ethnicity had been described 135 ways, including nationalities and colloquialisms that did not easily categorize into common racial and ethnic groups. We did a preliminary analysis to determine how best to categorize race. In one model we categorized 41,101 of these infants to white, black, Hispanic, Asian, Middle Eastern, Native American, We separately recategorized these subjects simply as non-Hispanic white versus minority. We then ran Bayesian logistic regression models using each of the race categorizations as a predictor of hearing loss. We compared the two models using leave one out cross validation. Both models performed nearly identically, suggesting that multiple categories do not improve the model fit and that the most predictive distinction was whether an infant was white vs non-white. Thus, all analyzed infants were assigned to “non-Hispanic white” or “minority” for further analysis.
We also evaluated the effect of area deprivation index (ADI), a neighborhood index of material disadvantage derived from US 2010 Census and 2009–2013 American Community Survey data 13,14. ADI was modeled at the level of census block groups. To determine whether additional unmeasured neighborhood-level factors were associated with hearing loss, we evaluated models that included a random intercept term for census block group.
Only 146 out of 47,274 infants in our overall patient cohort had been tested for CMV, a proportion too small to statistically analyze an association between CMV and hearing loss at the individual level. Thus, we opted to investigate a spatial association between infant hearing loss and maternal CMV. To accomplish this we computed 1 km radius circular buffers around each infant and counted the number of both CMV seropositive and CMV seronegative women falling within that buffer. We then constructed a spatial model that included the counts of CMV positive and negative women within each infant’s 1 km buffer. These 1 km buffer zones were similar in area to the census block groups in our study area (6.28 vs. 5.02 km2). This buffer size was large enough to enumerate CMV positive and negative mothers in the outlying areas with lower data density.
The code for our models can be found in Supplement 1.
Results
Cohort Description
We obtained records for 47,274 hearing-tested infants born during the 12 study years. We identified 21,798 infants who resided in Durham County. Of these 19,348 (89%) had all data elements needed for our analysis (including birth weight, gestational age, gender, and race/ethnicity). The prevalence of failed hearing screening was nearly identical in the analyzed cohort to the overall Durham County cohort. Among the analyzed infants 9531 were female (49.3%, 95% CI 48.6 – 50.0%) and13619 were minority (70.4, 95% CI 69.7 – 71.0%) (Table 1). The mean birth weight was 3.32 kg (range 1.08 – 5.32, 95% CI 3.31 – 3.33). Mean gestational age was 39.3 weeks (range 26 – 46, 95% CI 39.3 – 39.3). There were 259 infants who were admitted to the NICU (1.34, 95% CI 1.18 – 1.51%). The mean maternal parity (including the current pregnancy) was 2 (range 1–12), and 5005 mothers had a history of STI (25.8, 95% CI 25.2 – 26.5%). Dichotomizing these into “minority” and “non-Hispanic white,” yielded 13,725 minority and 5779 white infants (70.4%, 95% CI 70–71%).
Table 1. Demographics of study population.
Mean | Median | Range | 95% CI | |
---|---|---|---|---|
Race and Ethnicity | ||||
Non-Hispanic White | 5779 (29.9%) | 29.2–30.5% | ||
Minority | 13619 (70.4%) | 69.7–71.0% | ||
Gender | ||||
Female | 9531 (49.2%) | 48.6–50.0% | ||
Male | 9817 (50.7%) | 50.0–51.4% | ||
Birth Weight (kg) | 3.32 | 3.33 | 1.08–5.32 | 3.32–3.33 |
Gestational Age (weeks) | 39.3 | 39 | 26–46 | 39.3–39.3 |
NICU Admission | ||||
Yes | 259 (1.3%) | 1.18–1.51% | ||
No | 19089 (98.7%) | 98.5–98.9% | ||
Maternal Parity (prior pregnancies) | 2.52 | 2 | 1–12 | 2.51–2.54 |
Maternal STI | ||||
Yes | 5005 (25.9%) | 25.5–26.5% | ||
No | 14343 (74.1%) | 73.5–74.7% |
Hearing test results revealed 18,673 infants who either passed their initial hearing screen or failed their initial screen but passed follow up testing. The remaining 675 infants either failed both initial and follow up testing (191) or failed their initial testing and had no documented follow up (484), an overall failure rate of 3.6% (95% CI 3.4 – 3.9%) if all 675 of these infants were classified as failures. Minority infants were more than twice as likely than white infants to have hearing loss (4.2 vs 1.7%, OR 2.46, 95% CI 1.97–3.06).
Effect of Case Definition
As described in the Methods we evaluated four definitions to categorize the 484 infants who had a failed initial hearing test and lacked documented follow up. These classifications produced no substantial differences in the significant clinical predictors of hearing loss (Supplement 2A) or in the predicted spatial distribution of hearing loss (Supplement 2B).
Predictors of Hearing Loss
The probability of infant hearing loss was significantly associated with minority race/ethnicity, lower gestational age, and maternal history of STI. (Figure 1). Hearing loss was not significantly associated with birthweight, NICU admission, or maternal parity. While gender did not meet our criteria for significance, 92.6% of the posterior probability mass suggested an increased risk among males.
Spatial Distribution of Hearing Loss
Our unadjusted spatial model (Figure 2A) revealed marked geographic heterogeneity in the probability of infant hearing loss, corresponding to a probability locally ranging from 0.019 to 0.049. As compared with the average the local odds ratio varied from 0.59 to 1.39. A cluster of high odds was found in the urban neighborhoods of downtown Durham city. By contrast the lowest odds were found in more affluent, suburban neighborhoods.
Adjustment for gender, birth weight, gestational age, NICU admission, maternal STI history, and maternal parity somewhat blunted this spatial heterogeneity and diminished the probability range to 0.018 – 0.039. (Figure 2B). Adding race to this model reduced the probability range to 0.013 – 0.018 and eliminated any significant areas of clustering (Figure 2C).
Neighborhood Effects
A high ADI value, suggesting higher neighborhood-level socioeconomic disadvantage, was significantly associated with higher probability of infant hearing loss. There was a close spatial correspondence between probability of hearing loss and neighborhood disadvantage (Figure 3A). This effect, however, disappeared after adjustment for infant race.
Infant hearing loss closely corresponded to our previously published maps of CMV seroprevalence in pregnancy. Hearing loss was most prevalent in areas with high maternal CMV seroprevalence and less prevalent in areas where maternal CMV seroprevalence was low (Figure 3B). The probability of infant hearing loss was positively associated with the number of CMV seropositive mothers and negatively associated with the number of CMV seronegative mothers within 1 km of each infant. The confidence region for an effect of maternal CMV prevalence became non-significant, however, after adjustment for infant race.
Inclusion of a random intercept for block group identity did not substantially change the LOOIC cross-validation value. This was true in both adjusted and unadjusted models, and it was true regardless of whether individual coordinates were included in the model.
Discussion
We have identified marked geographic heterogeneity in the distribution of hearing loss among tested newborns, and this is strongly associated with race. The probability of hearing loss was more than twofold higher among newborns from the urban neighborhoods of Durham, NC, than for newborns from neighboring suburban communities. This difference persisted even after adjusting for individual clinical factors, such as birth weight and gestational age. Our models suggest a hearing loss incidence of 40 to 50 per 1000 infants in the highest risk areas, as compared with only 20 in the lowest. This would most heavily affect poor, urban communities that may have the fewest financial and social resources to accommodate this burden.
Minority race was the strongest predictor of hearing loss among the variables we had available. Because Durham County is quite racially segregated, adjustment for race largely eliminated the heterogeneous distribution of hearing loss. It is most likely, however, that race is a surrogate for other socioeconomic variables that affect care access and maternal health, ultimately increasing the risk of hearing loss.
The distribution of infant hearing loss was closely concordant with the distribution of CMV seropositivity among pregnant women from this community. In our previous study we reported that pregnant women from urban neighborhoods in Durham were much more likely to be CMV seropositive than women from surrounding suburban areas 4. While the hearing-screened infants in the current study were not (necessarily) related to the CMV-tested women, the odds of infant hearing loss were directly associated with the number of nearby CMV-seropositive women and inversely associated with the number of nearby CMV-seronegative women. Superimposing statistical maps of infant hearing loss and maternal CMV seropositivity demonstrated the close spatial relationship between the two.
While it is compelling that hearing loss may be more common in neighborhoods with high CMV seroprevalence, there are many other factors that must be considered. First, while CMV is certainly an important cause of infant hearing loss, a geographic relationship between the two could be coincidental. Both could independently result from unmeasured effects of socioeconomic disadvantage or other exposures common to urban environments. Nonwhite race was strongly predictive of infant hearing loss in this study and of maternal CMV in our prior study. Rather than a biologic predictor, however, we suspect that race is more likely a marker of local poverty, as both hearing loss (Figure 3A) and maternal CMV 3 are more common in neighborhoods with worse material disadvantage. Low gestational age and a maternal history of STI were significantly associated with infant hearing loss, factors that may be more common in socioeconomically disadvantaged communities. Adjusting for them, however, did not substantially affect our spatial models.
Our study suggests that much of the excess hearing loss observed in urban Durham is due to congenital CMV. Congenital CMV is treatable: early initiation of antiviral therapy can improve hearing outcomes in infected infants with hearing loss. It is not yet the standard of care to test all newborns for CMV, and only in a small number of states is CMV testing routinely done to evaluate a failed hearing screen. Implementation of universal newborn CMV testing and treatment may benefit many children in the highest prevalence communities through early intervention programs and antiviral therapy, thus reducing the overall burden of disability in these communities.
Our study has several limitations. Many infants with an abnormal initial hearing screen did not have documented follow up testing. Thus, we may have misclassified some infants as having failed their screening for lack of a confirmatory test. The number of infants admitted to a NICU was low, and it remains possible that infants requiring NICU care have a higher likelihood of hearing loss. We were unable to perform individual chart reviews of this large dataset to ascertain whether other important medical conditions might have best explained the hearing loss. Too few of the infants had been tested for CMV to directly study the distribution of congenital CMV. While race was an important predictor of hearing loss, race was recorded 135 different ways, forcing us to consolidate in order to use this variable analytically. We did find, however, that our dichotomous classification of race was highly correlated with hearing loss, and retaining more categories of race did not improve our statistical models. It is known that self-reported race is problematic, and it does not cleanly correlate with biological markers of ancestry. 15–17
Our geographic analyses were restricted to the recorded residential address in the newborn’s hospital record. Clearly this address does not entirely account for the totality of the mother’s exposures. Additionally we did not have information on prenatal care other than maternal STI in pregnancy. Finally, we were not able to identify which infants had other important causes of hearing loss, such as trisomy 21, trisomy 7, connexin deficiency, CHARGE association, anatomic inner ear abnormalities, and congenital toxoplasmosis. It is possible that some of these genetic or congenital conditions are also more common in socioeconomically disadvantaged areas and account for some of the excess infant hearing loss in those neighborhoods.
Our study has implications for both hearing screening and CMV testing policy. If disease due to congenital CMV is more common in socioeconomically disadvantaged communities, these communities may benefit from universal newborn CMV testing programs. We found that more than 2/3 of infants with a failed initial hearing screen did not have documented follow up – this suggests that hearing screening may not be an effective means to identify early-onset CMV-associated hearing loss. Universal CMV testing would identify many infants with asymptomatic CMV infection; it is uncertain whether there would be a public health benefit to this. Asymptomatically infected children appear to have academic outcomes comparable to uninfected children 18. Additional methods may be needed to identify late-onset hearing loss in children known to be CMV-infected. Finally, our study suggests that systems need to be put in place to ensure that hearing loss can be confirmed, ultimately helping affected infants get the medical evaluations and early interventions they need.
To conclude, we have found that infant hearing loss is spatially heterogeneous in the community, with particularly high prevalence among minority infants in poorer urban neighborhoods. Further research is needed to identify risk factors in these communities and to quantify the burden of morbidity and cost borne by these communities.
Supplementary Material
Acknowledgments
Funding Sources
Dr. Lantos was supported by the National Center for Advancing Translational Sciences of the NIH under award number KL2 TR001115.
Dr. Permar was supported by the NIH Director’s New Innovator Award DP2 grant, number HD075699.
Dr. Swamy was supported by the National Institute of Child Health and Human Development, Maternal Fetal Medicine Units Network, award number HD068258
Dr. Kind was supported by the National Institute on Minority Health and Health Disparities of the National Institutes of Health under award number R01MD010243.
The contents of this article are solely the responsibility of the authors and do not necessarily represent the official view of the NIH.
Abbreviations
- CMV
Cytomegalovirus
- ADI
Area Deprivation Index
- STI
Sexually Transmitted Infection
Footnotes
Financial Disclosures
The authors declare that they have no financial relationships or conflicts of interest relevant to this article to disclose.
Contributor Information
Paul M. Lantos, Email: paul.lantos@duke.edu.
Gabriela Maradiaga-Panayotti, Email: gabriela.maradiaga@duke.edu.
Xavier Barber, Email: xbarber@miumh.umh.es.
Eileen Raynor, Email: eileen.raynor@duke.edu.
Debara Tucci, Email: debara.tucci@duke.edu.
Kate Hoffman, Email: kate.hoffman@duke.edu.
Sallie R. Permar, Email: sallie.permar@duke.edu.
Pearce Jackson, Email: pearce.jackson@duke.edu.
Brenna L. Hughes, Email: brenna.hughes@duke.edu.
Amy Kind, Email: ajk@medicine.wisc.edu.
Geeta K. Swamy, Email: geeta.swamy@duke.edu.
References
- 1.Alam S, Gaffney M, Eichwald J. Improved newborn hearing screening follow-up results in more infants identified. J Public Health Manag Pract. 2014;20(2):220–223. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Cannon MJ, Davis KF. Washing our hands of the congenital cytomegalovirus disease epidemic. BMC Public Health. 2005;5:70. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Lantos PM, Hoffman K, Permar SR, et al. Neighborhood Disadvantage is Associated with High Cytomegalovirus Seroprevalence in Pregnancy. J Racial Ethn Health Disparities. 2017. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Lantos PM, Hoffman K, Permar SR, Jackson P, Hughes BL, Swamy GK. Geographic Disparities in Cytomegalovirus Infection During Pregnancy. J Pediatric Infect Dis Soc. 2017. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Lantos PM, Permar SR, Hoffman K, Swamy GK. The Excess Burden of Cytomegalovirus in African American Communities: A Geospatial Analysis. Open Forum Infect Dis. 2015;2(4):ofv180. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Lawson A, Banerjee S, Haining RP, Ugarte MaD. Handbook of spatial epidemiology. Boca Raton: CRC Press/Taylor & Francis; 2016. [Google Scholar]
- 7.Bürkner P-C. brms: An R Package for Bayesian Multilevel Models Using Stan. Journal of Statistical Software. 2017;80(1):28. [Google Scholar]
- 8.Carpenter B, Gelman A, Hoffman MD, et al. Stan: A Probabilistic Programming Language. Journal of Statistical Software. 2017;76(1):1–29. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Gabry J bayesplot: Plotting for Bayesian models. R package version 1.2.0. http://mc-stanorg/, Accessed August 31, 2017.
- 10.Gómez-Rubio V ggplot2 - Elegant Graphics for Data Analysis (2nd Edition). Journal of Statistical Software. 2017;77(Book Review 2):3. [Google Scholar]
- 11.Garnier S viridis: Default Color Maps from ‘matplotlib’. R package version 0.4.0. https://CRANR-projectorg/package=viridis, Accessed August 31, 2017.
- 12.Vehtari A, Gelman A, Gabry J. Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC. In: Statistics and Computing. 2016. [Google Scholar]
- 13.Kind AJ, Jencks S, Brock J, et al. Neighborhood socioeconomic disadvantage and 30-day rehospitalization: a retrospective cohort study. Ann Intern Med. 2014;161(11):765–774. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Kiefer DG, Vintzileos AM. The utility of fetal fibronectin in the prediction and prevention of spontaneous preterm birth. Rev Obstet Gynecol. 2008;1(3):106–112. [PMC free article] [PubMed] [Google Scholar]
- 15.Maglo KN, Mersha TB, Martin LJ. Population Genomics and the Statistical Values of Race: An Interdisciplinary Perspective on the Biological Classification of Human Populations and Implications for Clinical Genetic Epidemiological Research. Front Genet. 2016;7:22. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Mersha TB, Abebe T. Self-reported race/ethnicity in the age of genomic research: its potential impact on understanding health disparities. Hum Genomics. 2015;9:1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Lee YL, Teitelbaum S, Wolff MS, Wetmur JG, Chen J. Comparing genetic ancestry and self-reported race/ethnicity in a multiethnic population in New York City. J Genet. 2010;89(4):417–423. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Lopez AS, Lanzieri TM, Claussen AH, et al. Intelligence and Academic Achievement With Asymptomatic Congenital Cytomegalovirus Infection. Pediatrics. 2017;140(5). [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.