Measuring small-area violent crime: A comparison of observed vs model-estimated crime rates and preterm birth

Abstract

Purpose:

Research examining the association between crime and health outcomes has been hampered by a lack of reliable small-area (e.g. census tract or census block-group) crime data. Our objective is to assess the accuracy of synthetically estimated crime indices for use in health research by using preterm birth as a case study.

Methods:

We used violent crime data reported by 47 law enforcement agencies in 15 counties in Atlanta, Georgia and compared them with commercially estimated crime rates from the same year to assess (1) how two measures of crime were correlated; and (2) if the associations between violent crime rate indices and preterm birth (PTB) varied as a function of the source of crime index. To assess the association between violent crime and PTB, we used multilevel logistic regression and controlled for potential individual and neighborhood-level confounders.

Results:

Violent crime, both estimated and observed, was positively correlated with poverty, neighborhood proportion Black, and NDI, however the association was stronger using estimated rates as compared to observed crime rates. The association between living in a high violent crime neighborhood and pre-term birth was only consistent for white women across the two crime indices after covariate adjustment. For Black women, the association between living in a high violent crime neighborhood and PTB is systematically underestimated across all models when the estimated crime rate is used.

Conclusions:

There is evidence that model-estimated crime rates are not reliable proxies for crime in an urban area even when appropriate confounders are adjusted for.

Introduction

Over the past two decades, social epidemiologists have incorporated theoretical and conceptual tools from social science disciplines to substantially broaden the meaning of “exposure” in epidemiologic analysis of health in populations (1, 2). Neighborhood-level effects such as segregation, ethnic enclaves, social capital, material capital, collective efficacy, and social disorder are just a few examples of socio-ecologic processes that may influence health (3, 4). Exposure to neighborhood-level violent crime also has health implications, yet it remains understudied with respect to health outcomes.

Higher rates of area-level crime are thought to affect health through increased stress, a tendency for social isolation, and decreased physical activity (1, 5, 6). Individuals perceive crime in spatio-temporal terms, which makes the neighborhood environment a prominent setting for perpetuating fear and stress (7). Furthermore, neighborhood-level violent crime is stressful beyond perceived personal risks of victimization due to fear for the safety of friends and family, referred to as “altruistic fear” (8). Neighborhood-level violent crime can also lead to social isolation by promoting distrust of others, which has been linked to adverse physical health outcomes (9–11).

Pregnancy outcomes, such as preterm birth (PTB), are particularly sensitive to social stressors, including neighborhood violent crime because the central role of stress-mediated changes in inflammation and neuroendocrine function make pregnancy an especially context-sensitive life stage (12–17). For example, ubiquitous patterns of racial and economic disparities in perinatal outcomes are not reducible to individual-level risk factors. They appear to vary as a function of environmental exposures that differ, on average, between low- and high-income women and between white women and women of color due to racial and economic residential segregation (18, 19). In other words, some portion of racial disparities in perinatal health outcomes may arise because of racial differences in lived experience in neighborhoods. If neighborhood-level violent crime marks a stress-inducing environment, then low income and Black women who are at increased risk of living in high crime neighborhoods may exhibit excess perinatal risk.

However, one challenge with conducting an empirical study of the population impact of neighborhood-level violent crime on health outcomes is concern for measurement validity and quality. In previous studies, crime exposure has been characterized by both police crime reports and some index value created by a data aggregator firm (20–22). This variation in data sources could prove to be problematic if the source of the crime data matters. In particular, there is concern around access to reliable small area estimates of neighborhood-level violent crime rates. While violent crime data are collected by individual law enforcement agencies, spatial and temporal granularity are often lacking. Publicly available data from the Federal Bureau of Investigation’s (FBI) Uniform Crime Report (UCR) are reported to the Department of Justice and aggregated to the county level (23) yet these nationally available county-aggregated crime data may be too spatially coarse to be meaningful. To acquire sub-county estimates (e.g. neighborhood-specific), researchers must negotiate data sharing with a complex patchwork of law enforcement agencies serving an area. Such efforts to produce a consistent, geographically extensive, and widespread data collection (e.g. for a whole state or the country at large) is often impractical.

To fill this void, many commercial data aggregator firms including the Environmental Systems Research Institute (ESRI), Simply Analytics, and others have created small area modeled estimates of neighborhood-level violent crime rates (24). While specific methods used to generate estimates are often proprietary with only limited methodological detail provided, in general, they use statistical modeling techniques to decompose county level UCR crime counts and rates into small area (e.g. census tract or census block-group) counts and rates by using ancillary data including demographic structure and composition of local places. For example, a model might predict small area crime by applying the county level relationship between age, income, and race/ethnicity to the small-area demographic structure of neighborhoods. Population health researchers interested in understanding whether (or how) the relative rate of violent crime across neighborhoods might affect people differently by age, race, or income may have concerns that these modeled estimates introduce dependencies between local estimates of an exposure and population structure that obscure rather than enhance etiologic research. In other words, it is possible that some modeled estimates have ‘baked in’ assumptions about some of the very socially constructed population factors that the health researcher seeks to understand.

In this paper, we evaluate the validity of one ‘modeled’ small area estimate of violent crime against a geographically consistent summary of violent crime reported by law enforcement agencies to understand whether the choice of measure matters in estimating the association between crime and preterm birth. We seek to answer two questions: 1) How do observed and model-estimated neighborhood-level violent crime rate indices compare in the Atlanta Metropolitan Area? and 2) Does the association between neighborhood-level violent crime rate indices and PTB, stratified by race, vary as a function of the source of the crime index?

Materials and Methods

Data

Observed Violent Crime

Two investigators [author and project PI DFH and project PI Dr. Hannah LF Cooper] contacted the 47 law enforcement agencies (i.e., Sherriff Office, Police Department) with jurisdiction over Type 1 violent crimes (murder, non-negligent manslaughter, forcible rape, robbery, and aggravated assaults) in 15 counties in the Atlanta MSA (Clarke, Clayton, Cobb, Coweta, DeKalb, Douglas, Fayette, Fulton, Gwinnett, Henry, Newton, Oconee, Rockdale, Spaulding, and Walton) to request address or block-level violent crime data reported in each census tract in 2013. Data were reported at the point (e.g. lat/long) level, geocoded, and spatially joined to 2010 census tract boundaries. The location was where the crime occurred, not where the victim lived. We calculated the observed violent crime rate by dividing the total count of crime by the total population of the census tract as extracted from American Community Survey (ACS) 2009–2013 5-year summary (25). We considered this our ‘gold standard’ for violent crime rate. Of the 47 agencies contacted, three were unable to provide usable data (two did not maintain accessible record management systems and data from one agency was excluded because simple and aggravated assaults could not be distinguished). For another agency, 2012 data were used, as 2013 data were not available at the time of data collection.

Estimated Violent Crime

ESRI provided estimated violent crime indices, derived from UCR county level estimates, for each census tract (26). Five crime categories were included in the estimated rate: murder, non-negligent manslaughter, forcible rape, robbery, and aggravated assaults. ESRI modeled each of the five crime types separately, using a range of approximately 65 socioeconomic characteristics taken from the Census. The final crime rate estimates were weighted by population and aggregated to the national totals. In recent years, ESRI has been explicit in stating that they do not include any data variables relating to race, ethnicity, ancestry, or language spoken at home in their predictive modeling. The methodology from 2013 and earlier, however, is not clear on these points. No details are provided on the specifics of the 65 sociodemographic varaibles used in their modeling. More information on their methods can be found elsewhere (26).

The two indices were on different scales: the observed crime data are characterized as a rate with events divided by population. In contrast, the modeled-crime index from ESRI maintains rank-order and scale based on crime levels but is not numerically interpretable as events per capita. We aimed to make both continuous and categorical versions of each index comparable across the two measures of area-based crime. Because crime indices were skewed, each was log-transformed and then standardized so that zero represents the mean log-index value and each 1-unit represents a one standard deviation change in the log of crime. In addition, we categorized each tract-specific index into quartiles to make rank-order categories comparable. The first quartile (Q1) represented the lowest quartile of violent crime for each measure.

Population Health Data

To assess the association between violent crime rate indices and PTB, we used birth outcome data from fifteen counties in Atlanta, GA (Clarke, Clayton, Cobb, Coweta, DeKalb, Douglas, Fayette, Fulton, Gwinnett, Henry, Newton, Oconee, Rockdale, Spaulding, and Walton). Birth certificate data was made available by the Office of Health Indicators for Planning (OHIP) of the Georgia Department of Public Health (27). All births to women residing in these counties and delivering live-born infants between 2013 and 2015 were eligible. PTB is defined as a live born infant born before 37 weeks’ gestation (28). We estimated gestational age from the mother’s last menstrual period. We restricted analysis to live, singleton births among non-Hispanic white and non-Hispanic Black women (n = 52,712). Maternal residential addresses at the time of birth were geocoded by the Office of Health Indicators for Planning of the Georgia Department of Public Health to identify residential Census Tract.

Control Variables

Individual-level covariates included maternal age (categorized as ≤ 19, 20–24, 25–29, 30–34, and 35+), race/ethnicity (non-Hispanic white and non-Hispanic Black), and maternal education (less than high school, high school/GED, and more than high school). All individual covariates were extracted from birth certificates. These individual-level variables are established risk factors for PTB and possibly non-exchangeable (unbalanced) across strata of neighborhood violent crime (29, 30).

Neighborhood-level covariates that may affect the exposure-outcome relationship included poverty, racial residential segregation and local racial composition, and the neighborhood deprivation index. All neighborhood-level covariates were derived from the same ACS 5-year estimates (2009–2013). We calculated poverty by dividing the total population below the poverty level by the total population for each census tract. To calculate the proportion of the population that was non-Hispanic Black, we used an estimate for each census tract. The neighborhood deprivation index (NDI) is a composite measure of contextual socioeconomic status (31). It is a continuous variable scaled such that zero represents the “average” neighborhood material conditions. More positive values are indicative of higher deprivation (more poverty, higher unemployment rates, and lower human capital such as occupational status or education), while more negative values indicate healthier environments.

Data Analysis

We examined the prevalence of individual and neighborhood attributes by birth outcome status as well as stratified by race. We used the categorical quartile operationalization of crime to compare the number of women living in, and neighborhood characteristics of, each violent crime rate quartile for observed and estimated violent crime rates. We used the scaled continuous operationalization of crime to estimate the association between observed and estimated violent crime rates separately, and the odds of PTB stratified by race using multilevel logistic regression, adjusting for confounders and accounting for interaction of crime index with race. We adjusted for all individual-level confounders (age and education) in model 1, all neighborhood-level confounders (poverty, proportion Black, and NDI) in model 2, and both sets of potential confounders in model 3. We used SAS 9.4 for our statistical analyses.

Results

Descriptive statistics

Overall, there were 4,470 PTB’s in our study (7%) (Table 1). The rate of PTB among non-Hispanic Black women was almost twice the rate of PTB among non-Hispanic white women (9% versus 5%) (Table 2). Distributions of maternal age were different between the two groups. The majority of non-Hispanic white women were 30–34 years of age (37%), while the majority of non-Hispanic Black women were 20–24 years of age (30%). The greatest proportion of births for both non-Hispanic white and Black women was 35+ years of age (57% and 44% respectively) (Table 2). PTB occurrence primarily occurred in women with less than a high school/GED education level (11%) as compared to those with more than a high school education.

Table 1.

Distribution of individual and neighborhood attributes by singleton birth outcome¹ status among a sample living in selected counties² in metro Atlanta, GA (2013 – 2015)

	Total	Preterm (≤ 37 weeks)
	N	N	%
Overall	62,814	4,470	7.1
Maternal age (years)
≤ 19	1,698	155	9.1
20–24	13,031	982	7.5
25–29	6,801	539	7.9
30–34	9,879	669	6.8
35+	31,405	2,125	6.8
Maternal race/ethnicity
Non-Hispanic Black	34,447	3,002	8.7
Non-Hispanic White	28,367	1,468	5.2
Maternal education
< HS education	5,406	573	10.6
High school grad/GED	14,138	1,111	7.9
> HS education	43,270	2,786	6.4
Smoked during pregnancy
Yes	2,068	125	6.0
No	60,746	4,345	7.2
Number of previous births
Mother’s first birth	26,961	2,148	8.0
2–3	27,349	1,879	6.9
≥ 4	8,504	443	5.2
Violent Crime
Observed3
Low (0 – 0.9)	13,595	642	4.7
Medium (1.0 – 2.8)	16,453	1,188	7.2
High (2.9 – 6.0)	19,047	1,509	7.9
Very high (6.1 – 77.6)	13,719	1,131	8.2
Range; median	0 – 77.6; 2.8	0 – 77.6; 4.1
Estimated4
Low (0.4 – 8.3)	17,928	1,181	6.6
Medium (8.4 – 17.3)	19,090	1,175	6.2
High (17.4 – 45.9)	15,606	1,273	8.2
Very high (45.9 – 559.4)	10,190	841	8.3
Range; median	0.4 – 559.4; 17.3	0.4 – 559.4; 19.6

¹Birth data was used from vital birth records found on Georgia Department of Public Health website

²Fifteen counties in Georgia were included in this analysis: Clarke, Clayton, Cobb, Coweta, DeKalb, Douglas, Fayette, Fulton, Gwinnett, Henry, Newton, Oconee, Rockdale, Spalding, and Walton.

³The observed crime rate was calculated by dividing the total count of crime by the total population for the census tract

⁴The estimated crime rate is the modeled index value provided by the Environmental Systems Research Institute for each census tract

Table 2.

Race-stratified distribution of individual- and neighborhood-attributes among women living in selected counties¹ in metro Atlanta, GA (2013–2015)

	Total	Non-Hispanic White		Non-Hispanic Black
		N	%	N	%
Overall	52,712	23,983	45.5	28,729	54.5
Outcome
Preterm Birth (≤37 weeks)2	5,078	1,450	6.0	3,628	12.6
Individual-Level
Maternal age (years)
≤ 19	4,103	636	2.7	3,467	12.1
20–24	11,389	2677	11.2	8,712	30.3
25–29	13,245	5938	24.8	7,307	25.4
30–34	14,566	8976	37.4	5,590	19.5
35+	9,409	5756	24.0	3,653	12.7
Maternal education
< HS education	4,906	926	3.9	3980	13.9
High school grad/GED	14,690	3671	15.3	11019	38.4
> HS education	33,116	19386	80.8	13730	47.8
Smoked during pregnancy
Yes	1,848	890	3.7	958	3.3
No	50,864	23,093	96.3	27,771	96.7
Number of previous births
1	23,799	11,762	49.0	12,037	41.9
2–3	23,149	10,799	45.0	12,350	43.0
≥ 4	5,764	1,422	5.9	4,342	15.1
Neighborhood-Level
Violent Crime
Observed3
Low (0 – 0.6)	10,882	8,230	34.3	2,652	9.2
Medium (0.7 – 2.5)	13,937	7,871	32.8	6,066	21.1
High (2.6 – 6.6)	15,674	5,312	22.1	10,362	36.1
Very high (6.7 – 77.6)	12,219	2,570	10.7	9,649	33.6
Range; median	0 – 77.6; 2.8	0 – 77.6; 1.5		0 – 77.6; 4.8
Estimated4
Low (0 – 9.0)	14,199	8,359	34.9	5,840	20.3
Medium (9.1 – 22.5)	15,624	6,983	29.1	8,641	30.1
High (22.6 – 75.6)	13,390	4,622	19.3	8,768	30.5
Very high (75.7 – 559.4)	9,499	4,019	16.8	5,480	19.1
Range; median	0 – 559.4; 17.9	0.4 – 559.4; 14.8		0 – 559.4; 21.7

¹Fifteen counties in Georgia were included in this analysis: Clarke, Clayton, Cobb, Coweta, DeKalb, Douglas, Fayette, Fulton, Gwinnett, Henry, Newton, Oconee, Rockdale, Spalding, and Walton

²Birth data was used from vital birth records found on Georgia Department of Public Health website

³The observed crime rate was calculated by dividing the total count of crime by the total population for the census tract

⁴The estimated crime rate is the modeled index value provided by the Environmental Systems Research Institute for each census tract

Among women who had a PTB, 25% lived in a very high crime neighborhood (4th quartile) using the observed violent crime index while only 19% lived in a very high crime neighborhood using the estimated violent crime index (Table 1). Regardless of the crime index used, non-Hispanic Black women lived in neighborhoods with more crime (Table 2). The indices of crime rate differ with respect to demographic composition of tracts; for example, 31% versus 17% of women were non-Hispanic Black in the highest violent-crime-rate quartile using observed versus estimated respectively (Table 2).

Violent crime, both estimated and observed, was positively correlated with poverty, neighborhood proportion Black, and NDI, however the association was stronger using observed rates as compared to estimated crime rates (Figure 1).

Figure 1.

Characterizing the level of poverty, the racial composition, and the neighborhood deprivation in census tracts categorized by quartile of observed crime versus modeled crime in selected counties of metropolitan Atlanta, GA (2013–2015)

Observed versus Estimated Crime Rate

In the unadjusted model for observed indices, one standard deviation change in the log of observed violent crime was associated with an increased odds of PTB among non-Hispanic Black women (OR = 1.32, 95% CI: 1.13–1.55) but an essentially null association among non-Hispanic white women (OR = 0.97, 95% CI: 0.83–1.12) (Table 3). In the unadjusted model for estimated indices, one standard deviation change in the log of estimated violent crime was associated with an increased odds of PTB among non-Hispanic Black women (OR = 1.22, 95% CI: 1.06–1.41) and a decreased odds of PTB among non-Hispanic white women (OR = 0.72, 95% CI: 0.62–0.85).

Table 3.

Race stratified unadjusted, individual-level adjusted, neighborhood-level adjusted, and fully adjusted odds ratios and 95% confidence intervals for preterm birth¹ (<37 weeks gestation) in selected counties² in Atlanta, GA (2013 – 2015)

	Observed Crime3		Estimated Crime4
	White	Black	White	Black
Unadjusted	0.97 (0.83, 1.12)	1.32 (1.13, 1.55)	0.72 (0.62, 0.85)	1.22 (1.06, 1.41)
Individual adjusted	0.75 (0.63, 0.88)	1.39 (1.18, 1.62)	0.72 (0.62, 0.85)	1.19 (1.03, 1.38)
Neighborhood adjusted	0.76 (0.62, 0.93)	1.46 (1.19, 1.79)	0.73 (0.62, 0.86)	1.25 (1.07, 1.46)
Fully adjusted	0.77 (0.63, 0.94)	1.43 (1.17, 1.75)	0.74 (0.63, 0.87)	1.23 (1.06, 1.44)
The unadjusted model does not control for any confounders
The individual adjusted model controls for age and maternal education
The neighborhood adjusted model controls for poverty, the proportion of the census tract that is black, and the neighborhood deprivation index
The fully adjusted model controls for age, maternal education, poverty, the proportion of the census tract that is black, and the neighborhood deprivation index

¹Birth data was used from vital birth records found on Georgia Department of Public Health website

²Fifteen counties in Georgia were included in this analysis: Clarke, Clayton, Cobb, Coweta, DeKalb, Douglas, Fayette, Fulton, Gwinnett, Henry, Newton, Oconee, Rockdale, Spalding, and Walton

³The observed crime rate used in models is the standardized value of the log-transformed crime ratio calculated by dividing the total count of crime by the total population for the census tract

⁴The estimated crime rate used in models is the standardized value of the log-transformed crime index values provided by the Environmental Systems Research Institute for each census tract

After controlling for individual-level confounders alone, the direction of the association between observed violent crime and PTB remained, but the magnitude of the association was strengthened among both non-Hispanic Black women (OR = 1.39, 95% CI: 1.18, 1.62) and non-Hispanic white women (OR = 0.74, 95% CI: 0.63, 0.88). In contrast, as the log of estimated violent crime increased in models controlling for individual-level confounders alone, the association with PTB was attenuated slightly among non-Hispanic Black women (OR = 1.19, 95% CI: 1.03, 1.38) and remained unchanged for non-Hispanic white women (OR = 0.72, 95% CI: 0.62, 0.85).

When adjusting for only neighborhood-level confounders, the association between the log of observed violent crime and PTB increased among non-Hispanic Black (OR = 1.46, 95% CI: 1.19, 1.79) and only slightly changed for non-Hispanic white women (OR = 0.76, 95% CI: 0.62, 0.93). A similar pattern was seen when the estimated crime indices were used. The association between one standard deviation change in the log of estimated violent crime and PTB and controlling for neighborhood-level confounders alone increased among non-Hispanic Black women (OR = 1.25, 95% CI: 1.07, 1.46) and remained similar with overlapping confidence intervals for non-Hispanic white women (OR = 0.73, 95% CI: 0.62, 0.86).

In the fully adjusted model, the association between the log of observed violent crime and PTB was slightly attenuated as compared to the model adjusting for neighborhood-level confounders alone among both non-Hispanic Black (OR = 1.43, 95% CI: 1.17, 1.75) and non-Hispanic white women (OR = 0.77, 95% CI: 0.63, 0.94). A similar pattern was seen when the estimated crime indices were used. The association between one standard deviation change in the log of estimated violent crime and PTB, when controlling for all confounders, slightly decreased from the model adjusting for neighborhood-level confounders alone among both non-Hispanic Black (OR = 1.23, 95% CI: 1.06, 1.44) non-Hispanic white women (OR = 0.74, 95% CI: 0.63, 0.87).

Discussion

Social epidemiologists are interested in understanding the impact of the social and built environment on population health. However, this is not possible without a valid and reliable measurement of neighborhood-level processes. Investigators often find themselves facing the choice between more reliable and granular data of extremely limited areas, or possibly less reliable measures available over a wide spatial extent. In this study we find that at least one version of estimated small-area violent crime differs from observed values in ways systematically related to socially constructed dimensions of stratification and inequality. The encoding of structural inequality into modeled estimates of social context risks clouding understanding and inference.

However, we also find that with adjustment for available individual- and neighborhood-level covariates, the estimated and observed crime indices partially converge with respect to their association with PTB, particularly for white women. While several studies have used ESRI’s estimated crime indices in their research (32–34), this is the first study to assess the reliability of these measures compared to a gold standard reported directly by law enforcement agencies. Our study is unique because we contacted multiple law enforcement agencies to produce a geographically coherent observed crime dataset across an otherwise complex patchwork of agency jurisdictions; these data provided us with gold standard observed crime rate data to compare to estimated crime rates. While our study is limited to the Atlanta metropolitan area, prior work has found similar race-specific patterns between the violent crime and preterm birth (20). Furthermore, we provide empirical evidence that can inform future studies looking at crime as a potential exposure or confounding variable. By using the well characterized connection between stress and PTB, we were able to use PTB as a case study to examine the impact of measurement error on health outcomes.

Our study suggests that estimated crime indices may provide systematically biased results for studies analyzing and controlling for the effects of tract-level crime on health, even after adjustment for the individual and area-based factors that may be ‘baked into’ the modeled estimates. The two crime indices, different on a crude level, have profound effects when used in modeling and controlling for confounding. The association between living in a high violent crime neighborhood and pre-term birth was only consistent for white women across the two crime indices after covariate adjustment. For Black women, the association between living in a high violent crime neighborhood and PTB is systematically underestimated across all models when the estimated as compared to the observed crime rate is used. This may be because of how the estimated crime rate is constructed. The 2013 ESRI crime indices were modeled using a range of approximately 65 sociodemographic characteristics taken from the Census (26). However, the specific variables and the exact modeling methodology have not been defined for the public. It is plausible that the ESRI modeled crime estimates have incorporated information about racial, ethnic, or socioeconomic structure of neighborhoods with an assumption that all neighborhoods of a given demographic profile will share the same crime rate; this potentially faulty assumption could bias the estimates of association between crime and health.

In 2018, ESRI updated their description of methodology for estimating crime rates to specify that no data variables relating to race, ethnicity, ancestry or language spoken at home were used, though they remained vague on what specific variables they included in their models (29). It is unclear if this policy is new for 2018, or rather simply unspecified before. Given the degree to which racial and economic segregation produce a spatial stratification of environments and experiences, it is also unclear whether other variables used in synthetic model estimates might similarly bias metrics. For example, the divergent results in the association of violent crime with PTB by maternal race may reflect racial differences in experiences of law enforcement response to crime (e.g. Black and white women may not share the same level of ‘protection’ from police response). In fact, it is difficult to disentangle the role of actual violence being committed from the racially-distinct processes for surveilling and detecting crime, including concern for over-policing in predominantly Black and Brown neighborhoods (36). Atlanta is undergoing processes of gentrification during this time period as well. In such settings, white women who ‘select’ into neighborhoods with higher crime may be distinct from Black women who have lived there longer.

Our findings have important public health implications for spatial epidemiology and the modifiable area unit problem as it relates to crime measurement. For the majority of neighborhoods in the United States, crime data are only available at the county level. Commercial data aggregator firms, however, provide estimated crime rates at the census block-group and census tract level for the entire United States. Building an appropriate predictive model that correctly estimates highly variable small-scale crime rates from large scale county-crime rates can prove to be conceptually and methodologically challenging. Neighborhood units only provide a rough measure of neighborhood context and can lead to incorrect conclusions about the true effects of neighborhood characteristics (13, 37). Characteristics within each census tract can vary considerably, and individuals who reside in the same tract may have different lived experiences. The same level of variation may even be true for smaller levels of census aggregation (i.e. block groups) though these areas are likely to have more homogenous characteristics. Since the scale and boundaries for census units such as block-groups and tracts have limited meaning for the spatial distribution of crime, it is to be expected that estimating crime rates for these spatial units is difficult when extrapolating from crime data that are only available at the county level. The technical description provided by ERSI regarding its methods and model fit is rather vague. ESRI reports models fit account for over 85% of the variance in crime rates at the “jurisdiction level”. The jurisdiction level, however, is only one level below the county level for which the UCR provides crime data. This is a much easier task than predicting crime at the census tract-level. An additional limitation is it the possibility that reliability of reporting crime varies by agency, and this could produce added unobserved variation. No information is provided on model performance for the census tract level.

Our study has several limitations. We only use data from one urban area and predictive quality of the estimated crime indices might be variable in other urban and rural areas. Further validation studies comparing synthetically estimated crime rates to actual crime data are necessary to assess how the predictive performance varies across rural neighborhoods. We did not asses block-group level ESRI indices and cannot ascertain if crime measured at the block-group or block-level would have the same accuracy. Our analysis of violent crime was limited to reported crimes in Atlanta in 2013, yet our birth outcome data were for 2013–2015. It is difficult to know how much violent crime rates vary between years, and whether the rates for 2013 are a suitable proxy for the crime a woman would have experienced in her neighborhood during pregnancy. In addition, not all violent crimes are reported to the police. It is plausible that crime rates in some areas may be underreported. Reporting rates differ by race, age, and sex of the victim, as well as the type of crime, resulting in potential misclassification bias in the observed crime rate. Finally, most researchers are likely interested in assessing crime in a given year in order to assess a more direct impact of crime as an exposure or confounder, yet ESRI’s 2013 release consisted of a time series of 6 years of data from 2005–2010 (35).

Conclusions

Our findings suggest that ESRI estimated crime indices may produce biased estimates for studies analyzing and controlling for the effects of crime on health. Future research assessing the reliability of ESRI’s estimated crime indices after the update to the methodology in 2018 is needed to determine if these new crime estimates are more reliable. Estimated crime rates, if proven a valid substitute to observed crime rates, have the potential to save time and money in future research looking at neighborhood-level violent crime as an exposure or confounder to better understand the impact of crime on health. As it stands, however, ESRI estimated crime rates should be used with caution.

Abbreviations:

PTB

preterm birth

FBI

Federal Bureau of Investigation

UCR

Uniform Crime Report

ESRI

Environmental Systems Research Institute

Q1

first quartile

ACS

American Community Survey

NDI

neighborhood deprivation index