Ecological Research for Studies of Violence: A Methodological Guide

Abstract

Ecological research is important to the study of violence in communities. The phrases “ecological research” and “ecologic study” describe those research studies that use grouped or geographic units of analysis, such as zip codes, cities, or states. This type of research allows for the investigation of group-level effects and can be inexpensive and relatively quick to conduct if the researcher uses existing data. And, importantly, ecological studies are an efficient means for hypothesis generation prior to, and can be used to justify, costlier individual-level studies. Ecological research designs may be employed to study violence outcomes when the research question is at the population level, either for theoretical reasons, or when an exposure or intervention is at the population level, or when individual-level studies are not feasible; however, ecological research results must not be used to make individual-level inferences. This article will discuss reasons to conduct ecological-level research, guidelines for choosing the ecological unit of analysis, frequently used research designs, common limitations of ecological research, including the ecological fallacy, and issues to consider when using existing data.

The phrases “ecological research” and “ecologic study” describe those research studies that use grouped or geographic units of analysis, such as zip codes, cities, or states. Because features of the physical and social environment may impact the occurrence or frequency of violent events in an area, ecological research designs are regularly used to study violent outcomes. There are multiple data and design issues related to ecological research that make these studies distinct, and this article serves to highlight these issues and provide a guide to scholars who are new to conducting ecological research.

Ecological studies are characterized by the “group” level of analysis; however, there is great variation in the size of the group or geographic unit used as the units of analysis. The unit can be large (e.g., countries or states) or smaller (e.g., block groups or street segments), depending on the research question. Outcomes related to violence and crime rates that can be studied in ecological research include homicide, intimate partner violence, sexual assault, prosecution for specific violent crimes, and violence recidivism, among others. Ecological exposures, or treatments, are varied as well, including policies, programs, or other group-level factors that may influence violent outcomes, such as alcohol outlet density.

Ecological research allows for the investigation of group-level effects and can be inexpensive and relatively quick to conduct if the researcher uses existing data. And, importantly, ecological studies are an efficient means for hypothesis generation prior to costlier individual-level studies. However, ecological research also poses unique design and inferential challenges. This article will discuss reasons to conduct ecological research, guidelines for choosing the ecological unit of analysis, frequently used research designs, common limitations of ecological research, and issues to consider when using existing data (see Table 1).

Table 1. Factors to Consider in Ecological Research.

Domain	Factors to Consider
Reasons to conduct ecological-level research	The research question is at a population level The theory used is at a macro level To identify groups or geographic units at high risk for an outcome Feasibility (ecological-level data are available and inexpensive to access)
Ability to make inferences	To the ecological level at which the study is conducted Be aware of: Ecological fallacy Ecological bias
Choice of unit guided by	Theory Maximization of intra-unit homogeneity Data availability Statistical power Be aware of: Change of support problem
Existing data particulars	May be readily or publicly available May be inexpensive to access May make study quicker to conduct Be aware of: Whether data quality varies between units collecting it Between-unit differences in processes used to measure a construct that may be mistaken as differences in the construct Whether measurement methodology or instrument changed over time Whether coverage or representativeness of data source matches intended ecological unit

Reasons to Conduct Ecological-Level Research

The Research Question Is Population Level

The most basic reason to conduct an analytic ecological study is that the research question posed asks about group-level effects and the target level of inference for the study results is ecological. One may wish to investigate whether police staffing levels are related to changing homicide rates across cities or to the rate of police-reported intimate partner violence across law enforcement jurisdictions, for example. While the actions of individuals are aggregated to produce homicide or police-reported intimate partner violence rates, it is the rate at the group level that is of interest. Studies using cities or law enforcement jurisdictions, respectively, as the unit of analysis, then, are appropriate. Ecological studies may also be used to identify geographic areas or groups that are high risk for a particular outcome, and, therefore, in need of resources or other interventions (O’Campo et al., 1995). This use of ecological research may be especially appropriate when evaluating an area-level intervention or policy, or when seeking a practicable basis for allocating community-level resources.

Interventions are frequently put into effect at the population level and are intended to make ecological change. State laws are a good example of this, however, ecological interventions may also be intended to act in smaller geographic units, such as neighborhoods; see, for example, research on the Baltimore Safe Streets program (Webster, Whitehill, Vernick, & Curriero, 2012) or Project Safe Neighborhoods (McGarrell, Corsaro, Hipple, & Bynum, 2010). Relatedly, ecological interventions may be based on modifying the built environment, such as those aimed at restoring blighted areas (Branas et al., 2018), or greening interventions (Heinze et al., 2018). Ecological research can investigate the question of whether such interventions are associated with group-level change in the dependent variable, which is often crime rates in a geographic unit. It is important to note that ecological research does not provide evidence about individual-level mechanisms behind ecological outcomes. However, results of ecological research may be used to generate hypotheses at either the group or individual level (Pearce, 2000; Schwartz, 1994).

Theoretical Reasons

Beyond the need to test area- or group-level interventions, there are theoretical reasons to ask an ecological-level research question. In the field of public health, a socioecological model is widely used to explain the occurrence of violence (Bronfenbrenner, 1979; Dahlberg & Krug, 2002). This model posits that factors that influence the use of violence occur at multiple ecological levels—individual, relationship, community, and societal—and that factors at one level influence factors at other levels, and those levels interact to produce human behavior. Consequently, factors at the community or societal levels may influence individual behavior in ways that increase violence among groups of individuals encompassed by those levels. Ecological studies often focus on the outer levels of ecology (e.g., neighborhood or jurisdiction level, such as city, county, or state), but serve an important role in hypothesis generation at all levels.

Criminology also posits macro-level theories of crime, such as social disorganization theory. This theory suggests that social structural forces have an effect on institutions of informal social control, and this, in turn, affects aggregate violence rates. Specifically, violence, and crime in general, tend to concentrate in communities with certain structural characteristics, such as community-level low economic status, high residential mobility, and a high density of disrupted families (Sampson & Groves, 1989). Those community-level structural characteristics are theorized to disrupt local neighborhood organizations and institutions, resulting in a community structure that is unable to exert informal social control on its residents (Kornhauser, 1978). A test of this theory, therefore, could focus on the effect of social structural variables, such as family disruption and economic disadvantage, on neighborhood levels of violence.

Criminology also suggests that, similar to individuals, geographic units have their own routine activities and that some routines can increase the opportunities for crime (Brantingham & Brantingham, 1993). This is often hypothesized at the level of “micro” units of analyses, such as specific addresses, street segments, block, and block groups. For example, the presence of multiple liquor establishments on a block may increase the number of drunken brawls or other violent events on that block. As a result, ecological studies at this level would focus on answering questions related to the opportunity structures in the community, such as environmental design and presence of crime generators, and how they relate to violent crime rates. Taken as a whole, ecological theories suggest that many potential factors are influential in the occurrence of violence at group levels, which necessitates ecological research.

Feasibility

In addition to having a population-level research question, or a theoretical basis for an ecological study, an ecological research design may be the only viable research design. When individual-level data are unavailable or prohibitively expensive to collect, researchers may choose to conduct an ecological study with available group-level data despite originally having an interest in making individual-level inferences. When retrospectively studying an intervention conducted at a community level, it may not be feasible or possible to obtain relevant data on each individual exposed to the intervention. However, group-level data—such as mortality rates from violent causes, or crime rates within jurisdictions—may be obtainable, making an ecologic study feasible. Relatedly, when attempting to estimate the impact of a historical occurrence on violent outcomes, or otherwise conduct a retrospective analysis, the data needed for an individual-level study simply may not exist. Scenarios like this are exactly those where ecological studies are important for providing a missing evidence base or generating hypotheses for an individual-level effect. While results suggestive of a group-level impact do not provide evidence of an individual-level effect, these results may be used to further hypothesize an individual-level effect as the mechanism behind the group-level findings. In addition, results and implications of less expensive ecological studies may be used to justify the expense of a subsequent individual-level study.

Choice of Ecological Units

Decisions regarding the groups or other units to be compared in ecological research can influence findings and, therefore, must be considered carefully. In the context of spatial studies, the dependence of the unit of analysis on the final conclusions is termed the “change of support problem [COSP]” (Cressie, 1996; Gelfand, Zhu, & Carlin, 2001). In some cases, a relationship found at one unit of analysis is not present—or is even reversed—at another unit of analysis, a special case of which is called the “ecological fallacy,” where relationships between variables at the individual level are incorrectly deduced from relationships observed at the group level. Ideally, the choice of the grouping unit should be dictated by theory, data at the desired level of analysis are obtainable, and sufficient statistical power is attained. However, the choice of an ecological unit will not always be so straightforward: Theory may not be explicit in this regard, data aggregated at the level desired may not be obtainable, and/or statistical power may be low. Any or all of these challenges may be factors in the decision of what aggregate group or size of geographic unit to use. Issues and guidelines for choosing spatial or other groupings have been previously presented in the epidemiologic literature in relation to the study of disease (Arsenault, Michel, Berke, Ravel, & Gosselin, 2013; Susser, 1994). Many of the issues are similar in the study of violence outcomes. In this section, guidelines in choosing aggregate or geographic units are discussed.

First, the unit chosen should have theoretical justification. The researcher should match the unit chosen to the unit described in theory to the extent possible. However, theory may not unambiguously describe the group or spatial unit discussed. For example, an issue that plagues ecological research using social disorganization theory relates to whether the unit of analysis adequately captures the theoretical construct of a neighborhood. Social disorganization is meant to explain neighborhood-level processes, but the theory does not define the terms “neighborhood” or “community” (Sampson, 1993; Tienda, 1991). Researchers often struggle with either employing a social definition of neighborhoods and community or a spatial/geographic definition. Most empirical research on neighborhoods has focused primarily on using a spatial definition due to the existence of readily available data, such as the American Community Survey, compiled by the U.S. Census Bureau. However, some researchers argue that a neighborhood/community is not necessarily a geographic space (Sampson, 1993).

Relatedly, the unit chosen should be aligned with the level at which the exposure is expected to act (Arsenault et al., 2013). In practice, this means that a construct that theory suggests is influential at one level (e.g., neighborhood) may not be validly measured at a different level (e.g., state). Instead, it may represent a different construct at a different level of measurement even if it is aggregated from the same data at both levels. Accordingly, measurement of constructs important in individual-level theories may not be valid at the population level (Schwartz, 1994). For example, measurement of an individual’s income level and measurement of median income in the zip code in which the individual lives are proxy variables for two different constructs.

Second, researchers must consider the level of differences that exist on key factors within the geographic or group unit. Arsenault and colleagues (2013) refer to this as “intra-unit homogeneity.” There are many factors that may impact within-unit differences, one of which is unit size. The larger the population in a unit, the more likely it may be that individuals that comprise the population differ on important factors. Similarly, the larger a geographic unit is, the more resources may be unevenly distributed within that unit. For example, research suggests that domestic violence services are more common in counties with higher per capita incomes and in counties that contain universities or colleges (Tiefenthaler, Farmer, & Sambira, 2005). It may be hypothesized that the presence of domestic violence services in a community influences intimate partner homicide rates at an ecological level. Use of a larger unit of analysis in a test of this hypothesis, such as a state, would obscure the differences in access to domestic violence services that exist throughout the state in different cities and counties. This can result in aggregation bias, an issue encompassed by the COSP. Aggregation bias refers to the grouping of individuals, situations, or crimes as if they were homogeneous when, in reality, there are differences within the group (Hammond, 1973). Consequently, the relationship between two variables may be systematically different among different units of aggregation.

Third, data availability or obtainability for the desired ecological unit must be considered. Most commonly, the researcher must identify existing data sources for the study. The units used in those sources are often based on needs that are not research-related and may not capture what scholars are trying to model when utilizing them in their research. To illustrate, census tracts are often used as a proxy for communities in ecological research due to available data from the U.S. Census, but the Census does not contain data that focus on theoretically important variables to the explanation of outcomes of interest (e.g., violent behavior), such as neighborhood processes (Reiss, 1986). These types of limitations inherent to data not originally intended for research are often present at many ecological levels. These issues—which are discussed at length below—are a key limitation of most ecological studies.

Fourth, the group or geographic unit must have sufficient numbers of individuals (population size) to construct valid statistical comparisons (Arsenault et al., 2013). At a basic level, variability in the outcome variable is intimately related to statistical power, and the unit must be large enough to generate enough between-unit variability that it can be linked to covariates. When the event under study is rare, this may necessitate the use of larger geographic units, or potentially longer time units, to gain a greater frequency of the outcome. For example, intimate partner homicide is a rare event that researchers frequently study at the state level using annual data (e.g., Diez et al., 2017; Zeoli et al., 2018). Feasibility issues related to obtaining partner homicide rates at the neighborhood level aside, such a choice of unit would likely generate many zeroes, making it impossible to compare areas that may, in fact, have different risk levels, but their size and/or window of observations was insufficient to observe it. Dugan, Nagin, and Rosenfeld (2003), however, studied the impact of domestic violence resource availability on intimate partner homicide at the city level, disaggregating the grouped intimate partner homicide counts by race, gender, and marital status of the victim. As previously noted, the availability of domestic violence resources may differ greatly across different areas of a state, making cities a more appropriate unit of analysis. To overcome the lack of stability in city-level counts resulting from the rareness of intimate partner homicide among race, gender, and marital status groups, Dugan and colleagues (2003) used 3-year aggregated time units.

It is important to note that studies that have tested the differences between multiple sizes of geographic units of analyses in studies of crime and violence have found mixed results. Ouimet (2000) found no substantive difference in the macro-level predictors of crime between census tracts and social neighborhoods. Wooldredge (2002) also found that aggregation bias was nonexistent when comparing domestic violence rearrests in neighborhoods and census tracts. Sampson (2006) posits that the same spatial processes contribute to crime despite the level of aggregation:

empirical results have not varied much with the operationalization of unit of analysis. The place stratification of local communities by factors such as social class, race, and family status is a robust phenomenon that emerges at multiple levels of geography, whether local community areas, census tracts, political districts, and other neighborhood units.

(p. 35)

Research studies that have analyzed smaller spatial units with distinct methods have yielded differing results, however. Groff, Weisburd, and Morris’s (2009) analyses of the clustering of street segment hot spots in Seattle suggest that even though the high crime street segments cluster together for about 1.4 to 2.5 miles, the process affecting the street block clusters differed by block, thus pointing to the use of a smaller unit as a better approach.

At times, the choice of unit suggested by one guideline may differ from that of another guideline; a geographic unit small enough to maximize within-unit homogeneity may experience a rare outcome too infrequently for valid statistical comparisons, for example. While satisfying each criterion is not always possible, conscientious decisions—driven by previous research, theory, and logic—regarding which criteria to preference must be made. Decisions should be justifiable based on minimizing bias in the research. Above all, the researcher must be mindful of how the unit choice affects what group inferences may be made.

Ecological Research Designs

Ecological studies are frequently, but not always, observational. Researchers often study exposures or interventions that have been implemented by others (e.g., programs or laws) or that simply exist within the community (e.g., police staffing levels, alcohol outlet density) to generate a “natural experiment” instead of, for example, randomly assigned exposures or treatments. Rather than an exhaustive survey of ecological research designs, this section provides a brief introduction to a few commonly used designs, issues to consider in those designs, and examples of their use.

Ecological studies where the outcomes and predictors are measured simultaneously and at a single point in time are called cross-sectional ecological studies. As with cross-sectional studies in general, the temporal ordering of an association, which is a key aspect of causal attribution in epidemiological studies (Shadish, Cook, & Campbell, 2002), cannot be established. Relatedly, cross-sectional ecological studies do not permit separate estimation of between- versus within-jurisdiction effects. For example, if states with a particular policy are found to have different outcomes from states without that policy in a cross-sectional analysis, it cannot be empirically determined whether that difference is driven by within-state changes that occurred after the policy was implemented, or if states that adopted the policy were generally different from those that did not. This type of confounding generally relegates cross-sectional ecological studies to a role as an initial hypothesis-generating exercise that can be carried out with minimal resources, which, of course, can be important for allocating scarce research resources in costlier prospective or longitudinal studies.

Ecological research designs that include measurements of units of analysis over time, often referred to as time-series designs, can overcome some of the problems inherent to cross-sectional designs. A common approach to analyzing the natural crossover experiment represented by a new policy implementation is called the interrupted time series (ITS) design, in which the policy (or other intervention) is enacted or implemented at an identified time point in a geographic unit and is hypothesized to “interrupt” the time trend of the outcome variable. The inferential target in such studies is both how the trajectory shifts after the policy is implemented, and how the slope of the trajectory changes. For example, using data from 45 states from 1980 through 2013, Zeoli and colleagues (2018) considered whether state laws restricting individuals convicted of violent misdemeanor crimes from firearm access (among other laws) was associated with intimate partner homicide rates at the state level. An implicit assumption of the ITS design is that the pre-policy trajectory would have continued if the policy were never implemented; this argument can be strengthened by the use of comparable “control” jurisdictions (i.e., those that did not implement the policy), and control for other possible time-varying confounders.

An ITS design is a particularly strong ecological research design when the exposure occurs at different times (even including units where no exposure ever occurred) in different ecological units because it limits the possibility that a general secular trend confounds the results. State-level analyses of state policies or programs generally meet this standard because each state passes laws or implements programs on its own time. In general, for confounding to occur in this circumstance, changes in the trend of the outcome variable would have to be impacted by a third variable whose introduction or period of influence matched when the exposure under study was introduced in each state. If the possible confounder is measured, it can be statistically controlled; unmeasured confounding presents a more challenging problem but is partially addressed by the modeling of secular trends, and by having variability across units in the joint timing of the confounders and policy. This logic is exemplified in the concept of “coherent pattern matching,” which Shadish et al. (2002) explain as the decreasing likelihood that an alternative explanation is responsible for hypothesized associations found as the complexity of the patterns predicted in the data increase. Of course, this does not eliminate the possibility of confounding, and efforts can be made to determine whether any suspected confounders existed concurrently with the exposure. Indeed, the geographic and temporal specificity of an alleged confounder may aid in its identification (Ben-Shlomo, 2005). As in any research, known or suspected confounders should be controlled for to the extent possible.

Another advantage of time-series ecological studies is that they allow flexibility in modeling the timing of how long it may take for an intervention to have an effect, sometimes referred to as temporal lag effects. While a law may be enacted on a certain date, for example, implementation of this law may be more gradual, leading to a delayed impact. This was the issue that faced Humphreys, Esiner, and Wiebe (2013) when investigating whether the advent of a law that removed restraints on the times of day alcohol could be sold in England and Wales affected police-reported violent crime. To answer this question, they used a one-group ITS design, relying on weekly violent crime data in the City of Manchester over a period of 204 weeks, with the intervention occurring in week 95. A one-group time-series design has advantages (e.g., the researcher may be able to use data that are not available for other groups), but—as alluded to above—threats to internal validity (such as selection or history effects, or secular trends unrelated to the policy) cannot be ruled out. Because Humphreys and colleagues had conducted a process evaluation of the intervention before embarking on the outcome evaluation, they were aware that some alcohol establishments waited to change their business hours while the majority changed them on the first date allowed. They, therefore, hypothesized that the effects of the intervention could be immediate (in the 95th week) or delayed (in the 97th week) and tested for both.

When the analytic question is whether independent and dependent variables that exist throughout the study period co-vary and the variables are modeled at the same time points, temporal order between variations in the dependent and independent variables again become difficult to discern. For example, research has demonstrated a relationship between neighborhood economic disadvantage and elevated violent crime rates (Huebner, Martin, Moule, Pyrooz, & Decker, 2016), but it is unclear which occurs first or, if the relationship is cyclic, whether one condition is needed to start the cycle. The possibility of cyclic or bidirectional relationships between variables, often referred to as simultaneity (Shadish et al., 2002), further complicates statistical analyses and inferences that can be made. When there is no variability across units in the co-timing of the exposure and outcome, this is a general limitation, and the causal argument ultimately must be based on theory rather than empirical considerations. Time-lagged predictor effects used in conjunction with unit-level static fixed effects give a basis for estimating the within-unit association between the predictor (at a previous time point) and the outcome in a way that is un-confounded by static within-unit factors (Gunasekara, Richardson, Carter, & Blakely, 2013). When appropriate time-varying instrumental variables can be identified, their inclusion can also strengthen a directional causal argument (Angrist & Krueger, 2001). Other causal inference methods, such as matching (the simplest of which, in the case of a binary “treatment” such as a policy, is propensity score matching) or marginal structural models in the case of known time-varying confounders, can be used to pseudo-replicate an un-confounded design from observational data (Robins, Hernán, & Brumback, 2000; Rosenbaum & Rubin, 1983).

While often thought of as strictly observational, when the researcher can control the placement and implementation of the intervention under study, research conducted at the population level may also employ experimental designs. For example, Branas and colleagues (2018) conducted a citywide cluster randomized trial of the impact of restoration of blighted land on violent outcomes. From a complete sampling frame, they randomly selected blighted vacant lots in Philadelphia into the study, then randomly assigned them to the conditions of main restoration, any restoration, or no restoration. Random selection of lots helped ensure their sample was representative of the blighted vacant lots in the city, while random assignment theoretically minimized the differences between lots assigned to each condition. The researchers then reduced differences across the four sections of the city that might confound associations by matching sites within each section. To test their hypothesis that the restored lots would reduce violence in the surrounding area, the researchers created geographic units of analysis surrounding the selected vacant lots and aggregated police-reported crime to the level of the geographic unit-month over 38 months that spanned both pre- and post-intervention periods (Branas et al., 2018).

Challenges With Ecological Research

Despite their utility, ecological studies have been viewed as lesser study designs than individual-level studies (Ben-Shlomo, 2005) and are generally thought to supply less evidence than any type of individual-level study. This is mainly due to the design’s limitation known as ecological bias, which refers to the fact that associations seen between aggregated variables at the ecological level may not represent what is happening at the individual level (Morgenstern, 1995; Wakefield, 2008); incorrectly concluding individual-level effects from ecological correlations is generally referred to as the ecological fallacy. In addition, individual-level mechanisms that provide causal explanations for population-level associations cannot be investigated in population-level models (Cunradi, Mair, Ponicki, & Remer, 2011). Ecological bias can be conceptualized as an internal validity threat: Any relationship estimated between the ecological independent and dependent variables may arise due to unmeasured third variables (Schwartz, 1994).

The statistical phenomena known as Simpson’s Paradox arises from this kind of confounding that generates effects at one ecological level that are the reverse of effects at another level. The standard textbook example of Simpson’s Paradox comes from a study of gender bias in University of California (UC), Berkeley admissions (Bickel, Hammel, & O’Connell, 1975). Admissions data showed that males, in aggregate, had higher admission rates, but within six key departments comprising the total, women had higher admission rates; the unmeasured confounder was that women applied to more competitive departments—with lower overall admissions rates—at higher rates. Analogous dynamics can also explain the ecological fallacy and highlight the difficulty of causal inference from grouped data. Individual-level confounders are a concern in ecologic research because they may affect estimated group-level relationships (Saunders & Abel, 2014). While measures of possible confounders at the individual level are not included in purely ecological studies (though they may be for multi-level studies), and are often not available, units with greater homogeneity, as discussed in the section on choosing the ecological unit of analysis, may have a lower risk of individual-level confounders biasing ecological-level estimates (Arsenault et al., 2013; Susser, 1994).

Despite the broad potential for ecological fallacy, researchers may be tempted to use the results from ecological studies to make inferences at the individual level. In a study of 125 cross-sectional ecologic research papers published in journals in the discipline of epidemiology, Dufault and Klar (2011) found that only 27% clearly presented their level of inference as ecologic, whereas 34% of studies made inferences at the individual level, and 38% were ambiguous regarding the level to which results were inferred. As Schwartz (1994) highlights, if ecological research is being used as a substitute for individual-level research, ecological bias is a major limitation; if, however, ecological research is being conducted to examine group-level factors and associations, the inability to make valid individual-level inferences is not a large cause for concern. This limitation is a key reason that ecological research is generally only done when it is the only practicable basis for addressing a research question (e.g., when the research question is inherently group-level, as discussed above), and/or as a basis for supplying evidence to justify a costlier study.

Ecological research faces the same broad internal, external, construct, and statistical conclusion validity challenges as individual-level research. For an excellent discussion of these challenges, see the text by Shadish et al. (2002). However, there are some challenges that are unique to ecological research. One of these challenges is the possibility of heterogeneous exposure to the independent, or treatment, variable within groups or geographic units. In other words, not everyone who could be exposed to the treatment may be exposed. Consider the use of a state as the geographic unit under study and a new criminal justice intervention for intimate partner violence as the treatment variable. Within the state, it is possible that not everyone who would qualify for the intervention will experience the intervention. Application of the intervention to intimate partner violence cases may vary across governmental units within the state; for example, the intervention may be applied to a higher percentage of eligible cases in some counties than others. Thus, estimated intervention effects represent an inseparable mix between the actual intervention effect and factors such as policy implementation and outreach efforts, which are often unmeasured, generating unknown between-jurisdiction heterogeneity. Depending on the intervention, a smaller geographic unit than the state, such as a city or county, may produce a more homogeneous level of exposure to the treatment among the individuals within the unit.

Heterogeneity of exposure to the treatment based on individual-level factors may exist at any group level. If any sociodemographic factors are related to exposure to the treatment, moving to a smaller geographic unit will not eliminate this threat (though it may be reduced if the smaller geographic unit has a more sociodemographically homogeneous population). For example, extralegal factors, such as victim’s race, have been associated with whether prosecutors decided to charge a suspect with sexual assault (Spohn & Holleran, 2001). In temporal ecological studies, such confounders (measured or otherwise) can be effectively controlled using unit-level fixed effects as long as those factors do not change much over time—over a relatively short time frame, factors such as unit-level demographics plausibly fall into this category. In cross-sectional ecological studies, the researcher is advised to control for any available proxies for such factors (e.g., racial composition in the example above). As discussed, unmeasured and uncontrolled individual-level factors may confound estimated ecological-level relationships, just as any unmeasured or unknown population-level confounder may. If individual-level data are available, a multi-level model that considers the interaction between individual and ecological levels may be an appropriate analytic approach depending on the research question.

The possibility of differential implementation of a treatment, such as a community-level program or law, between units with the treatment (e.g., the different counties in the state referred to above) should also be taken into account. To the extent possible, data on implementation of the treatment for each ecological unit should be obtained. It may be possible to conduct process evaluations in multiple ecological units to determine implementation levels. This may be most easily accomplished when the intervention has not yet been put in place and the researcher can design a suitable data collection method for measurement of implementation; however, this is generally resource-intensive and, therefore, compromises a key advantage of ecological research (i.e., the comparatively low cost). If the researcher must retrospectively measure implementation, options for measurement are more limited or simply not available. Measuring implementation of an intervention in multiple ecological units also becomes more difficult (as available implementation data between units may not be comparable) and costlier as group units and time units under study are increased. It is, unfortunately, a limitation of most ecological studies that they do not include a measurement of implementation of the intervention under study because the magnitude of any association found may depend on breadth and thoroughness of implementation (Morgenstern, 1995).

Data Challenges

Data for geographic units is often more readily available than individual-level data. Ecological data may be available from governmental surveillance systems, although the availability, type, and quality of data may differ between governmental units within countries (e.g., states, provinces, counties, cities) and between countries. Because these data are rarely, if ever, recorded for research purposes, their use for research is always subject to associated limitations (Schwartz, 1994); a common example of these limitations is related to coding granularity in the data, and lack of standardized measurement across units. Whenever secondary data are to be used, researchers should familiarize themselves with the data collection instrument and methodology to determine the appropriateness of the data source for use in the research and to understand the limitations in the dataset. This section will briefly discuss population-level datasets and common limitations.

Surveillance systems often serve as sources of population-level data. These data are sometimes publicly available and are already aggregated or, if consisting of individual-level observations, de-identified. Ecological research is often, therefore, nonhuman subjects research. Existing governmental data may be readily downloadable from a repository or governmental website, or it may require agreements or permission to access. Data may also be obtained from organizations (governmental or otherwise), such as police departments, and aggregated into ecological units. For example, Cunradi and colleagues (2011) used publicly available data on alcohol retail licenses, obtained from the relevant licensing agency (the California Department of Alcohol Beverage Control), to create a measure of alcohol outlet density for a study testing whether outlet density is associated with neighborhood-level police calls and crime reports for intimate partner violence.

In addition, it must be recognized that the measurement method becomes part of the construct measured (Shadish et al., 2002). For example, when law enforcement records are used to aggregate crime, the construct necessarily includes the dimension of reporting to the police, which may be influenced by crime type and individual-level factors. The researcher must carefully consider whether the variable available in a secondary dataset or official record is similar enough to the desired construct to move ahead with the research, or if the construct is substantively different enough that it cannot provide a reasonable test of the hypothesis under study.

In addition, difficulties arise in ecologic studies when the measurement reveals differences in outcome levels between units that are due to the method of measurement and not the outcome itself. For example, a study of intimate partner violence recidivism after intervention (or control condition) in multiple police jurisdictions may use arrests for domestic violence as the outcome measure. If police jurisdictions differ in their arrest policies or arrest rates for intimate partner violence cases, between-jurisdiction variations in the outcome measure may reflect differences in arrest even if actual intimate partner violence recidivism rates are relatively similar across jurisdictions. Cross-sectional ecologic studies are particularly subject to this risk as there is no baseline level to establish between-unit differences prior to implementation of the intervention.

Ecological studies conducted over time are potentially limited by changes in the measurement apparatus during the study period, a problem known as instrumentation. More precisely, instrumentation refers to when the manner in which a measurement is taken changes over time, potentially resulting in differences in the outcome over time that could be incorrectly attributed to the exposure or treatment variable (Shadish et al., 2002). As a concrete example, Gorman, Labouvie, Speer, and Subaiya (1998) investigated the relationship between alcohol outlet density and police-reported domestic violence in New Jersey municipalities, using data on domestic violence rates from the New Jersey Uniform Crime Reports (which aggregate police-reported crime). During the course of the study, the researchers discovered that the domestic violence rate increased from 63.5 per 10,000 population in 1990 to 112.2 per 10,000 population in 1995. The researchers had to determine whether this almost doubling of the police-reported domestic violence rate reflected an actual increase in partner victimization, or if something else was responsible for the increase. After some investigating, the authors suggested changes during the study period in police training, investigating, and reporting procedures, which, combined with legislative changes that expanded the definition of domestic violence to include violence in dating relationships, may have been responsible for the substantially higher rate in 1995 (Gorman et al., 1998). In other words, the change in the domestic violence rate likely reflected the multiple differences in the pathway to reporting that occurred during the study period.

Additional issues to consider are whether the ecological units under study can be represented by the data source. For violent outcomes, consider whether the data source is designed to capture all events or a sample of events, and what ecological unit that sample might be representative of (if any). Challenges may be present concerning the geographic units of analysis that the datasets allow one to employ. In addition, theory may dictate that a hypothesis is made in a different type of geographical unit than those for which data are available (Dufault & Klar, 2011). For example, the National Crime Victimization Survey employs a complex sampling and weighting methodology designed to create a nationally representative survey of households in the United States (U.S. Bureau of Justice Statistics, U.S. Department of Justice, 2018), but it is not representative of smaller geographic units in the United States, such as states. The U.S. Federal Bureau of Investigation’s Supplementary Homicide Reports, in comparison, is designed to collect data on all homicide events in the United States (although nonreporting of homicides by jurisdictions is a well-known limitation; Fox & Swatt, 2009). Because the Supplementary Homicide Reports contain variables that identify the state, county, metropolitan statistical area, and police agency in which each homicide incident occurred, it may be used to study homicide at any of those levels or any additional groupings with those units (e.g., region). However, because the smallest geographic unit identifier is the police agency (which is often city-level), it cannot be used to study homicide at smaller geographic levels, such as zip code; for that, data on homicides may need to be directly collected from police jurisdictions.

While the choices of the researcher are often highly limited with regard to ecological studies—primarily because the scale and format of the data used are not under the control of the researcher—it is the researcher’s responsibility to understand the data limitations and temper their interpretations accordingly. For example, if using police-reported crime, one should consider whether some groups or some crimes are systematically less likely to report to, or be reported to, law enforcement. Or, the ideal unit of analysis may not be available to the researcher (e.g., the researcher wants to study neighborhood-level predictors of mortality but mortality reports are often not available below the county level). While many of these data limitations cannot be improved by the researcher, the researcher must be aware of them to determine whether the hypothesis can be adequately tested with the available data and to interpret any results with limitations firmly in mind.

Conclusion

Ecological research designs are a valuable tool for studying macro-level processes that may influence crime and violence. These studies may be relatively inexpensive and quick to conduct, facilitated by the use of secondary data. There are numerous challenges to conducting ecological research, some of which are unique to group-level research. However, with careful consideration of choice of ecological units, data availability and appropriateness, and research design, researchers can conduct a methodologically justified study that will advance the literature and provide needed evidence to justify further, more resource-intensive studies.

Biographies

April M. Zeoli is an associate professor in the School of Criminal Justice at Michigan State University. She conducts interdisciplinary research, with a goal of bringing together the fields of public health and criminology and criminal justice. Her main field of investigation is the prevention of intimate partner violence and homicide through the use of policy and law.

Jennifer K. Paruk is a doctoral student in the School of Criminal Justice at Michigan State University with a University Distinguished Fellowship. She holds a Master of Public Health degree from Boston University, and her current research interests include public health and criminal justice policies related to violence prevention.

Jesenia M. Pizarro is an associate professor in the School of Criminology and Criminal Justice at Arizona State University. Her research focuses on homicide and corrections policy. Her work has appeared in the American Journal of Public Health, Justice Quarterly, Criminal Justice and Behavior, and Homicide Studies.

Jason Goldstick is a research assistant professor in the University of Michigan Department of Emergency Medicine and the director of Statistics and Methods within the University of Michigan Injury Prevention Center. His substantive research focuses broadly on injury prevention, with particular emphasis on studies of substance use and violence.