Expanding Tools for Investigating Neighborhood Indicators of Drug Use and Violence: Validation of the NIfETy for Virtual Street Observation

Abstract

A growing body of evidence suggests that characteristics of the neighborhood environment in urban areas significantly impact risk for drug use behavior and exposure to violent crime. Identifying areas of community need, prioritizing planning projects, and developing strategies for community improvement require inexpensive, easy to use, evidence-based tools to assess neighborhood disorder that can be used for a variety of research, urban planning, and community needs with an environmental justice frame. This study describes validation of the Neighborhood Inventory for Environmental Typology (NIfETy), a neighborhood environmental observational assessment tool designed to assess characteristics of the neighborhood environment related to violence, alcohol, and other drugs, for use with Google Street View (GSV). GSV data collection took place on a random sample of 350 blocks located throughout Baltimore City, Maryland, which had previously been assessed through in-person data collection. Inter-rater reliability metrics were strong for the majority of items (ICC≥0.7), and items were highly correlated with in-person observations (r≥0.6). Exploratory factor analysis and constrained factor analysis resulted in one, 14-item disorder scale with high internal consistency (alpha=0.825) and acceptable fit indices (CFI=0.982; RMSEA=0.051). We further validated this disorder scale against locations of violent crimes, and we found that disorder score was significantly and positively associated with neighborhood crime (IRR=1.221, 95% CI=(1.157, 1.288), p < 0.001). The NIfETy provides a valid, economical, and efficient tool for assessing modifiable neighborhood risk factors for drug use and violence prevention that can be employed for a variety of research, urban planning, and community needs.

INTRODUCTION

A growing body of evidence suggests that characteristics of the neighborhood environment in urban areas significantly impact risk for drug use behavior and exposure to violent crime. Mental health, as well as associated drug use, is directly and significantly impacted by the neighborhood built environment and neighborhood disorder (Diez-Roux and Mair 2010; Furr-Holden et al. 2015; Mair et al. 2008). Physical disorder is the deterioration of the urban landscape—including graffiti, litter, and vacant lots—while social disorder indicates behavior which may be considered threatening, such as verbal harassment on the street or public intoxication (Sampson and Raudenbush 1999). Residents who report high disorder in their neighborhoods experience more depression, fearful anxiety, and signs of autonomic arousal than do those who report fewer neighborhood problems (Daniel et al. 2008; Hill et al. 2005; South et al. 2018). Neighborhood disorder undermines mental health as disordered neighborhoods exhibit fewer opportunities for social interaction and group involvement, increasing social isolation (Baum and Palmer 2002; Kim 2008; Wood et al. 2008). Previous research has shown that neighborhood disorder directly impacts risk for use of injection drugs (Latkin et al. 2005), crack-cocaine (Latkin et al. 2007), and marijuana (Furr-Holden, Lee, et al. 2011), as well as associated harms such as risk of fatal drug overdose (Hembree et al. 2005). At the same time, structural dilapidation and blight in urban neighborhoods contribute to negative outcomes for resident’s safety. The neighborhood environment may impact risk for community violence by shielding illegal activity and illegal guns (Branas et al. 2016), possibly increasing gun violence and crime (Branas et al. 2018).

Many neighborhood disorder risk factors for drug use and violence may be modifiable through targeted infrastructure improvements or other community greening schemes. For example, vacant lot remediation notably reduces gun violence (Branas et al. 2016, 2018) and improves residents’ mental health (South et al. 2018). Furthermore, these targeted infrastructure improvements are relatively cost effective and quick to implement (Branas et al. 2018; Sallis et al. 2011). Identifying areas of community need, prioritizing planning projects, and developing strategies for community improvement require inexpensive, easy to use, evidence-based tools to assess neighborhood disorder that can be used for a variety of research, urban planning, and community needs with an environmental justice frame. This study describes validation of the Neighborhood Inventory for Environmental Typology (NIfETy) (Furr-Holden et al. 2008), a neighborhood environmental observational assessment tool designed to assess characteristics of the neighborhood environment related to violence, alcohol, and other drugs, for use with Google Street View (GSV).

Neighborhood environmental audits are a form of Systematic Social Observation, a standardized method for directed observation of the physical, social, and economic characteristics of neighborhoods (Sampson and Raudenbush 1999). Trained researchers record indicators of neighborhood characteristics using a standardized assessment tool and following a strict data collection protocol. However, these neighborhood assessments can be time-consuming, expensive, and limited in their geographic scope (Bader et al. 2015; Clarke et al. 2010; Rundle et al. 2011). Because of the financial and temporal limitations inherent to in-person data collection, community organizations, policy makers, or other groups with limited resources may not be able to use these important tools effectively. GSV provides an alternative to in-person observation. GSV is a free tool offering panoramic, street-level images of city streets across the world. The user types in an address and can virtually “walk” forward or backward along a street, revolve 360 degrees, rotate vertically 290 degrees, and zoom in and out (Clarke et al. 2010; Rundle et al. 2011). Performing street audits with GSV allows for a large amount of data collection in a shorter period of time. One study of 850 intersections estimated that using GSV in place of in-person audits cut down data collection time from three person-years to one person-month (Koepsell et al. 2002; Mooney et al. 2016).

Despite the benefits and widespread use of GSV as a data collection tool, environmental observation tools previously validated for in-person use should not be immediately translated for use in GSV without further investigation. Because of the lack of fine detail available in images, subjective measures such as sidewalk maintenance may not be reliable (Griew et al. 2013; Vanwolleghem et al. 2014). Other indicators of neighborhood disorder are also difficult to capture, such as noise or social interactions. Obstruction also could be problematic as signage, particularly smaller signs such as speed limits or bus stops, as well as driveways and alley streets, may be blocked by trees, trucks, or other vehicles when the image was captured (Bader et al. 2015). GSV images are also not updated consistently across neighborhoods (Curtis et al. 2013). One GSV validation study found a third of intersection images were taken within one month of in-person observation and less than 30% were taken within one month for main roadways (Nesoff, Milam, Pollack, Curriero, Bowie, Gielen, et al. 2018). This temporal delay in images could reduce the reliability of GSV measures compared to in-person observation. For these reasons, the validation of environmental observation tools in GSV is necessary before these tools are applied to research questions.

The NIfETy is a valid and reliable, standardized inventory (Milam, Furr-Holden, Cooley-Strickland, et al. 2014). It has been widely used in previous studies to examine the impact of neighborhood characteristics on a variety of topics, including drug use risk factors (Furr-Holden, Lee, et al. 2011; Milam, Furr-Holden, Harrell, et al. 2014), exposure to community violence (Furr-Holden, Milam, et al. 2011; Rossen et al. 2011), and other injury outcomes (Nesoff, Milam, Pollack, Curriero, Bowie, Knowlton, et al. 2018). In-depth discussion of the conceptualization of the NIfETy and its unique contribution to the field of systematic social observation can be found in Furr-Holden et al. (2008) and Milam et al. (2016). By validating the NIfETy for use in GSV, we aim to provide a valid, economical, and efficient tool for assessing modifiable neighborhood risk factors for drug use and violence prevention that can be employed for a variety of research, urban planning, and community needs.

METHODS

Measures

The NIfETy instrument includes over 160 items operationalized into seven domains: physical layout; types of structures; adult activity; youth activity; physical order and disorder; social order and disorder; and violence, alcohol, and other drug exposures (VAOD) (Furr-Holden et al. 2008). The NIfETy has strong psychometric properties; the ICC for the total instrument is 0.84, 0.71 for the VAOD subdomain, and 0.67 to 0.79 across raters (Furr-Holden et al. 2010). Validity metrics are also strong (Furr-Holden et al. 2010). The NIfETY has also been validated for nighttime assessment of neighborhood disorder (Milam et al. 2016). In addition, violent crime data and self-reported assessments from adolescents have also been used for NIfETY validation (Milam et al. 2016).

Before we assessed the study blocks, we trialed the instrument on non-study blocks. Given the resolution available in GSV, the NIfETy instrument was slightly modified. VAOD items were removed because these items could not be observed through GSV (e.g. blunt wrappers on the street). Items that required specific time intervals (e.g. cars passing within 3 minutes) and items assessing noise levels were also removed. A total of 76 items from the NIfETy were evaluated in the GSV assessment.

Data Collection

Data collection took place on a random sample of 350 blocks located throughout the city of Baltimore, Maryland (Figure 1). These blocks were randomly sampled from Baltimore City Neighborhood Statistical Areas (NSA) (n=183) for a cluster average of 1.91 blocks per NSA (see Furr-Holden (2008, 2010) for more information on block face sampling). These 350 blocks, which had previously been assessed through in-person data collection (during the summer of 2012), were reassessed using GSV by a new set of raters. These raters received the same training as in-person raters but had not participated in in-person data collection. Data collection took place from May to July 2016. Locations were double coded by two raters who were instructed not to discuss or share their assessments. Blocks were coded using tablet devices programmed with Pendragon Software.

Figure 1.

Locations of randomly-selected street blocks for Google Street View data collection in Baltimore City (n=350)

Raters were instructed to type the address of the selected block face into GSV and scan the area as many times as necessary and from as many angles as necessary to thoroughly assess the block (Figure 2). Raters were asked to note the month and year the image was captured for each block to assess temporal stability of the images. Because different blocks are recorded by GSV at different times of year and with different frequency, images may “skip” through time when a rater moves along a block.(Curtis et al. 2013) When a GSV image “skipped” through time, raters were instructed to judge the block by the most recent image. Each assessment took approximately 15 minutes to complete. When disagreement occurred between two raters, we used the higher value; for example, if one rater said they observed 4-7 bags of trash and another said 1-3, we recorded 4-7 for the block. For binary items, if one person saw the item on a given block, the item was recorded as present; we used a similar procedure when aggregating count variables to binary items.

Figure 2.

Examples of data collection for NIfETy items using Google Street View

Example NIfETy measures:

A: Liquor store (domain: types of structures)

B: Broken windows (domain: physical disorder)

C: Trash in street (domain: physical disorder)

D: Loitering (domain: social disorder)

E: Adults sitting on steps (domain: adult activity)

Statistical Analysis

Inter-rater reliability was assessed for each GSV observation pair using intra-class correlation coefficients (ICC). Average rater reliability is reported for the two-way mixed effects ICC model from Shrout and Fleiss(1979) with the consistency agreement definition. Paired ratings for each of the block faces were given binary identifiers to assess degree of agreement, independent of individual rater identity or characteristics. ICC estimates ranging from 0 to 0.2 were classified as poor, 0.2 to 0.4 as fair, 0.4 to 0.6 as moderate, 0.6 to 0.8 as substantial, and estimates between 0.8 and 1.0 as almost perfect (Landis and Koch 1977).

Internal consistency reliability was measured using Cronbach’s alpha for the six operational subdomains and all items combined. Cronbach’s alpha ranges from 0 to 1, with higher values indicating a more reliable subdomain scale (Cortina 1993). Nineteen items with low prevalence (<5%) were removed: There were a total of 26 items with low prevalence; however, 7 were considered important and were kept for further analysis. These 7 items have been shown in the literature to have an important impact on neighborhood disorder and have been used in previous studies of neighborhood disorder using the NIfETy (e.g. count of liquor stores, count of bars, presence of memorials) (Milam, Furr-Holden, Cooley-Strickland, et al. 2014; Rossen et al. 2011). Due to the low prevalence of many items, all the items were dichotomized (presence versus absence of item) for the analyses. Five items with negative inter-item correlation were also removed.

Exploratory factor analysis (EFA) with principal component extraction and varimax rotation was employed to develop scales using MPlus 8.3. The purpose of EFA was to identify clusters of homogenous variables across all NIfETy items that could be used to assess neighborhood disorder without having to collect the entire data form. Eigenvalues of greater than 1 were used as criterion for factor extraction; items with loadings of less than 0.15 and double-loaded items were dropped. A Cronbach’s alpha of 0.6 or greater was accepted as a measure of internal consistency for each scale identified through EFA (Cortina 1993). Constrained factor analysis (CFA) was used to identify variables with the highest factor loadings for the most representative constructs, which were determined by thresholds (factor loading >0.4). Several model-fit indices, including the Root Mean Square Error of Approximation (RMSEA) and Comparative Fit Index (CFI), were used to evaluate model-fit. RMSEA values ≤ 0.05, CFI values ≥ 0.95, and TLI values ≥ 0.90 generally represent an excellent fit to the observed data (Marsh et al. 2009).

External validity of the GSV items were assessed using the in-person NIfETy data as well as violent crime data. Each GSV item was compared to in-person observation using tetrachoric correlation coefficient (Milam et al. 2016). We also validated the resulting disorder scale against crime data for 2016 obtained from the Baltimore City Police Department. We geocoded crimes, including aggravated assaults, rapes, homicide/manslaughters, shootings, and robberies, using Worlds Geocoding Service (n=12,017). We used a threshold on 90% or better for a match score. A total of 98% of crime locations were matched and 2% were tied. Approximately 200 crimes records did not include specific addresses; these all clustered around War Memorial Plaza, a public park in downtown Baltimore City, and were geocoded with this address. We created 500-ft network buffers (the approximate length of a residential block in Baltimore City)(Rossen et al. 2011; Salbach et al. 2015) around the NIfETy blocks using straight-line buffers in ArcGIS. The count of violent crimes within the buffer was determined using the spatial join tool (a tool used to append data from one map layer to another map layer using geographic location). We examined the relationship between disorder score and count of violent crimes within each buffer using negative binominal regression models in Stata 13.1. We used robust standard errors, adjusting for clustering by neighborhood (Neighborhood Statistical Areas)(Furr-Holden et al. 2008), and used the area of the buffer as an offset variable. Incident rate ratios (IRR) were calculated to convey the strength of association.

RESULTS

Inter-rater reliability estimates for GSV and in-person ratings are presented in online supplemental Table S1. Overall, GSV ratings showed strong reliability. Items in the physical layout domain and the type of structure domain showed the highest reliability across GSV raters, with the majority of items with ICC≥0.7. Items in the physical disorder, social disorder, and adult activity domains showed moderate agreement, with a majority of ICC≥0.5. Youth activity items showed the lowest inter-rater reliability, with most ICC≤0.4. Seventeen items had insufficient data to determine correlation between GSV and in-person observations.

The total instrument included 52 items and had a high internal consistency, with alpha of 0.781. Physical disorder included 27 items and also had a high internal consistency, with alpha of 0.766. Social disorder included 6 items and had a moderately low internal consistency with alpha of 0.642; internal consistency improved with the removal of loitering (alpha=0.757). The remaining domains had low internal consistency. Physical layout included 4 items and had an alpha of 0.257, types of structures had 9 items and alpha of 0.383, and adult activity had 9 items with alpha of 0.461.

Validity metrics (supplemental Table S1) against in-person items were strong for physical layout domain and the type of structure domain, with most items demonstrating significantly positive correlations (r≥0.8). Correlations with in-person observation for physical disorder, social disorder, and adult activity domains were moderate (r≥0.5). Seventeen items had insufficient data to determine correlation between GSV and in-person observations. We also examined the correlation between the in-person and GSV disorder scores, there was a strong correlation between the scales (r=0.685).

The EFA yielded one 14-item disorder scale with high internal consistency (alpha=0.825) using binary variables (Table 1): broken windows, boarded abandoned buildings, unboarded abandoned buildings, vacant houses, unmaintained property, trash in street, trash in alley, trash in other open spaces, vacant lots, damaged sidewalks, street cleaning, people loitering, memorials, and adults sitting on steps. Higher scores (range 0-14) indicate higher levels of disorder on a block. The CFA model had the same 14 items with acceptable fit indices: CFI of 0.982 and RMSEA of 0.051.

Table 1.

Prevalence of disorder scale items and fit statistics for exploratory factor analysis (EFA) and constrained factor analysis (CFA)

Item (binary)		EFA Fit Statistics	CFA Fit Statistics
	Prevalence across sampled blocks (n=350) n (%)	Cronbach’s alpha	Comparative Fit Index (CFI)	Root Mean Square Error of Approximation (RMSEA)
Broken Windows		0.825	0.982	0.051
Yes	58 (16.6)
Boarded Abandoned Buildings
Yes	97 (27.7)
Unboarded Abandoned Buildings
Yes	9 (2.6)
Vacant Houses
Yes	50 (14.3)
Unmaintained Property
Yes	186 (53.1)
Trash in Street
Yes	194 (55.4)
Trash in Alley
Yes	52 (14.9)
Trash in Other Open Spaces	193 (55.1)
Yes
Vacant Lots
Yes	63 (18.0)
Damaged Sidewalks
Yes	263 (75.1)
Days of the Week Posted for Street Cleaning
Yes	80 (22.9)
People Loitering
Yes	33 (9.4)
Memorials
Yes	4 (1.1)
Adults Sitting on Steps
Yes	103 (29.4)

Negative binomial regression showed a positive and significant relationship between the disorder scale and violent crime. For each unit increase in the disorder score, the rate of violent crime within 500 feet of the block increased by 22.1% (IRR=1.221, 95% CI=(1.157, 1.288), p < 0.001).

We also recorded the date of GSV images to account for disagreement related to temporal variability of images across locations(Curtis et al. 2013). One-quarter of images (n=85) were recorded August to September 2012 or roughly within one month of in-person observation. One-third (n=111) of images were taken a year or more prior to in-person data collection (September 2011 or earlier), and 44.0% (n=154) of images were captured a year or more after in-person data collection (August 2013 or later). In addition, 15.1% (n=53) of images had more than one image capture date recorded—meaning, the GSV image was not consistent for the entire block.

DISCUSSION

This study provides support for a valid and reliable method to assess neighborhood disorder risk factors for violence and drug use using virtual data collection through GSV. Inter-rater reliability metrics were strong for the majority of items (ICC≥0.7), and items were highly correlated with in-person observations (r≥0.6). EFA and CFA yielded one neighborhood disorder scale with high internal consistency. The strong inter-item reliability of the disorder scale is a strength of the NIfETy for use in GSV, especially considering the relevance of disorder to the risk for community violence and drug use (Furr-Holden, Lee, et al. 2011; Furr-Holden, Milam, et al. 2011; Milam, Furr-Holden, Harrell, et al. 2014; Rossen et al. 2011). We further validated this disorder scale against locations of violent crimes, and we found that disorder score was significantly and positively associated with neighborhood crime.

The NIfETy is a previously-validated tool that has been widely used to examine the impact of neighborhood disorder on drug use (Furr-Holden, Lee, et al. 2011; Milam, Furr-Holden, Harrell, et al. 2014) and violence (Furr-Holden, Milam, et al. 2011; Rossen et al. 2011). Previous studies of environmental observation tools for neighborhood disorder have evaluated their tool’s reliability for GSV use without first testing the reliability of the tool itself (Bader et al. 2015; Mooney et al. 2014). Therefore, this study extends the utility of the NIfETy to community organizations, policy makers, or other groups who, because of the financial and temporal limitations inherent to in-person data collection, may have been unable to use it previously. For example, the NIfETy can be used to assess specific aspects of neighborhood disorder such as vacant lots or abandoned buildings to locate community features in need of remediation. It can also be used to assess the prevalence of specific community features throughout a neighborhood, such as the prevalence of vacant lots in areas of high versus low violent crime. In addition, the disorder scale can be used to assess overall neighborhood disorder without having to collect all NIfETy measures, saving time and resources when rapid data collection is necessary for pressing public health and health policy issues. An example application of the NIfETy to drug policy can be found in Furr-Holden et al. (2016).

As with previous evaluations of observational tools to assess physical disorder (Bader et al. 2015; Mooney et al. 2014; Odgers et al. 2012), we found that the use of GSV provides a reliable alternative to in-person street audits. GSV is a low-cost, easy-to-implement alternative to in-person audits that produces relatively quick turn around on data collection (Mooney et al. 2016; Rundle et al. 2011). Consequently, GSV allows for a wider area to be surveyed compared to in-person audits, without the need for additional resources or time (Clarke et al. 2010). GSV also provides a safer alternative to in-person assessments as raters can conduct audits remotely.

Several limitations of this research merit discussion. We were unable to collect time-varying measures and other qualitative measures such as those related to noise levels or smells. This may limit the NIfETy’s ability to capture the actual, lived social behavior and climate of the block. As noted in the Introduction, GSV as an observational tool has several limitations, and these may contribute to decreased reliability for certain measures. Only a quarter of GSV images were captured during the same time period as in-person data collection; it is possible that some of the discrepancies between in-person data collection and GSV data collection stem from temporal variations in when images were captured (Curtis et al. 2013). Because of the lack of fine detail available in images, we were unable to collect VAOD domain measures. We attempted to account for these missing items by validating the disorder scale against crime incidence near blocks where data collection occurred. Our results support that, even though VAOD measures cannot be collected using GSV at this time, the disorder scale is a valid tool for assessing neighborhood risk factors for violence exposure. We did not have access to data on drug exposure, but future research should validate the disorder scale against other, valid measures of neighborhood drug use. Finally, our predictive validity estimates were based on crime counts within a 500-foot straight-line buffer around the centerline segment of the block (i.e., an imaginary line that goes through the center of the block face and includes the even and odd sides of the street). It is possible that there was a slight overestimation of violence within these buffers as the counts were not specific to the sample block alone and potentially picked up crimes from neighboring block faces. Previous studies have shown that the 500-foot straight-line buffer produces results comparable to a walking distance buffer (i.e., a quarter mile or 1,320 feet); furthermore, the smaller and more block-specific 500-foot straight line buffer has been shown to minimize potential overestimation and is conceptually a more sound approach (Milam et al. 2016). While robust standards errors were used, we were unable to perform multilevel modeling because we only had a cluster average of 1.91 blocks per NSA. Future studies should examine the relationship between the NIfETy disorder scale and violence in a multi-level context and determine if the relationship is moderated by neighborhood characteristics.

Conclusion

In this study, we have identified a valid and reliable virtual assessment tool that can characterize the built and social neighborhood environment for risk of violence, alcohol, and other drug problems using GSV. The NIfETy provides a valid, economical, and efficient tool for assessing modifiable neighborhood risk factors for drug use and violence prevention that can be employed for a variety of research, urban planning, and community needs.