The NIfETy Method for Environmental Assessment of Neighborhood-level Indicators of Violence, Alcohol, and Other Drug Exposure

Abstract

There are limited validated quantitative assessment methods to measure features of the built and social environment that might form the basis for environmental preventive interventions. This study describes a model approach for epidemiologic assessment of suspected environmental determinants of violence, alcohol and other drug (VAOD) exposure and fills this gap in current research. The investigation sought to test the feasibility of a systematic and longitudinal assessment of residential block characteristics related to physical and social disorder and indicators of VAOD exposure. Planometric data were used to establish a stratified random sample of street segments within defined neighborhoods of an urban metropolitan area. Field rater assessments of these neighborhood street segments were conducted using the Neighborhood Inventory for Environmental Typology (NIfETy). This report provides a detailed description of the NIfETy Method, including metric properties of the NIfETy Instrument and outcomes of training procedures and quality control measures. Also presented are block-level characteristics and estimates of observable signs of VAOD activity. This work is a first step toward developing future community-level environmental preventive interventions geared to reduce community VAOD exposure among youthful urban populations and may prove to be useful to other public health research groups as well.

Introduction

This investigation sought to develop a systematic, city-wide assessment of residential neighborhood environmental characteristics of violence, alcohol, and other drug (VAOD) exposure. The approach, termed the NIfETy Method, includes a comprehensive assessment tool (i.e., the NIfETy Instrument), training protocol, and quality measures employed simultaneously to identify potentially malleable characteristics of the built and social environment. The utility of this work is threefold. First, the NIfETy Method provides a baseline assessment of the physical and social environment. Second, when combined with youth data on VAOD exposure, the NIfETy Method provides an indication of features of the environment that, if modified, might produce changes in youth exposure. Third, the NIfETy Method serves as an evaluation tool to assess the impact of environmental interventions within communities and might also be sensitive to environmental changes linked to non-environmental interventions.

Despite the growing body of evidence linking deleterious environments to disease and poor health outcomes, there has been little focus on the environment when considering VAOD prevention strategies (Pollack et al. 2005; Romley et al. 2007; Scribner et al. 2000). Instead, this major public health issue has traditionally been addressed within the criminal justice system. The limited body of work in this area has been conducted by criminologists, epidemiologists, psychologists, and policy analysts. Much of their work has been guided by theoretical frameworks that aid in understanding the role of the environment in the occurrence of crime and incivility and, more specifically, youth VAOD exposure (Mair and Mair 2003). What is missing is a systematic approach to quantify features of the community environment; i.e., features of the built and social environment, that, if modified, might signal a decrease in VAOD-related behavior, exposure, and subsequent health outcomes. In the next section we will look at the limited body of published work in this area.

Environmental Studies of Inner-City Neighborhoods and VAOD Exposure

A 20-year study of the decline of Baltimore City neighborhoods (Taylor 2001) was the first study of its kind to use longitudinal quantitative and qualitative methods to classify environmental indicators. Taylor's 1981 study included 66 randomly selected neighborhoods in Baltimore city and examined reactions to crime and decline. Approximately 25 residents from each of the neighborhoods were interviewed on perceived incivilities, including physical problems (e.g. vandalism, vacant houses, vacant lots) and social problems (e.g., insults, noise) (Taylor 2001). Investigators also assessed conditions of street blocks in 30 neighborhoods in Baltimore City. This work suggests that the street unit block is a key mediating structure linking changes in the physical neighborhood with responses of individuals and small groups of residents, although the mechanism by which these links occur was unclear (Taylor 1997). More recently, Taylor (2001) purports that reducing or eliminating signs of disorder (e.g., dilapidated buildings, uncivil behavior among youth) are useful as a partial solution to reducing crime. What was lacking from Taylor's work was a replicable approach that would allow systematic assessment of neighborhood decline and decay.

Taylor's work informed several recent studies aimed at assessing neighborhood context to inform child and family heath promotion. Caughy et al. (2001) thoroughly reviewed the strengths and limitations of existing neighborhood studies, and introduced a brief instrument designed to measure urban neighborhood characteristics that built upon the strengths of the previously conducted studies. The result was an objective instrument that provided a comprehensive measure of physical disorder, but lacked any measure of social disorder. James McDonell (2007) used an observational measure of neighborhood characteristics to examine parent reports of the safety of neighborhood children. McDonell's study did underscore the need for a method of neighborhood observation aimed at gauging neighborhood characteristics that may impact child safety, but the neighborhoods used were sampled by convenience and neighborhood experiences were not validated by youth self-report. Neither of these studies was longitudinal in design but they do form the basis for work in this field of research.

Perhaps the most comprehensive work in this area to date were conducted by Sampson and Raudenbush (Sampson and Raudenbush 1999; Raudenbush and Sampson 1999; Raudenbush et al. 2003; Sampson and Raudenbush 2005), who conducted systematic social observations of public spaces on more than 23,000 street segments in Chicago. Their environmental measures were both quantitative and qualitative in nature. They found that neighborhoods with high levels of responsibility and trust had low levels of violent crime (Sampson et al. 1997). However, there may be cause and effect operating in both directions. The authors did not control for prior neighborhood violence which might impact neighborhood responsibility, furthering neighborhood decay. As was the case with their observational studies, relationships were identified but causality could not be determined. The observational studies were conducted on a large number of residential blocks and entailed an extensive list of observational measures. While there was much breadth to the study, the depth of the investigation was insufficient to establish casual relationships and identify possible features of the environment that, if modified, would impact residential exposure to violence.

These aforementioned theoretical frameworks and earlier environmental studies of city neighborhoods form the underpinnings for our present work. The forthcoming methods describe an epidemiological approach to evaluate characteristics of residential neighborhoods that might signal an increase in VAOD exposure, crime, and victimization for neighborhood residents. This work fills a gap in the current literature in that the methods are quantifiable, replicable, and designed to be longitudinal so one can begin to assess the casual relationship between environmental characteristics and individual and group behavior.

Methods

Sample Selection

Baltimore Residential Neighborhoods

Planometric data obtained from the Baltimore City Mayor's Office of Information and Technology contained 272 stratification units classified as Neighborhood Statistical Areas (hereafter referred to as neighborhoods). Neighborhood boundaries were based on 2000 Census Data, City Planning and Services Data, and local resident input gathered at community organization meetings. Neighborhoods were almost always clusters of census blocks and census block groups, but sometimes used natural boundaries (e.g., water-body, highway) instead of census boundaries.

The 272 Baltimore neighborhoods were stratified into residential and nonresidential neighborhoods and all nonresidential neighborhoods were eliminated, including 26 with fewer than 100 residents (primarily industrial areas, many of which had zero to five residents). Another four neighborhoods were excluded because they represented boundaries of a university or medical campus (whose primary residents are dormitory residents or patients in residential health facilities).

Random Census-Based Block Face Selection

After eliminating non-residential neighborhoods, a random sample of census blocks within neighborhoods was selected. Within the census block, a block face (comparable to a unit block) was randomly selected. The number of block faces selected within each neighborhood was a function of the number of census blocks within the neighborhood. One block face was sampled for neighborhoods with 1–10 populated census blocks; two block faces for neighborhoods with 11–100 populated census blocks; and three block faces for neighborhoods with more than 100 populated census blocks. This method of weighting the neighborhoods by rating one to three block faces depending on the number of populated census blocks in the neighborhood resulted in 446 block faces to represent the 242 residential neighborhoods. This initial approach restricted the number of sample block faces in very large neighborhoods (e.g., those with more than one hundred block faces) and was employed primarily to conserve costs as the amount of resources needed to establish the baseline sample at the time was unclear. More than 80% of the neighborhoods had 40 or less census blocks, while only 8 of the 242 residential neighborhoods had more than 100 census blocks but none had more than 171 census blocks. We subsequently devised an approach to balance this undersampling which included sampling 1 block face for every 20 census blocks in each residential neighborhood. Future reports will be based on these data.

Starting with a list of populated census blocks for each neighborhood, each populated census block face received a computer-generated random number. These numbers were then ranked from lowest to highest. The three census blocks with the lowest random numbers were selected as a pool from which to draw specific block faces for this study.

To randomly select one block face (i.e., a street segment where both sides have a combined potential address range of 1–100 units; for example, 1–99 Main Street, 400–499 Main Street, or 1200–1299 Main Street), we used the city's planometric dataset which contains unique numeric identifiers for every street centerline. These centerlines identify a segment of a street that extends between two intersectors, such as another street or an alley. Geocoded street centerline datasets are used by governments, utility companies, and others for identification of jurisdictional boundaries and street addresses, land use and development, road management, etc. ArcGIS software was used to extract street centerlines by neighborhood from the street centerline database if they fell within or adjacent to the selected census blocks. Extracted centerlines classified as “alleys,” “expressways,” or “ramps” were not of interest and were removed from the viable survey list. Additionally, some street centerlines either by error or by the nature of the city's land base data did not contain valid addresses as indicated by a value of “0” within the field denoting the lowest value to be found on the left side of a street segment. These centerlines were also removed from the viable survey list. The viable extracted street centerline segments were assigned a computer generated random number. The segments were then ranked from lowest to highest random number for each census block. Raters were given a list of the multiple street centerline segments with the lowest and next lowest random number for each selected census block and were instructed to travel to the block face with the lowest random number until a ratable block face was located.

Replacement Block Faces

When raters traveled to block faces, they sometimes found that the identified block face (i.e., the city block face containing the centerline segment with the lowest random number) had no residential dwellings or no apparent signs that people actually lived on the block face (e.g., all houses appeared abandoned, no mail boxes or mail slots were present). If the first block face represented by the street centerline with the lowest random number had no sign of residential activity, it was deemed unratable, and raters proceeded to the block face with the second lowest random number. In some cases, the second lowest number also was unratable and they traveled to successive block faces according to the random assignment procedures described above. This process was continued until a ratable block face was identified.

BPP Block Face Sample

In addition to the random sample of census block faces, a second sampling frame was used that corresponded to the residences of an existing sample of youth. Separate and distinct from the census-driven sample described above, the BPP sample was used to validate the NIfETy instrument and method. Archival individual-level data were obtained from the Second Generation Baltimore Prevention Program (BPP) at the Johns Hopkins University Bloomberg School of Public Health Prevention Intervention Research Center (PIRC). The sample in this longitudinal epidemiological study is comprised of 678 high-risk Baltimore City youths (and their caregivers) who have been assessed annually from the fall of their entry into first grade, in 1993, to the present. Each comprehensive annual youth assessment includes multi-item modules to assess constructs such as VAOD exposure, familial management, deviant peer exposure, manifestations of anxiety and depression, injury, behavioral changes, and neighborhood/community disadvantage. These measures will be used in subsidiary analyses to compare NIfETy ratings with youth-rated neighborhood environment and community-level exposure to VAOD. Each year the caregivers are also assessed on constructs such as household structure, neighborhood/community disadvantage, and parenting practices. A subset of these constructs was used in conjunction with the NIfETy to identify specific factors within communities that are associated with increased community VAOD exposure, as reported by the youths. See Furr-Holden et al. (2004) for a more detailed description of the BPP sample and instruments for this longitudinal study.

The BPP investigators provided the unit block information for the addresses of their study participants but not actual addresses. For example, if a BPP participant lived at 1614 N. Wolfe Street, the unit block information (i.e., 1600 N. Wolfe Street) was provided with an encrypted unique identifier for the participant. The NIfETy rating team was then sent to assess the entire 1600 block of N. Wolfe Street. The raters were blind to block face sampling frame; i.e., they were unaware whether they were rating a randomly sampled block face or a block face containing the residence of a BPP project participant.

Assessment Procedures

Neighborhood Inventory for Environmental Typology (NIfETy)

We used the multi-item Neighborhood Inventory for Environmental Typology (NIfETy) assessment in our study of Baltimore neighborhoods. The items that comprise the NIfETy were grounded in the theoretical contexts described earlier, but are operationalized with seven domains that correlate to how the assessments are actually conducted, namely: (1) physical layout of the block face, (2) types of structures, (3) adult activity, (4) youth activity, (5) physical disorder and order, (6) social disorder and order, and (7) VAOD indicators. Table 1 lists sample items for these domains. Wave One (July–November 2005) included 78 quantitative and 10 qualitative items and Wave Two (February–May 2006) included 114 quantitative and 15 qualitative items. Quantitative items had standardized prompts and response options. Each domain included an open-ended field positioned as the last item in each domain where raters wrote clear and concise qualitative narratives to relay related observations not captured with the quantitative items. In addition, there were additional fields within each domain to capture qualitative data (e.g., the type of commercial businesses and type of other youth activity). Trained raters entered their observations on Palm OS Zire 31 Personal Digital Assistant (PDA) devices programmed with Pendragon Forms 5.0 software to include skip patterns and branch trees for related items, to reject illogical entries, and to require each field be complete before advancing to the next item. Using electronic devices allowed the project team to readily distribute updates of the NIfETy Instrument to data collectors, and allowed data-collection and data-entry to occur simultaneously. When devices were synchronized via the Internet from remote locations such as their home or office, data were transferred from the devices to the secure server, old forms were removed and new forms with updates were downloaded to the devices.

Table 1.

Domains and sample items of the neighborhood inventory for environmental typology (NIfETy)

Domain	Sample items
Physical layout of the block face	Length and width of block; presence of alleys; presence of medians
Types of structures	Number and type of residential and commercial properties; number of churches
Adult activity	Number of adults on the street; adults supervising youth; adults exercising
Youth activity	Number of children on the street; youth riding bicycles; youth doing drugs
Physical disorder and order	Number of broken windows; abandoned houses; vacant lots; presence of trash; evidence of vandalism; number of potholes; number of abandoned vehicles; evidence of landscaping
Social disorder and order	Presence of homeless people, people yelling, fighting, loitering, intoxicated persons; evidence of prostitution; positive adult interaction
VAOD indicators	Shell casings, police tape/outlines, memorials on the block; number of people smoking tobacco, consuming alcohol, using or selling drugs

Daytime Ratings

Raters traveled in pairs to conduct daytime ratings. Daytime ratings were conducted from 11 a.m. to dusk. During times when precipitation fell or snow even partially covered the ground, raters did not conduct assessments. If raters were in the process of assessments and rain began, they completed the block face they were assessing and did not continue to another block face until the rain subsided. A standard procedure for completing the seven domains of the NIfETy was established to ensure consistency across raters and independent assessments. Without discussing their observations, raters traveled opposite sides of the street within visible range of one another and traversed the block face a minimum of three times at a modest walking pace. With each pass raters walked up one side of the street and down the other side. On the first pass, raters completed physical layout and structure type items and mentally noted the activity level and people present. Once they entered the physical layout data, they were instructed to pick a spot in the middle of the block face during their first walk, count the number of adults and children who passed that spot in a 3-min time window, and then to enter the activity data. During the second pass of the street, raters entered data on physical signs of disorder and mentally noted signs of social disorder. They then entered the social disorder domain items. During the final pass they observed and entered items of the VAOD indicators domain. They were instructed to look down alleys, in gutters, and in yards visible from the street. They were not to walk down alleys or on private property and were not to touch or move anything. The goal was to capture the experience of the neighborhood from the perspective of a keenly observant pedestrian. Upon completion of domain-specific items, qualitative narratives were completed for each domain.

Nighttime Ratings

After daytime NIfETy assessments were completed, one of the two daytime raters was assigned to complete a ‘drive by’ nighttime rating at 5–10 mph (i.e., the assessment occurred without the rater leaving their personal vehicle and as he or she cruised the block). Nighttime ratings were conducted between 10 p.m. and 3 a.m. on Thursday, Friday, Saturday and Sunday nights. Nighttime ratings followed the same procedures as daytime ratings with the following three exceptions. First, nighttime ratings included only the domains of youth and adult activity and social disorder/order because a rater within a car would be unable to see indicators of physical (dis)order and VAOD at night and the Principal Investigator made the a priori judgment that physical layout as well as structures and physical (dis)order were not likely to change between day and night. Second, raters were not required to work in pairs. Though only one rater was assigned the nighttime rating the assigned rater often opted to have a partner present for aid in data-entry and driving duties. Third, raters did not leave their cars for nighttime ratings. If necessary, the rater cruised the block face multiple times to capture anything missed in the first cruise. After cruising, the rater, if working alone, continued to a safe place, parked the car, and using the handheld device, entered the night ratings for the activity and social disorder/order domains. When working as a team, one rater drove and the other rater entered the assessment data. By design, nighttime ratings were to be completed within the same week and no more than 7 days from the corresponding daytime rating. Data were recorded on how many raters were present in the vehicle during the nighttime ratings and will be included as a covariate in future analyses. This procedure has since been changed and two raters are now required for all nighttime assessments.

Safety Procedures

Safety of our raters was a top concern. Our raters were instructed to travel in pairs for the daytime rating, carry a cell phone at all times, to be aware of their surroundings and to proceed immediately to their car and drive away if they felt uncomfortable being on an assigned block face. If they felt discomfort before beginning the assessment, they were instructed to drive to the next block face. In either case, they were instructed to call the field supervisor immediately to explain the nature of the event that resulted in their departure from the block face. If approached while on the block face, raters were instructed to explain briefly that they were working on a project to identify community needs for improving the environment. Raters carried business cards of the Principal Investigator and were instructed to share them when people had questions. Solo nighttime raters were instructed to drive to the next block face and call the field supervisor immediately if they felt discomfort on any assigned block face.

Training of Neighborhood Assessment Teams

Training of Raters

The first phase of training involved three 3-hour in-office evening sessions. The first session included an overview of the research project as well as personnel and scheduling issues. The second session focused on the NIfETy instrument and included explanations of each survey item, coding, and qualitative narratives collected. This session also included demonstration on the use of the electronic version of the instrument. The third session included further discussion and clarification of items in the NIfETy, practice in syncing PDAs, uploading data from remote stations and formation of team schedules and initial assignments.

The second phase of training consisted of field work on residential block faces not selected for inclusion in the study. Approximately 10 block faces were identified, representing a variety of neighborhoods and providing trainees with experience in coding the various NIfETy items. Field training occurred on three half-day sessions when groups of raters, working with the Principal Investigator or Project Manager, traveled to the selected block faces and conducted progressively more independent ratings. After each field session, project staff convened at a nearby location to debrief, review responses and answer questions. Following the first wave of data collection, select staff were trained and promoted to field trainers.

Block Face Assignments

Once training was completed, the Project Manager used e-mail to distribute assigned groups of approximately 10–20 block faces clustered by proximity to each pair of raters. Paper copies of all packets were also maintained by the Project Manager and sometimes delivered directly to a team while in the field.

Quality Assurance and Quality Control

Quality assurance measures included clearly-defined hiring strategies, precise training methods, instrument standardization, high-quality PDA programming (with skip patterns and safeguards to reduce/prevent erroneous or invalid entries), field exercises and remedial training.

In preparation for training, a team of senior staff members created gold standard ratings (to which the aspiring raters were blind), used in comparison with the ratings completed by aspiring raters during the field exercise component of training. Block faces used for the field exercise were within close proximity to one another but varied in VAOD indicator prevalence (low, medium, and high) and were not in the BPP or random census block face samples. Together, the team completed an abbreviated NIfETy containing only 10 survey items with relative permanence (i.e., those 10 items are not likely to undergo change in prevalence within 24 h). Examples of items with relative permanence included broken sidewalks, broken windows, landscaping, and bus stops. Examples of items with less permanence included trash, alcohol containers, adult activity and youth activity.

Quality control measures included quality control field sessions, booster trainings, weekly conference calls and rapid data-cleaning procedures. Weekly conference calls were conducted with the raters once teams began independent ratings. The Principal Investigator and Project Manager were present for conference calls, which provided a forum for answering raters' questions, discussing project updates or protocol changes, and reviewing basic procedures and data quality issues. All rater pairs were subject to at least one surprise bi-weekly quality control session with a field supervisor. The field supervisor assessed the validity of the rater pair's assessments with the supervisor's own “quality control rating.” These quality control sessions were also opportunities to detect departure from protocols and to offer tips for more efficient work in the field. In addition to collecting quality control ratings, field trainers were instructed to assess the quality of the rating session using a brief assessment tool, also programmed with Pendragon Forms 5.0 software, to assess punctuality, raters' adeptness with the maps and directions, adherence to rating and safety protocols, and raters' receptivity to being coached. The resulting data were used to assess the need for periodic booster trainings and to inform weekly conference call discussions. All data were cleaned with the goal of maintaining a maximum of 48 h between data collection and data cleaning and to quickly remedy any discrepancies.

Results

Sample Selection

Random Census-Based Block Face Selection

Using the stratification system outlined in the “Methods” section, 47 of the 242 residential neighborhoods in Baltimore City had one block face sampled, 186 neighborhoods had two block faces sampled, and 9 neighborhoods had three block faces sampled.

Replacement Block Faces

Three-hundred-thirteen (70.2%) of the block faces in the original sample represented ratable block faces. As revealed by the NIfETy field raters, the remaining 133 randomly-selected block faces had no residential dwellings or no apparent sign that people actually lived on the block faces and had to be replaced with the next random centerline segment. Eighty-eight of these replacements were ratable, leaving 45 replacement block faces that needed a second replacement. Twenty-five of the second replacements were ratable, leaving 20 block faces that needed further replacement. For these 20 block faces, scouts were dispatched with an ordered list of 15 possible replacement block face selections for the neighborhood. The scouts visited the block faces in the order listed, checking for block face ratability, until a block face with evidence of residential occupancy was encountered. This method was more efficient for these problematic neighborhoods that needed review of 3 to 14 block faces before a ratable block face was encountered in that neighborhood. Additionally, one block face was ratable at Wave One but needed a replacement for Wave Two because the area was demolished and reconstructed with new street names between the Wave One and Wave Two data collections.

Assessment Procedures

Neighborhood Inventory for Environmental Typology (NIfETy)

During data collection, a total of 203 narrative entries were generated for the various domains during data collection. The resulting narratives were assembled and evaluated at the end of Wave One. A research assistant trained in qualitative research flagged narrative themes that appeared repeatedly across raters. The flagged themes were submitted to the Principal Investigator for inclusion in the NIfETy Instrument, resulting in an additional 36 new items in the inventory.

Day ratings were completed at an average of 1.8 per hour, including travel time to and between block faces. Preliminary analysis of the first two waves of daytime data indicated that the amount of time raters spent on block faces correlated positively with prevalence of VAOD indicators. On average, raters completed 5.84 night ratings per hour when working alone. When night ratings were done in pairs, the ratings were completed at a rate of 7.6 ratings per hour and with better employee morale at the end of the session.

Selection and Training of Neighborhood Assessment Teams

Field Exercise Methods

After Wave Two, all raters were required to demonstrate observation skills on a field exercise that served as the third training component for subsequent waves. During the field exercise, aspiring raters were required to accurately rate six block faces within close proximity in 3 h. Accuracy was determined by raters' correlations to the gold standard rating created by trainers.

The field exercises were held within 24 h of the gold standard rating. Teams of 3–4 raters assessed the six block faces selected during the creation of the gold standard ratings. Teams were assigned the block faces in such a way that no two teams would be on the same block face at the same time. During the field exercises, teams were accompanied by one trainer or senior staff member to: (a) ensure that proper protocol was used and (b) field any questions from community members. Field exercise data were analyzed by survey item using the Pearson's product moment correlation coefficient to determine whether the gold standard rating and data collector's rating varied in the same direction and magnitude across block faces. Raters whose correlations for six block faces rated within 3 h were greater than 0.65 were deemed passing. Aspiring raters failing to meet this standard on the first attempt were offered remedial training and re-tested during a second field exercise. Remedial training occurred during the first two weeks of data collection. Rater pairs with a rater in need of additional training were assigned a trainer for coaching during actual ratings. The trainer rated block faces with the pair as a third rater and the trainer's data were used instead of the remedial rater's data until the rater passed a field exercise. After three failed field exercises, raters were relieved of service on the project. The first field exercise was completed by 11 raters. Two raters did not pass after three field exercise were subsequently released from the project. Of the nine raters who passed, the average correlation coefficient between ratings of aspiring raters and gold standard rating was 0.75, with no single rater having a correlation below 0.65.

Quality Control

Quality control session data were analyzed by comparing the data collectors' ratings to trainer's with the quality control rating. The result was an average correlation coefficient of 0.68. These sessions, in conjunction with immediate data cleaning, revealed numerous errors across raters related to the block face ID input. To reduce error, a three-step protocol was added to the block face rating protocol. A rater would (a) input block face ID into his PDA, (b) read the inputted ID to his partner, and (c) listen to his partner read his inputted ID back to him, checking for consistency with his own data. The data cleaning also exposed the wide variation in lag time between completion of the day rating and its associated night rating. Corrective action was taken to ensure minimum time (i.e., no more than 7 days) between day and night ratings.

Sample Block Face Characteristics

In total, 844 block faces were assessed in the Wave One NIfETy data collection. These block faces represent: (a) a stratified random selection of 446 block faces selected from the 242 residential neighborhoods of Baltimore City and (b) the 398 BPP youth living in Baltimore City. Figure 1 displays the distribution across Baltimore City of sampled block face depicted by black dots. The red dots represent the remaining 398 block faces inhabited by participants in the Baltimore Prevention Program (BPP). The BPP participants are spread across the city, although they reside in only 109 of the city neighborhoods. BPP participants cluster in distinct geographic regions within the city. Approximately one-half of them reside in the northwest section of Baltimore City.

Fig. 1.

Randomly selected census and Baltimore Prevention Program residential blocks: a sample of Baltimore City centerline segments for NIfETy Method assessment

Table 2 displays prevalence summaries for the core domains of alcohol, drug, and violence indicators for both BPP participant and random block faces. As shown, BPP participant residential environments are not appreciably different from those of the randomly sampled Baltimore City block faces. Detailed metric analyses of the NIfETy Instrument are forthcoming in future reports and described briefly below.

Table 2.

Waves One and Two observed violence, alcohol and other drug indicator prevalence: Estimates by sampling frame

	Wave One				Wave Two
	0	1	2	3+	0	1	2	3+
Alcohol n (%)
BBP n=398	254 (63.82)	80 (20.10)	62 (15.58)	2 (0.50)	153 (38.44)	54 (13.57)	175 (43.97)	16 (4.02)
Census n=446	226 (50.67)	87 (19.51)	124 (27.80)	9 (2.02)	158 (35.35)	87 (19.46)	174 (38.93)	28 (6.26)
Drugs n (%)
BBP n=398	328 (82.41)	9 (2.26)	31 (7.79)	30 (7.54)	195 (48.99)	104 (26.13)	65 (16.33)	34 (8.54)
Census n=446	313 (70.18)	12 (2.69)	70 (15.70)	51 (11.43)	251 (56.15)	106 (23.71)	62 (13.87)	28 (6.26)
Violence n (%)
BBP n=398	346 (86.93)	43 (10.80)	9 (2.26)	0 (0.00)	356 (89.45)	26 (6.53)	13 (3.26)	3 (0.75)
Census n=446	349 (78.25)	65 (14.57)	30 (6.73)	2 (0.45)	389 (87.02)	44 (9.84)	12 (2.68)	2 (0.45)

Metric Properties of the NIfETy

Internal Consistency Reliability

Internal consistency reliability for the Total NIfETy scale was exemplary with an interclass correlation (ICC) of 0.84. Alpha coefficients ranged from minimal to exemplary for each of the subscales (ICC=0.27 to 0.90). Alpha estimates were as follows: ICC=0.90 Physical Layout subscale; ICC=0.79 Youth Activities subscale, ICC=0.74 VAOD subscale; ICC=0.71 Physical (Dis)Order subscale; ICC=0.63 Adult Activities subscale; and ICC=0.60 Social (Dis)Order subscale. Type of Dwelling/Edifice had the poorest internal consistency estimates (ICC=0.27 or minimal), due primarily to the exclusivity of items within this scale (e.g., row houses vs. single family homes).

Inter-Rater Reliability

Inter-rater reliability was substantial. Average rater reliability is reported for Shrout and Fleiss' (1979) two-way mixed effects ICC model (Case 3) with the consistency agreement definition. Average measure reliability uses the mean of all raters as the unit of analysis, and was reported given that similar ICC values were obtained when compared to a random sample of one rater per block. The preponderance of significant ICC values were in the substantial to almost perfect range. ICC values for all items by scale were in the following ranges: Physical Layout (0.61–0.98), Type of Structures (0.71–0.94), Physical Order/Disorder (0.60–0.99), Social Order/Disorder (0.70–0.82), Adult Activity (0.69–0.85), Youth Activity (0.62–0.82), and VAOD Indicators (0.67–0.79).

Validity

Validity of the NIfETy Method and NIfETy Instrument was established in this investigation. Using spatial overlay in ArcGIS, we compared NIfETy VAOD subscale data to Baltimore City crime data related to violence, alcohol and drugs. Our subscale predicted areas with high crime rates and revealed pockets of low levels of VAOD activity in areas with little or no crime, perhaps revealing upcoming shifts in the geography of crime in the City. Bivariate correlations were computed between composite VAOD indicators from Wave Two of the NIfETy and various BPP youth-report items. A composite violence variable was created for each block face by making the violence indicators binary and then summing the number of endorsements of each item. There was a positive correlation between the composite NIfETy violence items with BPP youth-reported counts of violence victimization and witnessing violence [r (324)=0.17, p≤0.05; r (324)=0.11, p≤0.05], the NIfETy composite alcohol item and the BPP youth item: “In the morning or later in the day, I often see drunk people on the street in my neighborhood” [r (324)= 0.24, p≤0.05]. There was also a significant positive correlation between the NIfETy composite drug item and the BPP youth item, “I have seen people using or selling drugs in my neighborhood,” [r (324)=0.29, p≤0.05].

Discussion

This report describes successful development of a systematic city-wide assessment of residential neighborhood environmental characteristics of VAOD exposure. The work undertaken in this investigation established the feasibility of the NIfETy Method and also refined the assessment tool, training protocol, and quality control and assurance procedures. Several lessons were learned in the conduct of this work. First, it is important that quality assurance measures are implemented for data collectors or raters to ensure that they are well trained, operating within the defined protocol, and producing quality assessments over time. Retaining raters across waves was associated with increased performance and quality. In addition, open-ended qualitative measures in the assessment tool allowed us to identify additional observable quantitative features of the environment. These features, subsequently added to the inventory, were in line with our theoretical model of how the environment influences behavior. We were able to establish the metric properties of the NIfETy Instrument and demonstrate that the instrument, and overall NIfETy Method for environmental assessment, is a valid and reliable approach to measure environmental factors associated with adolescent VAOD exposure.

Three limitations of this work merit discussion. First, these data do not provide insight on whether or not the observable environmental features were generated by residents of the community or from outsiders. Given that crime and incivility are mobile, we suspect that some proportion of community decay is attributable to residents outside of the community. If, in fact, this is the case, the signals of VOAD exposure may be more an indication of passive exposure (e.g., witnessing) or victimization rather than perpetration (e.g., drug use or committing violent acts). In line with our theoretical framework, however, signs of decay and disregard in a community increase the risk for further community decline and decay and, thus, despite this limitation, merit investigation. This issue is slightly less problematic when using other local data such as police data which parcels out where crime happens and where criminals live, but this data also has limitations and the police data records of where crimes occur are often inaccurate. For example, if an officer does not know exactly where a crime occurred, either the police station or the location where the crime was reported (e.g. the address of the person that called to report a crime) is recorded as the event location. There is also similar bias in the reporting and recording of the residential address of the person who committed the crime. The next stage of metric analysis of the NIfETy Method will include multilevel analysis of multiple sources of data including police data on crime and arrests, as well as calls-for-service data of non-criminal activity.

The second limitation of our approach is the reliance on adolescent reports of exposure to validate NIfETy ratings. It could be that the environmental hazards have a differential impact across the life span and children or younger adolescents have a different response to environmental stimuli and conditions. To address this limitation we have initiated collaboration with the Multiple Opportunities to Reach Excellence Project and in future investigations will validate our measures in a sample of younger school-aged children. This will include a longitudinal psychosocial study of the children and accompanying environmental assessments using the NIfETy Method. In this next stage of research we will clarify the link between community-level factors and youth exposure. We will also examine how relocation to different residential neighborhoods influences subsequent behavior and neighborhood exposure.

The final limitation of this work is the limited geographic areas in which the NIfETy Method has been implemented. We have extensive data on Baltimore City and have recently begun applying the NIfETy Method to environmental studies in the Greater Washington Metropolitan Area. Future investigations will implement our method in other geographic areas and also explore comparable methods with utility in less urban or rural settings. We conducted a small scale, 239-block face pilot study of rural and semi-rural block faces in Maryland. We found that NIfETy scores were very low in these areas and that our existing measures are less sensitive to environmental features in these locations that are associated with VAOD exposure. This was further confounded by very low base rates of reported VAOD exposure among the children who lived in these rural and semi-rural communities. Future investigations will expand the scope of environmental assessments to include measures for rural communities and we will also follow this semi-rural population as they age and initiate drug use.

Despite these limitations, this work presents a model approach to local area environmental assessment. To our knowledge, no other environmental assessment method of this nature exists that has been demonstrated to be both valid and reliable, and is comprehensive, as well as longitudinal. The NIfETy Method has been successfully deployed for six successive waves since the inception of the project in 2005 between February and April (Winter/Spring Assessments) and between June and September (Summer Assessments). Future NIfETy Method assessments will happen once during each of the annual seasons. Future reports from this team will include an in-depth metric analysis of the NIfETy, a detailed report on the association with youth reports of VAOD exposure and a longitudinal analysis to draw causal inferences regarding the relationship between VAOD exposure and environmental exposure. This work will be used to guide future environmental preventive interventions.