Can Virtual Streetscape Audits Reliably Replace Physical Streetscape Audits?

Abstract

There is increasing recognition that the neighborhood-built environment influences health outcomes, such as physical activity behaviors, and technological advancements now provide opportunities to examine the neighborhood streetscape remotely. Accordingly, the aims of this methodological study are to: (1) compare the efficiencies of physically and virtually conducting a streetscape audit within the neighborhood context, and (2) assess the level of agreement between the physical (criterion) and virtual (test) audits. Built environment attributes associated with walking and cycling were audited using the New Zealand Systematic Pedestrian and Cycling Environment Scan (NZ-SPACES) in 48 street segments drawn from four neighborhoods in Auckland, New Zealand. Audits were conducted physically (on-site) and remotely (using Google Street View) in January and February 2010. Time taken to complete the audits, travel mileage, and Internet bandwidth used were also measured. It was quicker to conduct the virtual audits when compared with the physical audits (χ = 115.3 min (virtual), χ = 148.5 min (physical)). In the majority of cases, the physical and virtual audits were within the acceptable levels of agreement (ICC ≥ 0.70) for the variables being assessed. The methodological implication of this study is that Google Street View is a potentially valuable data source for measuring the contextual features of neighborhood streets that likely impact on health outcomes. Overall, Google Street View provided a resource-efficient and reliable alternative to physically auditing the attributes of neighborhood streetscapes associated with walking and cycling. Supplementary data derived from other sources (e.g., Geographical Information Systems) could be used to assess the less reliable streetscape variables.

Introduction

Increasingly, public health researchers are realizing the associations between urban form and health outcomes.^1–3 Specifically, there is an emergent field examining the relationship between the built environment and physical activity behaviors. Much work has been done at the neighborhood level and it is now recognized that local streets are important facilitators for walking and cycling engagement for both recreation and transport purposes.^4–6

The Systematic Pedestrian and Cycling Environment Scan (SPACES) audit tool has been specifically developed by Australian researchers to capture the contextual elements of neighborhood streets that promote or inhibit walking and cycling across different domains.^7,8 Briefly, Pikora et al⁸ used a two-stage process (stakeholder interviews and a Delphi study—a systematic communication process used to reach consensus between experts) to develop the SPACES audit tool, and identified four overarching factors that likely supported physical activity within the neighborhood environment: function, safety, aesthetics, and destinations. Factors comprised a number of elements that were, in turn, made up of a number of survey items (refer Table 1). The SPACES tool has demonstrated appropriate reliability for most variables (kappa ≥ 75% agreement).⁹ The New Zealand-SPACES (NZ-SPACES) is a modification of this tool and both instruments have been used successfully in physical activity research.^7,10

Table 1.

Factors, elements, and items for walking and cycling

Factor	Element	Element weight	Item	Item weight
Walking
Function	Walking surface	1.00	Negotiating of footpath	0.05
			Type of path	0.20
			Path smoothness	0.10
			Path material	0.10
			Slope	0.20
			Smoothness/condition	0.10
			Continuity	0.20
			Curb type	0.05
	Neighborhood permeability	1.00	Other routes available	0.50
			Neighborhood legibility	0.50
	Walking infrastructure	0.33	Seats	0.50
			Trees/verandas	0.50
Safety	Streets (lanes)	0.33	Number of lanes	1.00
	Fixed traffic controls	0.33	Traffic control devices present	1.00
	Path safety	0.66	Path location	0.30
			Fixed obstacles on path	0.10
			Street lights	0.20
			Surveillance	0.20
			Graffiti/vandalism	0.20
	Traffic safety	0.66	Crossing type	0.50
			Crossing aids	0.30
			Visible driveways	0.20
Aesthetics	Streetscape aesthetics	0.66	Trees	0.25
			Gardens maintained	0.25
			Verges maintained	0.25
			Cleanliness	0.25
	View aesthetics	0.66	Views	1.00
	Subjective walking assessment	0.33	Attractiveness	0.50
			Physical difficulty	0.50
Destinations	Land use mix	1.00	Number of destinations present	Out of 10
Cycling
Function	Cycling surface	1.00	Path type	0.30
			On-road cycle lane	0.30
			Slope	0.20
			Road condition	0.10
			Curb type	0.10
	Neighborhood permeability	1.00	Other routes available	0.50
			Neighborhood legibility	0.50
	Cycling infrastructure	0.33	Bicycle storage	1.00
Safety	Streets (lanes)	0.66	Number of lanes	1.00
	Fixed traffic controls	0.66	Traffic control devices present	1.00
	Traffic safety	0.33	Crossing type	0.50
			Crossing aids	0.30
			Visible driveways	0.20
Aesthetics	Streetscape aesthetics	0.33	Trees	0.25
			Gardens maintained	0.25
			Verges maintained	0.25
			Cleanliness	0.25
	View aesthetics	0.33	Views	1.00
	Subjective cycling assessment	0.33	Attractiveness	0.50
			Physical difficulty	0.50
Destinations	Land use mix	1.00	Number of destinations present	Out of 10

Simultaneously, technological advancements are providing opportunities to examine the built environment remotely. Common examples include geographical information systems (GIS) and online mapping software. For example, satellite imagery overlaid with GIS was recently used to measure the openness, size, and greenness of walking trails in Indianapolis,¹¹ and GIS techniques have been used extensively to derive measures of neighborhood walkability.^4,6,10 Another tool, Google Street View,¹² potentially allows for the contextual streetscape environment to be assessed from an Internet-linked computer. Google Street View is accessed through Google Maps¹³ or Google Earth,¹⁴ and allows users to remotely navigate through interactive 360° panoramic images at the street level. Users are able to view these virtual spaces as if they were traveling through them in reality. In areas where there is extensive Google Street View coverage (i.e., Australasia, Europe, North America, and parts of Asia) it is possible to navigate through whole cities at the street level. One of the few published examples of its use as a research tool has been to monitor differences in rates of recovery and continued abandonment between New Orleans neighborhoods post Hurricane Katrina.¹⁵ To our knowledge, however, Google Street View has not been used to undertake streetscape audits for research purposes. Thus, the potential use of Google Street View to audit these settings has important methodological implications for the built environment research field in terms of possibly increasing the efficiency of environmental measurement. The reliability of the tool for this purpose, however, first needs to be established.

Accordingly, the objectives of this study are firstly to, explore the efficiencies of physically and virtually conducting the NZ-SPACES audit within the neighborhood context, and secondly, to assess the level of agreement between the physical (criterion) and virtual (test) audits. Establishing the relative efficiency and reliability of virtual streetscape audits compared with on-site (physical) audits could have important methodological implications for future built environment and health-related physical activity research.

Methods

Street Segment Selection

As part of an ongoing study,¹⁰ four neighborhoods (Waimumu, Matipo, Unsworth Heights, Cheltenham) in Auckland, New Zealand, were selected using GIS procedures. Neighborhoods were defined as five contiguous meshblocks with similar physical characteristics. Please refer to Badland et al¹⁰ for more detailed methodology regarding neighborhood identification, selection and characteristics. Within each of these neighborhoods, 12 street segments (defined as intersection to intersection) were selected, and these were audited physically and virtually. The total length of the street segments audited in the neighborhoods ranged from 1,765 m in Cheltenham to 2,654 m in Waimumu. The initial street segment start point for each neighborhood was randomly generated using GIS procedures and thereafter the segments to be audited were selected sequentially. Figure 1 provides a map of Matipo illustrating the 12 street segments selected in the neighborhood. Scores from each street segment were combined to provide two neighborhood streetscape values for each element, an on-site audit measure and a virtual measure.

FIGURE 1.

Map of Matipo study neighborhood illustrating street segments.

NZ-SPACES Audit Tool

The NZ-SPACES tool was used to undertake the physical and virtual audits of the selected street segments. The field researcher was trained to use NZ-SPACES by a supervisor and provided with a training manual based on SPACES protocols adapted to include New Zealand specific reference images. The field researcher had limited prior experience of Google Street View but he found it user friendly, and quick and easy to become proficient in its use. He undertook practice physical and virtual audits prior to data collection. The NZ-SPACES tool is available from the authors on request.

Physical Audit

The field researcher conducted the physical audits by walking along both sides of the nominated street segments while simultaneously completing the NZ-SPACES on a laptop computer. The time taken to complete the audit (including travel time) and distances traveled to and within the neighborhood were also recorded.

Virtual Audit

Virtual audits of the streetscape were conducted by the same researcher using Google Street View software. The same procedures were applied as for the physical audit, but the virtual audit was conducted remotely by examining computer-generated streetscape images derived from Google Street View. The Google Street View images for the selected neighborhoods were recorded between January and July 2008 (personal communication, A. Baxter, Public Relations Manager, Google Australia and New Zealand, 17 February 2010). The time taken to complete the virtual audits and the Internet bandwidth used were also documented. To reduce potential order effects, two neighborhoods were audited virtually first followed by the physical audits, and the remaining two neighborhoods were assessed in the reverse order. Further, to reduce the effect of taking repeated measures of the same stimuli (streetscapes), the physical audit of a specific neighborhood did not directly follow the virtual auditing of the same neighborhood, and vice versa. The two audits were completed at least 5 days apart and within this time period at least one alternative neighborhood had been audited either physically or virtually.

Factor and Weighting Development

The contribution of each streetscape item assessed in the street segments was determined using a modified version of the SPACES conceptual framework developed by Pikora et al.⁸ In the present study, we made minor item and weighting modifications to the SPACES, and used the revised framework to evaluate the perceived importance of the neighborhood environment for two separate behaviors, walking and cycling, in the New Zealand context. Response categories for each item were given a value between a 1 and 0 (between two and four levels were employed for each item), where 1 equaled the most supportive environment for the behavior. The factors, elements, items, and weights used in this study for walking and cycling are presented in Table 1 and this template was applied for both the physical and virtual streetscape audits. The same streetscape data were used for calculating walking and cycling specific factor scores, but different weightings were applied for the outcome variables.

Statistical Analysis

Comparisons of efficiency between the two approaches were made with regard to time taken to complete the physical and virtual audits. Level of agreement between the physical (criterion) and virtual (test) streetscape measures was determined using two-way mixed model intraclass correlation coefficients (ICC). The ICC value cut-off ranges used were: 0.0–0.20 (weak agreement), 0.21–0.40 (poor agreement), 0.41–0.60 (moderate agreement), 0.61–0.80 (substantial agreement), and 0.81–1.00 (almost perfect agreement).¹⁶ Overall, an ICC ≥ 0.70 was considered an acceptable measure of reliability between the two measures.¹⁷ One-way analysis of variance (ANOVA) statistics were incorporated to further determine if systematic differences existed between the measures.¹⁸ All reliability analyses were conducted with SPSS v16.0 software (SPSS Inc., Chicago, IL).

Results

Overall, 48 street segments were assessed both physically and virtually across four neighborhoods using the NZ-SPACES tool. The order of the physical audits was: Unsworth Heights, Matipo, Cheltenham, and Waimumu. The order of the virtual audits was: Matipo, Cheltenham, Unsworth Heights, and Waimumu. Figures 2 and 3 show the time taken to complete the audits by neighborhood location and order, respectively. Overall, it was quicker to conduct the virtual audits; mean minutes (±SD) to complete each neighborhood audit were 115.3 ± 52.7 (virtual) and 148.5 ± 6.4 (physical). The time taken to complete the audits decreased as familiarity of the measures increased; this was most evident for the virtual audit (Figure 3). Additional costs for the physical audit included the travel time to the neighborhoods which totaled 215 min over 158 km and the virtual audits consumed 335 MB data.

FIGURE 2.

Comparison of the time taken to complete the physical and virtual audits by neighborhood location.

FIGURE 3.

Comparison of the time taken to complete the physical and virtual audits by neighborhood order.

The second objective of the study was to test if the virtual audit could reliably be used in place of the physical streetscape assessment. Data presented in Table 2 demonstrated the physical and virtual audits were within the acceptable levels of agreement for the majority of the elements being assessed (ICC ≥ 0.70). Indeed, five elements showed perfect agreement between the two instruments. For walking, the ICC values between the physical and virtual audits for the number of fixed traffic controls, neighborhood permeability, and land use mix were below the level of acceptable agreement. Neighborhood permeability and land use mix were also below the acceptable threshold for cycling. It was not surprising that similar elements did not reach acceptable reliability for walking and cycling, as they were constructed from the same item data. Apart from cycling path surface (p value = 0.015), the non-significant ANOVA values indicated there were no systematic element differences between the physical and virtual audits.

Table 2.

Test–retest reliability levels of agreement between the physical and virtual audits for walking and cycling

Factor	Element	Physical audit score	Virtual audit score	ICC	F	p value
Walking
Function	Walking surface	1.20	1.24	0.95a	2.97	0.18
	Neighborhood permeability	0.61	0.62	0.60	1.00	0.39
	Walking infrastructure	0.16	0.14	0.94a	8.00	0.07
Safety	Streets (lanes)	0.22	0.22	1.00b	–	–
	Fixed traffic controls	0.05	0.04	0.21	0.35	0.60
	Path safety	0.45	0.42	0.94a	2.03	0.25
	Traffic safety	0.12	0.08	0.84a	1.49	0.31
Aesthetics	Streetscape aesthetics	0.60	0.59	0.99a	6.00	0.09
	View aesthetics	0.66	0.66	1.00b	–	–
	Subjective walking assessment	0.23	0.25	0.95a	6.82	0.08
Destinations	Land use mix	2.46	2.68	0.62	0.52	0.53
Cycling
Function	Cycling surface	0.84	0.85	1.00b	25.00	0.02
	Neighborhood permeability	0.61	0.62	0.60	1.00	0.39
	Cycling infrastructure	0.00	0.00	1.00b	–	–
Safety	Streets (lanes)	0.44	0.44	1.00a	–	–
	Fixed traffic controls	0.10	0.04	0.76a	5.54	0.10
	Traffic safety	0.06	0.04	0.77a	1.41	0.32
Aesthetics	Streetscape aesthetics	0.30	0.29	0.92a	1.42	0.32
	View aesthetics	0.33	0.33	1.00b	–	–
	Subjective cycling assessment	0.23	0.25	0.92a	5.40	0.10
Destinations	Land use mix	2.46	2.68	0.62	0.52	0.53

F analysis of variance, ICC intraclass correlation coefficient

^aAcceptable agreement

^bPerfect agreement

Discussion

These findings demonstrated that Google Street View was, for the most part, an efficient and effective tool to measure the streetscape context at the neighborhood level. Once familiarity of Google Street View software was gained, the virtual audits were quicker to administer when compared with the physical street assessments. Furthermore, acceptable levels of agreement were demonstrated for the majority of the elements tested. For the variables that did not reach acceptable limits of agreement (land use mix (walking and cycling), neighborhood permeability (walking and cycling), fixed traffic controls (walking)), other forms of supplementary information sourced from GIS could be utilized to accurately measure these variables. As such, these findings have important methodological implications for measuring the contextual setting; Google Street View is an acceptable tool when combined with NZ-SPACES to measure streetscape environments that support or inhibit physical activity engagement.

It was unexpected, however, to see agreement discrepancies between the physical and virtual audits for fixed traffic controls in the walking assessment. One potential explanation could be the time delay between the virtual audit imagery (January–July 2008) and the physical audit being conducted (January–February 2010). It was unknown whether substantial traffic control device construction occurred between these time points in the study neighborhoods. Accordingly, the date/s of the virtual imagery should be considered when selecting research approaches. If more recent relevant GIS databases are available, these should be used to assess elements/items. On the other hand, it was unsurprising that land use mix and neighborhood permeability did not reach acceptable levels of agreement. On Google Street View, a building’s purpose may not have been clear and any pedestrian and cyclist “cut-throughs” that would enhance the legibility of the neighborhood would be harder to detect using remote means. When available, objectively derived (GIS) measures of land use mix and street connectivity should be used to accurately assess these variables.

The major advantages of auditing streetscapes virtually arise from the researcher’s ability to access locations remotely. Research-related costs such as transportation time and mileage are substantially lower than when conducting physical audits. In this study, the field researcher traveled 158 km to access the four neighborhoods (NZ$110.60 costed at NZ$0.70 per kilometer) compared with 335 MB of data used to conduct the virtual audits (NZ$6.70 costed at NZ$0.02 per MB). The monetary advantages are compounded when the costs of establishing intra- and inter-rater reliability are included.

Another key benefit of using virtual audits is comparability in the street-level data on which international comparative studies are based. Increasingly, built environment and physical activity research seeks to compare data internationally, for example the International Physical Activity and Environment Network¹⁹ currently conducting neighborhood-level research across eight countries. Having the ability to effectively assess diverse environments in a timely manner will assist in identifying the optimal physical environments for physical activity engagement. Furthermore, Google Street View images are currently being updated (since November 2009) with near high-definition photography, but these images were not available at the time of conducting the present research. It is anticipated that Google will update Street View images every 18 months (personal communication, A. Baxter, Google representative, 17 February 2010), and it is likely that as the resolution improves, so will the ability to contextualize the neighborhood streetscape, therefore increasing the reliability of virtual streetscape auditing.

A limitation of Google Street View as a research tool is the time lapse between the recording of the street imagery and the physical audit date. This will be problematic for some items more than others. For example, the slope of the streets and footpaths will likely not change, and neighborhood permeability is less likely to change over time than the condition of the footpaths. There may also be limitations in its use to compare attributes between areas if data were recorded in substantially different time periods. As such, the date the images were captured should be considered when developing streetscape assessment methodology. Intra- and inter-rater reliability were also not established in this study, and this work should be conducted prior to using Google Street View in large-scale studies.

To summarize, Google Street View is a potentially valuable data source for measuring the contextual features of neighborhood streets that facilitate or inhibit health behaviors. Using the NZ-SPACES tool, the study found that Google Street View provides a resource-efficient and reliable alternative to physically auditing the attributes of neighborhood streets associated with walking and cycling. This likely has important methodological implications for assessing urban form in future built environment and health research.