Table 3.

Agreement for Each Item Within Domains by Method (Street- and Parcel-Level) and Source (Direct and Virtual)

	Inter-rater agreementa Direct observation (n=84 ratings)				Inter-rater agreementa Virtual observation (n=84 ratings)				Intra-rater agreementb Street- versus parcel- level Methods (n=84 ratings)
	Street- level (n=42 ratings)		Parcel- level (n=42 ratings)		Street- level (n=42 ratings)		Parcel- level (n=42 ratings)		Direct (n=42 ratings)		Virtual (n=42 ratings)
Items	% Agree	K	% Agree	K	% Agree	K	% Agree	K	% Agree	K	% Agree	K
Physical disorder
Moderate/heavy garbage or litter	64	0.27	83	0.56	67	0.29	67	0.23	81	0.58	83	0.66
Graffiti present	88	0.48	83	0.27	93	--	93	0.69	92	0.62	92	0.59
Cluttered residential porches present	69	0.04	71	0.39	83	0.27	76	0.35	80	0.49	87	0.57
Physical decay
Fair/poor residential conditionc	83	0.52	76	0.42	80	0.32	83	0.49	79	0.44	81	0.37
Fair/poor residential yard conditiond	83	0.52	90	0.55	80	0.32	85	0.20	86	0.55	90	0.58
Fair/poor businessese	91	0.81	89	0.8	73	0.48	67	0.34	89	0.78	90	0.80
Fair/poor vacant lotsf	65	0.37	67	0.22	53	0.15	59	0.05	70	0.36	66	0.27
Fair/poor streetg	98	1.00	100	1.00	98	0.66	100	1.00	99	0.66	98	0.79
Fair/poor sidewalksh	86	0.23	86	0.38	86	−0.07	89	−0.04	95	0.72	96	0.65
Street safety
Speed limit Signs present	83	0.56	78	0.51	88	0.70	93	0.82	88	0.73	95	0.88
Bike lane present	100	--	100	--	100	--	100	--	99	--	99	--
>50% of street lit	48	−0.05	56	0.13	64	0.29	55	0.08	90	0.81	86	0.71
Safety
Rated as unsafe place to live	69	0.26	71	0.32	77	0.33	69	0.23	96	0.90	97	0.91
Rated as unsafe to walk at night	59	0.23	68	0.35	68	0.31	60	0.14	97	0.94	96	0.90
Alarm systems/security signs present	86	0.70	90	0.79	80	0.61	93	0.85	90	0.79	87	0.74
Land usei
Residential	100	1.00	100	1.00	100	1.00	100	1.00	100	1.00	100	1.00
Commercial, business	100	1.00	93	0.81	93	0.81	93	0.81	96	0.91	95	0.87
Industrial, warehouse, manufacturing	100	--	100	--	100	--	95	--	100	---	98	---
Institutional	98	0.88	100	--	95	0.77	98	0.88	99	0.93	99	0.94
Recreational facility	100	--	100	--	98	--	100	--	100	--	98	--
Vacant lot, open space	79	0.57	93	0.85	90	0.79	95	0.90	94	0.88	93	0.85

Notes: Observed agreement >75% is considered substantial.^{19, 20}

Kappa values: 0–0.20=poor, 0.21–0.40=slight, 0.40–0.61=moderate, 0.61–0.80=substantial, 0.80–1.000=almost perfect.²¹ Kappa coefficients were not reported for items with a base rate <10% as the expected chance agreement is inflated and the Kappa is lowered in these cases.²²

^aInter-rater agreement was measured as observed agreement and a simple kappa coefficient between two raters.

^bIntra-rater agreement was measured as observed agreement and a simple kappa coefficient between virtual vs. direct ratings performed by the same rater.

^cResidential condition rated on a 4 point scale (poor/badly deteriorated; fair; good/well-kept; no residential units on the street). A dichotomous variable was created good/well-kept vs. fair and poor/badly deteriorated. Prevalence reported for fair, poor/badly deteriorated.

^dResidential yard condition rated on a 4 point scale (poor/badly deteriorated; fair; good/well-kept; no residential yards on the street). A dichotomous variable was created good/well-kept vs. fair and poor/badly deteriorated. Prevalence reported for fair, poor/badly deteriorated.

^eCommercial/business condition rated on a 4 point scale (poor/badly deteriorated; fair; good/well-kept; no commercial/businesses on the street). A dichotomous variable was created good/well-kept vs. fair and poor/badly deteriorated. Prevalence reported for fair, poor/badly deteriorated.

^fVacant lot condition was assessed using three question on physical features used to categorized as (poor/badly deteriorated; fair; good/well-kept; no commercial/businesses on the street). A dichotomous variable was created good/well-kept vs. fair and poor/badly deteriorated. Prevalence reported for fair, poor/badly deteriorated.

^gStreet condition were rated on a 4 point scale (poor/badly deteriorated; fair; good/well-kept; under construction; not present). A dichotomous variable was created good/well-kept vs. fair and poor/badly deteriorated. Prevalence reported for fair, poor/badly deteriorated.

^hSidewalk condition were rated on a 5 point scale (poor/badly deteriorated; fair; good/well-kept; under construction; not present). A dichotomous variable was created good/well-kept vs. fair and poor/badly deteriorated. Prevalence reported for fair, poor/badly deteriorated.

ⁱPercent of street segments containing at least one parcel of specified land use category.