Hedonic property values and water quality: A meta-analysis of commodity, market, and methodological choices

Abstract

This study quantitatively reviews the hedonic literature examining surface water quality to assess how attributes of the commodity, housing market, and methodological choices lead to variation in the significance and expected sign of the estimated property value effects (i.e., elasticities). We conduct a meta-analysis of 29 studies with 290 unique estimates, published or released between 1985 and 2017, and find evidence based on probit meta-regression models that some of the definitions and decisions made in primary studies do influence the estimated relationship between water quality and home prices. Our most robust evidence suggests that methodological choices (e.g., accounting for spatial dependence, or if the water quality measure was based on something other than in situ measurement) have a critical role in determining the likelihood of finding a significant and theoretically expected result; and perhaps most importantly, it is not always selections that reflect best practices that lead to this finding. This study can help identify potential concerns with data and modeling choices in the collective hedonic literature focused on water quality.

1. Introduction

The number of hedonic property value studies continues to grow as data on sales and house attributes, including surrounding environmental conditions, become more detailed and readily accessible (Bishop et al., 2020; Guignet and Lee, 2021; Petrolia et al., 2021). This growth in the literature provides an opportunity to analyze how key decisions made by researchers, including the definition of the environmental commodity, housing market, and methodological choices affect the results from hedonic property value models. This paper focuses on hedonic models that examine the capitalization of water quality in housing values. Although several reviews of the hedonic property value literature exist (e.g., Boyle and Kiel, 2001; Kiel, 2006; Wilde et al., 2012), only Nicholls and Crompton (2018) review the hedonic literature focused specifically on water quality. Our study goes beyond their narrative review and empirically analyzes how primary study choices can affect the capitalization of water quality into home values. The resulting quantitative generalizations from our meta-analysis provide useful insights regarding the collective hedonic literature focused on water quality.

Meta-analysis uses a variety of statistical approaches to analyze previously reported scientific results and draw broader conclusions (Stanley, 2001; Nelson and Kennedy, 2009; Stanley and Doucouliagos, 2012). It can be used for purposes of benefit transfer, and/or to draw inferences from the collective body of literature (Boyle and Wooldridge, 2018). Our prior analysis of these meta-data focused on 18 hedonic studies that examined the relationship between house prices and water clarity (see Guignet et al., 2022). The expressed purpose was for benefit transfer. The focus for the current study is to draw inferences from the hedonic literature focused on water quality more broadly, using a larger meta-dataset of 29 studies, and that includes numerous measures of water quality (not just clarity).

The capitalization of water quality into waterfront home values may vary not only due to variations in the environmental commodity itself, but also variation in study design and methodological choices. A meta-regression model allows one to systematically examine how such variation impacts the primary study results. Often the dependent variable in a meta-regression model is the estimated effect size or summary statistic of interest from the primary studies (e.g., willingness to pay or elasticities). The dependent variable can also represent whether the effect size is significant and has a theoretically consistent or expected sign (e.g., Smith and Huang, 1993; Wehkamp et al., 2018; Ohlendorf et al., 2021). The inclusion of various independent variables representing different study characteristics such as methodology, data, and functional form, allow one to identify the effect of these choices on the primary study results (Stanley, 2001).

There are numerous applications of meta-analysis in the environmental economics literature, and in particular in the realm of non-market valuation (Nelson and Kennedy, 2009). Previous meta-analyses related to economic valuation also consider methodological choices, but many still focus on improving benefit transfer (e.g., Rosenberger and Johnston, 2009; Schütt, 2021; Guignet et al., 2022). Including methodological attributes in valuation meta-analyses have been shown to avoid problems of omitted variable bias (e.g., Johnston et al., 2006a; Johnston et al., 2006b; Stapler and Johnston, 2009).

Insight for our paper was provided by Smith and Huang (1993), who estimated meta-regressions examining the effects of air pollution on property values, Kuminoff et al. (2010) who examined how omitted variables affect hedonic models, and three recent “best practices” articles (Taylor, 2017; Bishop et al., 2020; Guignet and Lee, 2021). Smith and Huang (1993), who had the same objectives as this paper, evaluated how study characteristics related to finding a negative and theoretically consistent relationship between air pollution and property sales. They found that using actual sales price (as opposed to census data or assessed values), assuming a linear hedonic model, and using more than one air pollution measure, all decreased the likelihood of finding a significant and theoretically expected result. Market conditions, as measured by home vacancy rates, were also included in their meta-analysis. Kuminoff et al. (2010: p. 157) ran an internal meta-analysis with their set of simulations to assess how omitted variables affect marginal implicit price estimates. They suggest controlling for functional form, sample size, whether the primary study controls for omitted variables, and whether the study estimates time-constant implicit prices, are important when estimating a meta-regression.

The three “best practices” articles play an important role in ensuring we account for recommended modeling choices in our meta-analysis and allow us to compare the differences in results to those recommended practices. The objective of our meta-analysis is not to necessarily promote hedonic modeling choices that lead to significant and expected results. Rather, we aim to identify how commodity, market, and methodological choices made by researchers affect the collective results found in the hedonic property value literature.

We find that, all else constant, the decisions made by primary study researchers can greatly influence the estimated relationship between water quality and home prices. In particular, the results reveal robust evidence that methodological choices – like whether spatial dependence is accounted for, whether actual transaction prices are used, or if the water quality measure is based on in situ measurements – have a critical role in determining the likelihood of finding a significant and theoretically expected result; and perhaps most importantly, these findings do not always align with study decisions that reflect best practices. Our study, therefore, provides information that identifies potential concerns with data and modeling choices in hedonic studies focused on water quality.

2. Meta-dataset

2.1. Meta-dataset development and structure

The complete meta-dataset contains a comprehensive set of hedonic property value studies that examined objective measures of surface water quality in the US and were published or released between 1979–2017.¹ We developed the meta-dataset based on guidelines by Stanley et al. (2013), and the subsequent update by Havránek et al. (2020). Studies were not limited to a single or “preferred” observation (Boyle and Wooldridge, 2018); there are sometimes numerous observations from a single study. Such a clustered (or panel) structure and within study variation of specification choices is advantageous given our objective of identifying how primary study assumptions affect the hedonic results.

For the current study, we include property value studies examining a variety of water quality measures, not just water clarity (i.e., Secchi disk depth). This means we cannot focus on a common effect size, such as elasticity,² because the different water quality measures make it difficult to define a consistent metric. A 1% change in Secchi disk depth can mean something very different compared to a 1% change in chlorophyll a concentrations or fecal coliform counts. Therefore, price elasticities with respect to these different water quality metrics cannot necessarily be pooled and directly compared. At the same time, we lack enough observations (outside of water clarity) to constrain this analysis to only a single water quality measure.

Our protocol for identifying primary studies for inclusion in the meta-dataset is described in the supplementary material. In total, 65 studies in the peer-reviewed and gray literature were identified for potential inclusion. Focus was then drawn to the subset of studies that used an objective water quality measure, were inside the US, were not redundant with a later peer-reviewed publication (e.g., an earlier working paper), and were a primary hedonic study (i.e., not a literature review or another meta-analysis). These screening criteria led to the exclusion of 29 studies. The remaining 36 primary studies in the meta-dataset include both waterfront and non-waterfront observations. From the 36, we select the 29 primary studies and the 290 unique house price elasticity estimates that correspond to waterfront homes.³ This final set of studies includes results from both the published and gray literature. Twenty-one studies are from published peer-reviewed journals, three are master’s or PhD theses, two are government reports, one is a book chapter, one is from a presentation, and one is a working paper.

A variety of water quality measures have been examined in the hedonic literature; 17 different metrics are observed in our meta-data. For each measure, we have an estimate of the house price elasticity with respect to water quality, as well as an estimated standard error. Hedonic studies use a variety of functional forms, and hence report their findings in different ways (e.g., elasticities versus semi-elasticities). We convert the primary study coefficient estimates to common elasticities based on study specific model-by-model derivations. The corresponding standard errors are then computed based on Monte Carlo simulations. These simulations are performed separately for each of the 290 meta-observations across the 29 included studies.⁴

Some of the elasticities represent improvements in water quality, while the majority of elasticities relate to a degradation in quality (e.g., higher fecal coliform indicates lower water quality). The elasticities for Secchi disk depth, percent water visibility, and dissolved oxygen are all generally expected to be positive because increases in these three measures are often considered improvements. Table 1 lists each of the 17 water quality measures used in the primary studies, the expected sign of the relationships with house prices, the corresponding primary studies, and the number of observations pertaining to each water quality measure in our final meta-dataset.

Table 1.

Water quality measures, expected sign, list of primary studies, and number of meta-observations.

Water Quality Variable	Water Quality Measure in Hedonic Model	Expected Signa	Primary Studies	Observations
Clarity	Secchi disk depth	+	Ara (2007); Boyle et al., (1999); Boyle and Taylor (2001); Gibbs et al., (2002); Horsch and Lewis (2009); Hsu (2000); Kashian et al., (2006); Krysel et al., (2003); Liao et al., (2016); Liu et al., (2014); Michael et al., (2000); Olden and Tamayo (2014); Poor et al., (2001); Walsh et al., (2011a); Zhang and Boyle (2010); Zhang et al., (2015)	120
	Light attenuation	−	Guignet et al., (2017); Walsh et al., (2017)	57
	Percent water visibility	+	Bin and Czajkowski (2013)	2
Nutrients	Chlorophyll a	−	Liu et al., (2017); Walsh and Milon (2016); Walsh et al., (2011b)	18
	Nitrogen	−	Liu et al., (2014); Poor et al., (2007); Walsh and Milon (2016); Walsh et al., (2011b)	7
	Phosphorus	−	Liu et al., (2014); Walsh and Milon (2016); Walsh et al., (2011b)	6
	Trophic state index	−	Walsh and Milon (2016); Walsh et al., (2011b)	4
	Lake trophic state index	−	Feather et al. (1992)	2
Sediment	Total suspended solids	−	Netusil et al., (2014); Poor et al., (2007)	7
	Sediment	−	Liu et al., (2014); Yoo et al., (2014)	4
	Sedimentation rate	−	Bejranonda et al. (1999)	2
	Turbidity	−	Brashares (1985)	2
Bacteria	Fecal coliform	−	Ara (2007); Brashares (1985); Leggett and Bockstael (2000); Netusil et al., (2014)	36
	E. coli	−	Netusil et al. (2014)	5
Biochemical	Dissolved oxygen	+	Bin and Czajkowski (2013); Netusil et al., (2014)	10
	Temperature	−	Netusil et al. (2014)	6
	Salinity	−	Bin and Czajkowski (2013)	2

^aExpected Sign is based on the original study’s water quality measure and whether increases in the water quality measure generally denote a degradation or improvement.

Using the computed standard errors, we calculate whether the estimated elasticity for each meta-observation is significantly different from zero. With the expected sign of the elasticity based on the water quality measure and the estimated standard errors, we characterize whether each elasticity estimate is significant and of the theoretically expected sign (Smith and Huang, 1993; Wehkamp et al., 2018; Ohlendorf et al., 2021). Both the estimation of the standard errors and the critical value rely on the original study’s sample size. The conversion of water quality elasticities into a binary variable, Y, does admittedly result in a loss of information about the size of the effect. However, as discussed earlier, such a conversion is necessary in order to pool meta-observations based on very different measures of water quality, and ultimately to draw general conclusions from the broader literature.

The dependent variable in our main model uses p < 0.05 from a standard two-tailed t-test to determine if the estimated elasticity is statistically significant.⁵ About 52% of the estimated elasticities are considered significant and have the expected sign (151 observations). There are 129 observations that are statistically equal to zero (83 of those insignificant observations have the expected sign). The primary objective of our quantitative review is to examine under what conditions the hedonic model tends to yield the expected results, focusing specifically on the commodity attributes, market characteristics, and methodological choices discussed next.

2.2. Commodity, market, and methodological characteristics

While the definition of the water quality commodity and the market definition are often chosen by the researcher to match the primary study objectives, and are sometimes constrained by the availability of both water quality and property sales data, the methodological and estimation choices made by researchers can have an important influence on a given study’s results. Including methodological characteristics in the meta-analysis is motivated in part to help understand the importance of these decisions. The role of the publication process and potential associations with the likelihood of observing a statistically significant and expected result are also discussed.

Descriptive statistics can be found in Table 2. Many of the variables are binary indicators or dummy variables. The 29 studies included in the meta-data were published between 1985 and 2017 and provided elasticity estimates ranging from −2.64 to 8.32.

Table 2.

Definition of variables and summary statistics.

Variable	Mean	Standard Deviation	Minimum	Maximum
Elasticity	0.0876	0.7002	−2.6376	8.3202
Dependent variable:
Y (=1 if elasticity is significant with a p-value <0.05 and has the theoretically expected sign)	0.5207	0.5004	0	1
Environmental commodity variables:
Lake or Reservoira (=1 if observation is for lake or reservoir)	0.5724	0.4956	0	1
Estuary (=1 if observation is for estuary)	0.3379	0.4738	0	1
River (=1 if observation is for river)	0.0897	0.2862	0	1
Claritya (=1 if water quality measure in hedonic study relates to clarity, see Table 1)	0.6172	0.4869	0	1
Nutrients (=1 if water quality measure in hedonic study relates to nutrients, see Table 1)	0.1276	0.3342	0	1
Sediment (=1 if water quality measure in hedonic study relates to sediment, see Table 1)	0.0517	0.2219	0	1
Bacteria (=1 if water quality measure in hedonic study relates to bacteria, see Table 1)	0.1414	0.3490	0	1
Biochemical (=1 if water quality measure in hedonic study relates to biochemical, see Table 1)	0.0621	0.2417	0	1
Study area and market characteristics:
Northeasta (=1 if observation is from northeast regionb)	0.3034	0.4605	0	1
Midwest (=1 if observation is from midwest regionb)	0.1793	0.3843	0	1
West (=1 if observation is from west regionb)	0.1241	0.3303	0	1
South (=1 if observation is from south regionb)	0.3931	0.4893	0	1
Mean House Price (thousands, 2018$)c (average house price updated to 2018$)	317.9502	220.6630	8.0196	1245.9600
Single Subcountya (=1 if observation uses a sample constrained to county, but does not use all property sales)	0.7276	0.4460	0	1
Multiple Counties/Subcounties (=1 if observation uses a sample from multiple counties or multiple subcounties)	0.2724	0.4460	0	1
Sample Yearsc (number of years in the sample)	8.8586	5.0435	1	24
Bubble (=1 if sample includes any of the years 2006–2009)	0.3655	0.4824	0	1
Methodological variables:
Double-loga (=1 if observation comes from a double-log specification)	0.3172	0.4662	0	1
Log-lineara (=1 if observation comes from a log-linear specification)	0.2966	0.4575	0	1
Linear-loga (=1 if observation comes from a linear-log specification)	0.2759	0.4477	0	1
Log-quadratica (=1 if observation comes from a log-quadratic specification)	0.0448	0.2073	0	1
Linear (=1 if observation comes from a linear specification)	0.0655	0.2479	0	1
No Spatial Methoda (=1 if observation was derived from a model using no spatial method)	0.4138	0.4934	0	1
Spatial Fixed Effects (=1 if observation was derived from a model that included spatial fixed effects)	0.2897	0.4544	0	1
Spatial Lag (=1 if observation was derived from a spatial autoregressive model)	0.3276	0.4701	0	1
Spatial Autocorrelation (=1 if observation was derived from a spatial error model or allowed for clustered errors within a spatially defined group)	0.3276	0.4701	0	1
Assessed Housing Values (=1 if observation used assessed housing values)	0.0828	0.2760	0	1
Not In Situ (=1 if water quality measure was based on something other than in situ measurement)	0.3483	0.4772	0	1
More than One WQ Variable (=1 if observation included more than one water quality variable in the primary hedonic model)	0.2448	0.4307	0	1
Sample Sizec (original study sample size)	13,066.1	36,002.8	21	281,951
Publication characteristics:
Time Trend (year published)c (0 = 1985 to 32 = 2017)	24.1379	7.5817	0	32
Journal (=1 if observation was from a peer-reviewed journal article)	0.7310	0.4442	0	1

Unweighted descriptive statistics presented for n = 290 unique elasticity estimates in meta-dataset. Variables are binary indicator variables unless otherwise noted.

^(a)Denotes reference category.

^(b)Regions of the US are defined following the US Census Bureau’s four census regions. Accessed on April 14, 2023, at: https://www2.census.gov/geo/pdfs/maps-data/maps/reference/us_regdiv.pdf.

^(c)Denotes independent variables that are continuous.

2.2.1. Environmental commodity

To synthesize the water quality hedonic literature, we begin with the environmental commodity (i.e., the type of waterbody and water quality measure examined in the primary studies). The hedonic property value studies focusing on surface water quality tend to examine lakes or reservoirs (57%) as a group, identified as our reference category, compared to the variables Estuary or River.

For tractability, we organized the 17 different objective water quality measures in the original studies into five broader categories or variables – Clarity, Nutrients, Sediment, Bacteria, and Biochemical (see Table 1). Some categories are more likely to be directly observed by homebuyers, and others could be considered proxies for perceived water quality. Our variables are intended to group measures that reflect similar water quality issues and processes, and that also may be perceived similarly by homebuyers and sellers. The reference variable Clarity is the most common measure in this meta-dataset (62%), followed by Bacteria (14%) and then Nutrients (13%).

2.2.2. Study area and housing market characteristics

Most elasticities were estimated for housing markets and waterbodies in the south (39%), followed by the northeast region, midwest and west. Indicator variables for Midwest, West, and South are included in the meta-regression (Northeast is the reference category). In addition to the broad US regions, the characteristics defining the assumed housing market in a primary study may play an important role in whether a study yields the expected result. Taylor (2017), Bishop et al. (2020), and others emphasize the need to define a market, both geographically and temporally, by the “law of one price” – meaning that within the assumed market, identical housing bundles will sell for the same price. In other words, a single hedonic equilibrium price surface should apply throughout the entire housing market, and that equilibrium is not changing over the assumed time period and spatial definition of that market.

The housing market characteristics considered in our meta-analysis are the average house price, the spatial and temporal definitions of the market (i.e., whether multiple counties were pooled together and the number of years in the study period, respectively), and whether the sample years include the 2006–2009 housing market bubble burst. Using the consumer price index, we update the Mean House Price to 2018$ in each study based on the reported year or, if not reported, the last year of the sample.⁶ To assess the effects of spatial definitions on elasticities, we initially identify whether observations defined a market as a subcounty area, multiple subcounties, or multiple counties.⁷ Most observations were at the subcounty level (73%). We combined multiple counties and multiple subcounties (Multiple Counties/Subcounties) as the variable to represent market definition (Table 2). In order to examine the temporal definition of the market, we use the number of years in the sample. Our interest lies in whether hedonic results vary when estimated from sales data over longer study periods, and hence where the “law of one price” assumption is less plausible. The variable Sample Years has noticeable variation, ranging from 1 to 24 years.

Economists have discussed the implications of the 2006–2009 housing market bubble and burst on both hedonic methods and the interpretation of results (e.g., Boyle et al., 2012; Taylor, 2017; Bishop et al., 2020). The structural shifts that occurred during the housing market bubble and subsequent burst clearly affected market equilibriums. Hedonic models that are estimated with samples that included transactions both pre- and post-the market bubble burst, and that do not properly allow the entire hedonic surface to shift with that new equilibrium in their models, violate the “law of one price” and are theoretically invalid (Bishop et al., 2020). Defining the burst from 2006 to 2009, which matches Taylor’s (2017) definition, we create a dummy variable (Bubble) that is equal to 1 if any of the sample years include the bubble burst (about 37% of the observations).

2.2.3. Methodological characteristics

Independent variables that characterize methodological decisions can also lead to variation in the significance and the expected sign of elasticity estimates. The methodological variables examined include choices of functional form, methods to account for spatial dependence, whether actual transaction prices are used (as opposed to assessed values), and decisions about the water quality data and variables included in the hedonic models.

The choice of functional form leads to different interpretations of the coefficients, but there is still relatively little guidance on the appropriate functional form assumptions for hedonic price models. Cropper et al.’s (1988) seminal study suggested that simpler functional forms (e.g., log-linear) outperform more complex models in the presence of omitted variables, but more recently Kuminoff et al. (2010) found that flexible functional forms may perform better when combined with spatial and temporal fixed effects, and quasi-experimental methods. Otherwise, the only firm guidance is that a linear specification is generally not theoretically appropriate (Bockstael and McConnell, 2007; Taylor, 2017; Bishop et al., 2020). Most observations in the meta-data are based on a double-log specification, but linear-log and log-linear are also commonly used. The fewest observations come from linear and log-quadratic specifications (Table 2). We only include the variable Linear in our later meta-regressions because this is the only specification where there is clear consensus that it goes against best practices.

Spatial dependence occurs when primary study observations are correlated with geographically nearby observations. Depending on the nature of the spatial correlation, such dependence can lead to less precise, and even biased estimates from the primary study regression models (LeSage and Pace, 2009). Spatial dependence is a long-standing concern in the hedonic property value literature (Kuminoff et al., 2010; Guignet and Lee, 2021). A variety of approaches have emerged to address the issue, such as spatial econometric specifications (Anselin and Lozano-Gracia, 2009), quasi-experimental methods (Parmeter and Pope, 2013), and spatial fixed effects (Taylor, 2017; Guignet and Lee, 2021). With the exception of Olden and Tamayo (2014), who use an instrumental variable approach to address endogeneity concerns, no studies in our meta-data utilized quasi-experimental methods (e.g., difference-in-difference, regression discontinuity). About 41% of the observations did not use any approach to explicitly address spatial dependence (see Table 2), while 170 observations (59%) used some combination of spatial fixed effects, spatial lag models, and/or approaches to account for spatial autocorrelation.

Eighty-four observations were derived from models that included spatial fixed effects (e.g., neighborhood and watershed, town, city, or lake level). While no observation was based only on a spatial autoregressive model, which includes a spatial lag of price (LeSage and Pace, 2009), 95 observations were estimated from these models in combination with the other approaches. In addition, 95 observations accounted for spatial autocorrelation either through a formal spatial error model (LeSage and Pace, 2009) or allowing for clustered errors within a spatially defined group.⁸ The variables Spatial Fixed Effects, Spatial Lag, and Spatial Autocorrelation are compared to the reference No Spatial Method in our models.

Sales price data have become more accessible, either through private companies (e.g., CoreLogic, Zillow) or directly from county and state property assessor offices. However, data of assessed or predicted property values are sometimes easier to acquire and are available for a larger sample of homes. Although more comprehensive by not just reflecting homes that are sold, assessed or predicted values do not directly reflect market transactions, and hence revealed preferences. As identified by Bishop et al. (2020), the assessed values may have measurement error that in turn could affect the results from subsequent hedonic models. At the same time, Smith and Huang (1993) found that the use of actual sales prices reduced the likelihood of finding a significant and theoretically consistent elasticity. They suggested that this may occur because of higher variability or noise in actual sales data. Most observations in our meta-dataset are based on actual sales prices, but roughly 8% of the observations are based on assessed values (Assessed Housing Values = 1).

The hedonic literature uses a variety of approaches for acquiring measures of water quality, including: in situ measurements, spatial interpolation, model prediction, and satellite imagery (e.g., remote sensing). Most of the observations in the meta-data are from studies that used in situ measurement. Water quality measures based on spatial interpolated data were the second most common, followed by predicted measurements from water quality models. A relatively new approach for hedonic models that will certainly become increasingly common is the use of satellite-based measures (e.g., Wolf and Kemp, 2021; Zhang et al., 2022). During the period of our meta-data, however, Horsch and Lewis (2009) are the only ones to use water quality measures based on remote sensing data. We represent these approaches in the meta-regressions with the variable, Not In Situ.

Another methodological decision is how many water quality variables to include in the primary hedonic model. For our study, most observations included just one water quality variable. The remaining 24% used up to seven water quality variables in a single model, with most using either two or five (e.g., Bin and Czajkowski, 2013; Netusil et al., 2014; Walsh and Milon, 2016). We represent this decision in our meta-analysis with an indicator variable – More than One WQ Variable. In their meta-analysis of hedonic studies on air pollution, Smith and Huang (1993) found that as the number of air pollutant variables included in the hedonic model increased, the results were less likely to yield a statistically significant relationship with house prices.

Finally, studies with larger sample sizes will yield more precise estimates, all else constant. And so, if there is in fact a true effect, then as sample size increases, a primary study would be more likely to yield a statistically significant estimate of the expected sign. To account for this effect, we include the square root of the sample size (Sample Size) as an independent variable (see Stanley, 2001; Card et al., 2010; Wehkamp et al., 2018). The sample sizes across meta-observations range from just 21 to over 281,000 home transactions.

2.2.4. Publication characteristics

In addition to the study choices described above, we want to control for two publication characteristics. The first is a linear time trend variable that reflects the year of publication. As seen in Table 2, the variable Time Trend ranges 0 = 1985 through 32 = 2017. Such a trend variable may partially capture changes in methods, data availability and quality (e.g., water quality monitoring), the publication process and selection criteria, preferences, and perceptions of water quality over time that may affect significance and theoretical consistency of the primary study estimates. To complete the set of independent variables, we differentiate peer-reviewed journal articles from book chapters and gray literature with the dummy variable, Journal. This variable helps account for study quality, at least partially, because the peer-review process will generally result in improved methodologies and screen out subpar studies. On the other hand, this variable may also help account for possible publication selection bias. All else constant, reviewers and editors may be more likely to accept a study where a statistically significant effect is found, compared to a study yielding null results. We have attempted to minimize the publication selection bias through our approach for identifying relevant studies, by casting a wide net and identifying studies that are and are not published in the peer-reviewed literature.⁹ Following earlier studies, both publication variables also should help control for potential selection biases (e.g., Smith and Huang, 1993; Wehkamp et al., 2018; Ohlendorf et al., 2021).

3. Meta-regression analysis

Meta-regression analysis allows us to investigate how researcher decisions (x), such as the commodity and housing market to analyze, data choices, and methodological assumptions, influence the subsequent hedonic results. Our interest is in whether a primary study result is significant and of the expected sign based on intuition and theory – i.e., $Y=1$ . A meta-regression model can estimate the probability of $Y=1$ as: $Prob\left(Y=1|\mathit{x}\right)=F\left({\mathit{x}}^{\prime }\mathbf{\text{β}}\right)$ , where $\text{F}\left(\cdot \right)$ denotes a cumulative distribution function. Estimates of the coefficients $\mathbf{\text{β}}$ directly align with our study objectives by revealing how attributes of the commodity, housing market, and methodological choices lead to variation in the significance and expected sign of the estimated property value effects.

We start with the framework for Generalized Linear Models (GLM, McCullagh and Nelder, 1989; Wilson and Lorenz, 2015) where the dependent variable does not have to follow a normal distribution. Many common models fit under the GLM framework, and it provides an approach for estimating a linear relationship even if a dependent variable has a nonlinear relationship with its independent variables. An important assumption for the GLM is that all observations are independent. Because our meta-data sometimes contains observations from the same study or dataset, we use an extension of the GLM called the Generalized Estimating Equation which can account for the correlation within clusters (GEE; see Liang and Zeger, 1986; Zeger and Liang, 1986; Cameron and Miller, 2011).

Following Guignet et al. (2022), clusters are defined as unique study and housing market combinations, leading to a total of J = 98 clusters in the meta-data. Each cluster has a total number of observations defined as ${\text{N}}_{\text{j}}$ . ${\text{Y}}_{\text{ij}}$ is the binary outcome variable denoting whether the corresponding primary study elasticity estimate is significant and of the expected sign for observation $i$ in cluster $j$ . We define ${\text{p}}_{\text{ij}}$ as the probability that ${\text{Y}}_{\text{ij}}$ is equal to 1 and define a function that connects ${\text{p}}_{\text{ij}}$ to the linear predictor variables $({\mathit{x}}_{ij})$ . The inverse of this function defines the link function. For this study, the function $\text{Φ}\left(\cdot \right)$ is the standard normal cumulative distribution function for the population-averaged probit model (Cameron and Miller, 2011).

$$\begin{gathered} Prob\left(Y=1|{\mathit{x}}_{\mathit{ij}}\right)=E\left(Y_{ij}|{\mathit{x}}_{\mathit{ij}}\right)=p_{ij} \\ =\text{Φ}\left({\mathit{x}}_{\mathit{ij}}\text{'}\mathbf{\text{β}}\right) \end{gathered}$$ (1)

Estimating a standard probit model that ignores the clustered structure of our meta-data provides a simple approach for estimating $\mathbf{\text{β}}$ , provided that cluster-robust standard errors are also estimated (Cameron and Miller, 2011). However, other approaches exist to address the clustered nature of our data.

One approach is to estimate a cluster-specific model that uses the standard probit but adds a cluster-specific variable, ${\text{α}}_{\text{j}}$ , such that

$$Prob\left(Y=1|{\mathit{x}}_{\mathit{ij}},{\text{α}}_j\right)=p_{ij}=\text{Φ}\left({\mathit{x}}_{\mathit{ij}}^{\text{'}}\mathbf{\text{β}}+{\text{α}}_j\right)$$ (2)

where ${\text{α}}_{\text{j}}$ can be estimated as a random or fixed effect (Cameron and Miller, 2011). However, we are less interested in results for specific clusters and more interested in the average, or population, effects of hedonic study choices on significance and theoretical consistency.

The GEE, using a quasi-likelihood methodology that requires few assumptions about the distribution of $Y$ , provides greater flexibility in identifying the correlation structure within clusters, and provides population-averaged results (Liang and Zeger, 1986; Zeger and Liang, 1986; Cameron and Miller, 2011). The family of GEE models have rarely been used in environmental economics, but there are a few examples (e.g., Johnston et al., 2002; King and Anderson, 2004). With GEE models, a cluster-specific variable is not specified (as in Eq. (2)), so we estimate (Eq. (1)), but do not ignore $j$ . Instead, expectations are defined for the jth cluster

$$E\left({\mathit{Y}}_j|{\mathit{x}}_j\right)={\mathit{p}}_j\left(\text{β}\right)$$ (3)

where ${\text{p}}_{\text{j}}=\left[{\text{p}}_{\text{i,1}},\dots ,{\text{p}}_{\text{i,Nj}}\right]$ is the marginal expectation of ${\text{Y}}_{\text{j}}$ . Using the quasi-likelihood method, the set of GEE parameters, $\text{β}$ , solves (Pendergast et al., 1996; Cameron and Miller, 2011):

$$S\left(\mathbf{\text{β}}\right)={\sum }_{j=1}^J\frac{\partial {\mathit{p}}_{j^{\prime }}}{\partial \mathbf{\text{β}}}{V}_j^{-1}\left({\mathit{Y}}_j-{\mathit{p}}_j\left(\mathbf{\text{β}}\right)\right)=0$$ (4)

If this were the GLM with independent observations within a cluster, we would have the variance matrix $V_j=A_j$ , where ${\text{A}}_{\text{j}}$ is a diagonal matrix of variances of ${\text{p}}_{\text{ij}}$ as the jth diagonal element (Liang and Zeger, 1986; Pendergast et al., 1996).

Unlike the GLM, Liang and Zeger (1986) broadened the choices of correlation possibilities within clusters with ${\text{R}}_{\text{j}}\left(\text{α}\right)$ , defined as the working correlation matrix. ${\text{A}}_{\text{j}}$ is again the diagonal matrix of variances and $\text{φ}$ is a scale parameter.

$$V_j=A_j^{\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$2$}\right.}{R}_j\left(\mathit{\alpha }\right)A_j^{\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$2$}\right.}/\varphi$$ (5)

The matrix ${\text{V}}_{\text{j}}$ for cluster $j$ is what differentiates the GEE model from the GLM (Pendergast et al., 1996).

Although no specific approach exists for identifying the correct correlation structure, a number of choices can be tested (Zorn, 2001). The working correlation matrices typically used include: independence, exchangeable, unstructured, and user-defined matrices (see Wilson and Lorenz, 2015). The independence correlation matrix assumes observations within the same cluster are not correlated. The exchangeable matrix assumes observations within the same cluster have the same correlation. If an unstructured matrix is chosen, each pairwise correlation is estimated, but having too many observations in a cluster or having unbalanced clusters can cause problems with the model (e.g., Shults et al., 2009). The more representative the working correlation structure is of the data, the more efficient the estimators. An incorrect choice of the working correlation structure does not affect the asymptotic consistency of the estimators as J→∞, but it can affect the consistency of the variance estimate (Zorn, 2001). Therefore, a cluster-robust estimate of the variance-covariance matrix is almost always recommended because it is consistent as long as J→∞ is met (Liang and Zeger, 1986; Cameron and Miller, 2011). A potential limitation of the GEE, then, is when the number of clusters is small (Ballinger, 2004).

Another potential limitation of the GEE is testing model fit (Zorn, 2001; Ballinger, 2004). Because the GEE uses quasi-likelihood, and not maximum likelihood like the GLM, an alternative approach for evaluating relative model performance is needed. Pan (2001) developed the quasi-likelihood for independence criterion (QIC) which is similar to the Akaike Information Criterion (see also Hardin and Hilbe, 2003). The smallest QIC can help identify the appropriate working correlation structure and best model fit. A simplification of the QIC, the QIC_u – which substitutes in a penalty for the number of parameters – can also be used, but only to identify the appropriate set of variables (Pan, 2001). We use a probit link function and test independence and exchangeable working correlation matrices using the QIC. Once the appropriate correlation structure is identified, we choose the preferred set of variables using the QIC and QIC_u.

As described earlier, ${\text{p}}_{\text{ij}}$ is the probability that ${\text{Y}}_{\text{ij}}$ is equal to one. Similar to Rosenberger and Johnston (2009), the linear predictor variables described above can be divided into four vectors, $q_{ij}$ , $m_{ij}$ , $z_{ij}$ , and $d_{ij}$ , along with the corresponding coefficients, $\mathbf{\text{β}}$ , so that we have the function:

$$p_{ij}=\text{Φ}\left({\mathit{q}}_{ij}{\mathbf{\text{β}}}_q+{\mathit{m}}_{ij}{\mathbf{\text{β}}}_m+{\mathit{z}}_{ij}{\mathbf{\text{β}}}_z\right)$$ (6)

The vector $q_{ij}$ denotes variables describing the commodity (i.e., type of waterbody and water quality measure). The vector $m_{ij}$ represents market characteristics such as the average home price, study region and spatial and temporal definitions of the market. The vector $z_{ij}$ represents methodological choices in the primary study model (e.g., use of assessed housing values, functional form, accounting for spatial dependence, approach for acquiring measures of water quality). The last vector $d_{ij}$ denotes the publication characteristics, type of publication and time trend for the year published. Estimates of ${\mathbf{\text{β}}}_q$ , ${\mathbf{\text{β}}}_m$ , ${\mathbf{\text{β}}}_z$ ), and ${\mathbf{\text{β}}}_d$ will provide insight as to how primary study choices regarding the environmental commodity, housing market, methodology, and publication characteristics, respectively, affect the probability that a hedonic study yields results that are statistically significant and consistent with expectations. Table 2 displays the variables and definitions for each category.

4. Results

4.1. Probit Generalized Estimating Equation results

We estimate separate probit GEE meta-regressions for the independence and exchangeable correlation structures using SAS 9.4 (SAS Institute 2013). Our first GEE model tests independence correlation structure which assumes there is no correlation within each cluster but estimates cluster-robust standard errors. We use the 98 unique study-by-housing market combinations to define the clusters across the 290 observations. The results are presented in Table 3. Model (1) includes the vector q_ij that represents the variables describing the type of water body and water quality measures examined, but none of the variables are significant.

Table 3.

GEE probit meta-regression results (independence correlation structure).

Empty Cell	Variable	(1)	(2)	(3)	(4)	(5)
	Intercept	0.16	0.18	0.44	0.58	0.92
		(0.204)	(0.253)	(0.365)	(0.444)	(0.708)
Environmental commodity variables
	River	−0.25	0.06	0.21	3.41***	3.49***
		(0.497)	(0.651)	(0.630)	(1.186)	(1.235)
	Estuary	−0.39	−1.62***	−1.05**	−0.97	−1.01
		(0.340)	(0.359)	(0.501)	(0.741)	(0.765)
	Nutrients	0.46	0.44	0.90	2.27***	2.06***
		(0.472)	(0.369)	(0.651)	(0.798)	(0.769)
	Sediment	−0.49	−0.26	−0.78	−0.62	−0.56
		(0.577)	(0.613)	(0.544)	(0.653)	(0.644)
	Bacteria	0.33	0.43	0.09	0.38	0.39
		(0.641)	(0.515)	(0.394)	(0.268)	(0.315)
	Biochemical	−0.64	−0.50	−0.96	−0.24	−0.22
		(0.605)	(0.629)	(0.806)	(0.684)	(0.697)
Study area and market characteristics
	Midwest		−0.35	0.11	−0.48	−0.85*
			(0.405)	(0.447)	(0.628)	(0.505)
	West		−0.59	−0.21	−1.97***	−2.16***
			(0.595)	(0.590)	(0.614)	(0.721)
	South		1.35***	1.51***	0.17	0.16
			(0.355)	(0.524)	(0.700)	(0.752)
	Mean House Price (thousands, 2018$)			−1.00E-04	1.90E-03**	1.90E-03**
				(0.001)	(0.001)	(0.001)
	Sample Years			−0.08**	−0.15**	−0.18**
				(0.040)	(0.061)	(0.073)
	Multiple Counties/Subcounties			0.48	0.36	0.39
				(0.405)	(0.477)	(0.452)
Methodological variables
	Linear				0.10	0.33
					(0.346)	(0.312)
	Spatial Fixed Effects				−1.35***	−1.34***
					(0.380)	(0.376)
	Spatial Lag				−0.73**	−0.74**
					(0.308)	(0.319)
	Spatial Autocorrelation				−0.71**	−0.67**
					(0.339)	(0.335)
	Assessed Housing Values				1.90***	2.02***
					(0.639)	(0.769)
	Not In Situ				1.88***	1.97***
					(0.455)	(0.502)
	More than One WQ Variable				−1.29**	−1.29**
					(0.568)	(0.571)
	Sample Size (square root)				0.01***	0.01***
					(0.002)	(0.002)
Publication characteristics
	Time Trend (year published)					0.01
						(0.031)
	Journal					−0.46
						(0.603)
	Observations	290	290	290	290	290
	QIC	436.12	398.19	391.38	334.81	340.95
	QICu	395.99	369.06	361.42	318.40	321.37

Dependent variable: Y based on p < 0.05.

^***p < 0.01

^**p < 0.05

^*p < 0.1.

Cluster-robust standard errors in parentheses; clustered according to the J = 98 study-housing market combinations.

Model (2) adds variables denoting the study region of the US. For studies that estimated hedonic models for estuaries, the coefficient is now negative and significant. This suggests that compared to lakes/reservoirs, hedonic studies of estuaries are less likely to yield statistically significant results with the expected sign. Studies of waterbodies and housing markets in the south tend to be more likely to yield significant and theoretically expected results, relative to studies in the northeast region (the omitted category). None of the variables representing the categories of water quality are statistically significant, suggesting that, at least in this model, hedonic studies of the various water quality categories are equally as likely to yield the expected result (or not), all else constant.

Model (3) in Table 3 adds the remaining market definition variables from vector m_ij. The previous findings regarding estuaries and the south region remain robust, as does the finding that the types of water quality variable are statistically insignificant. The market characteristic variables – Mean House Price and Multiple Counties/Subcounties – are insignificant. However, Sample Years is significant and negative, suggesting that as the study period (and hence the duration of the assumed hedonic equilibrium) increases in length, the likelihood of the estimated elasticity being insignificant and/or theoretically inconsistent increases. This result is robust in subsequent meta-regression models that control for methodological features of the primary study.

Model (4) in Table 3 adds the vector representing methodological choices. We see large variability in the results. For example, the coefficient corresponding to River is now positive and significant, suggesting that hedonic studies examining rivers are more likely to yield the expected results, all else constant. The variable Nutrients is also now significant, suggesting that studies examining the impact of nutrient pollution on house prices have a higher likelihood of yielding an elasticity estimate that is significant and of the expected sign, compared to studies of water clarity (the omitted category). In Model (4), the evidence suggests that hedonic studies of waterbodies and housing markets in the west are less likely to yield the expected results (compared to studies of the northeast). In addition to Sample Years being negative and significant, another market characteristic, Mean House Price, is positive and significant – perhaps suggesting that studies of higher-end housing markets are more likely to find the expected results.

Seven of the eight methodological variables in Model (4) are significant, emphasizing the importance of primary researchers’ data decisions and modeling assumptions. For example, not using in situ water quality data (Not In Situ) and not using actual sales prices (Assessed Housing Values) lead to a higher probability of a study yielding statistically significant results that are of the expected sign. These methodological choices may reduce variability in the data, possibly facilitating more precise estimates in the primary hedonic studies (Smith and Huang, 1993). We emphasize, however, that finding the expected result does not necessarily imply the correct result. Including more than one water quality variable in the model leads to a higher likelihood that the elasticity will be insignificant and/or have an unexpected sign. This is also in line with Smith and Huang’s (1993) meta-analysis of hedonic studies on air quality. Our initial modeling approach for examining the role of controlling for spatial dependence on primary study results was to include a dummy variable that was equal to 1 if any spatial modeling approach was used. Because the coefficient was negative and significant, we were interested to see how the different approaches for modeling spatial dependence (i.e., Spatial Fixed Effects, Spatial Lag, and Spatial Autocorrelation) affect significance and the sign of the elasticity estimates. Somewhat surprisingly, the corresponding coefficients for all variables representing spatial dependence modeling approaches are significant and negative. Again, this suggests that studies are less likely to find the expected result if they control for spatial dependence. Sample Size is positive and significant, suggesting that the larger the sample size, the more likely it is that a study will yield statistically significant results that are of the expected sign. This is again consistent with Smith and Huang (1993).

Comparing Model (4) to Model (5), we see very little change in the results. The coefficients on the added publication time trend and Journal variables in Model (5) are insignificant, suggesting that, all else constant, the likelihood of hedonic studies yielding significant results of the expected sign is not different for peer-reviewed journal articles compared to other publication types, and has not changed over time. Although not perfect, perhaps this alleviates some concern regarding potential publication biases in this branch of the hedonic literature.

The same five models are re-estimated using the probit link function and exchangeable correlation structure (Table S1 in online supporting information). When comparing results that use different correlation structures, the model yielding the smallest QIC is the one that best fits the meta-data. Across the board, the independence correlation structure models in Table 3 yield the lowest QIC.¹⁰ When identifying the most appropriate model within a correlation structure, the smallest QIC_u can also be used. For those models using the independence structure, Model (4) (in Table 3) appears to best fit the meta-data.

Although not reported, we also ran Models (3), (4), and (5) with the dummy variable equal to 1 if any of the sample years included the housing market bubble burst (2006–2009). The results were very similar to those already presented in Table 3, but Bubble was statistically insignificant.

4.2. Robustness checks

Robustness checks on our results for an elasticity that is significant at the 5% level can be found in the online supplementary material. The robustness models use alternative dependent variable definitions. The first is a more stringent definition, where ${\text{Y}}_{\text{ij}}=1$ only if the elasticity was significant with a p-value less than 0.01 and had a sign consistent with economic theory, and the second is less stringent, where ${\text{Y}}_{\text{ij}}=1$ if the primary study elasticity estimate is of the expected sign and statistically significant based on a p-value less than 0.10. For the more stringent dependent variable we have 112 observations where ${\text{Y}}_{\text{ij}}=1$ , and for the less stringent dependent variable we have 173 observations where ${\text{Y}}_{\text{ij}}=1$ . These can be compared to our main dependent variable definition, where 151 observations had ${\text{Y}}_{\text{ij}}=1$ (see Table S2 for summary statistics). However, we drop the robustness model based on the 10% critical significance level because two of the independent variables, Nutrients and Assessed Housing Values, become nearly perfect predictors.

We identify the results from Model 4 in Table 3 as most appropriate and compare Y to Y_0.01 (see Table S3 in online supplementary material). Out of the eight methodological variables, only Linear, Spatial Lag, and Spatial Autocorrelation differ. For the study area and market characteristics variables, only Sample Years is significant across both models. For the environmental commodity variables, the results are not robust across different definitions of the dependent variable.

5. Discussion

The primary objective of this study was to answer the following question:

What does the hedonic literature examining surface water quality generally reveal about how the type of commodity, market characteristics, methodological decisions, and publication characteristics affect the significance and theoretical consistency of the estimated property value impacts?

The results from the GEE meta-regression models provide evidence that several of the definitions and decisions made in primary studies do affect the estimated relationship between water quality and property value impacts. In particular, our meta-analysis indicates that methodological choices play a critical role in whether a hedonic study will produce results that suggest that water quality impacts home values in the expected way.

For many of the independent variables, we were unsure a priori as to what the estimated impact would be on the probability of the expected price effect. For those variables where we were able to hypothesize the effect, we did not always find the expected result. Smith and Huang (1993) hypothesized and confirmed that both linear specifications and hedonic models that included more than one air pollution measure would lead to insignificant or inconsistent results. In contrast, in the context of water quality we find a null effect with respect to assuming a linear model; although this may be due to the limited number of studies assuming this subpar functional form. On the other hand, when more than one water quality measure was included in a hedonic model, we find a negative and significant result, which agrees with Smith and Huang’s (1993) meta-analysis.

When considering the commodity definition (i.e., waterbody type and water quality category), we find that the results are somewhat sensitive across the different meta-regressions. Limited observations for some waterbody types and specific water quality measures may be one reason. Publication selection bias also may play a role in the results related to the waterbody types. For example, holding all else constant, a study about a river may only be published if it has a significant result.

One market factor that we thought might be important was the 2006–2009 housing market bubble burst. Although we find no significant effect, how housing market expansions and contractions might impact implicit price estimates of interest should be examined more closely in future research.

Generally, methodological choices appear to have a very important and consistent role in determining the estimated relationship between water quality and housing prices. The use of assessed housing values and predicted or modeled water quality data lead to a similar, higher likelihood of finding a significant estimated price impact that is of the expected sign. Actual housing prices and in situ measurements may have more random variation in the data, which could obscure the true effects researchers are trying to estimate. On the other hand, assessed values do not directly reflect market behavior, and modeled water quality values can introduce prediction error.

Nonetheless, assessed values and predicted water quality measurements may sometimes be necessary, especially with recent trends in the hedonic property value literature to analyze increasingly large, even national, study areas (e.g., Moore et al., 2020; Zhang et al., 2022; Mamun et al., 2023). Some states do not disclose or only provide limited housing market data (e.g., Nolte et al., 2023), and in such cases assessed values may be needed to fill data gaps. In addition, finding consistent water quality measures across the country is difficult, meaning that studies going forward will likely rely heavily on modeled water quality measures or data generated from algorithms and satellite imagery, rather than in situ measurements. Although best practices for hedonic modeling should be followed (Bishop et al., 2020), data limitations may require solutions for missing observations. When feasible, primary study researchers could consider comparing assessed housing values with market prices or comparing in situ water quality measurements with predicted or modeled water quality data (Wolf and Kemp, 2021). Such within-study comparisons would provide a firmer understanding of whether such data choices affect the estimated property value effects, as our meta-analysis suggests.

Given the importance of trying to minimize spatially correlated omitted variable bias in hedonic property value models, we paid particular attention to methodological choices meant to account for spatial dependence – e.g., spatial econometric specifications (Anselin and Lozano-Gracia, 2009), quasi-experimental methods (Parmeter and Pope, 2013), and spatial fixed effects (Taylor, 2017; Guignet and Lee, 2021). As noted above, our meta-dataset did not include some of the more novel types of analyses that account for spatial dependence. We find that controlling for spatial dependence is actually associated with a decrease in the likelihood of a primary study yielding a significant result of the expected sign. Although speculative, one possible explanation is that the true water quality price effects are often relatively small and controlling for spatially correlated confounders better identifies that near zero effect. On the other hand, if the role of spatial dependence in the true data generating process is minimal, then spatial fixed effects and other spatial modeling approaches may be over-parameterizing the models, making it less likely that a study would identify a significant effect if there is one. It is also possible that some approaches to address spatial dependence, such as spatial fixed effects, may be absorbing much of the price variation of interest (Abbot and Klaiber, 2011). Variation in housing prices could be more due to spatial, rather than temporal, variation in water quality (Kung et al., 2022). In such cases, it is more difficult to isolate variation due to water quality from the spatially correlated omitted variables; making it less likely that one would find the expected result.

For the 120 observations that were derived from models where no spatial methods were implemented, 78 (65%) were significant and matched expectations. It is possible that no spatial dependence was found in some of these cases; in such instances finding the expected result is reasonable. Two studies (out of the 15 that do not use spatial methods in some or all of their models) test for spatial autocorrelation and do not find it in their data (Feather et al., 1992; Liao et al., 2016). Ten studies do not mention spatial dependence in their papers, suggesting that the elasticity estimates could potentially be biased, or at least inefficient depending on the nature of the spatial dependence (e.g., Nelson, 2008; Chi and Zhu, 2020). The last three studies use spatial methods in some of their observations as a comparison to observations that do not. The negative sign in our meta-regressions that corresponds to estimates from models that did account for spatial autocorrelation suggests that standard errors could be underestimated when primary studies do not account for spatial dependence in the hedonic model. Testing for spatial dependence and using spatial methods, when appropriate, is generally considered best practice.

Estimating the effects of water quality on house prices can be challenging because researchers are attempting to statistically isolate relatively small effects from numerous other correlated factors that affect house prices. As such, a greater sample size is generally desirable in order to obtain more precise estimates. Consistent with this notion, we find that sample size is associated with an increased likelihood of estimating a statistically significant price effect that is of the expected sign. At the same time, obtaining a larger sample by expanding the study period, in and of itself, may not be desirable. All else constant, our meta-regression results suggest that an increase in sample years is associated with a decrease in the probability of estimating a significant and theoretically consistent result. This finding is in line with research emphasizing the importance of accounting for shifts in the hedonic price equilibrium over time (e.g., Kuminoff and Pope, 2014; Bishop et al., 2020; Banzhaf, 2021).

6. Conclusion

Hedonic property value methods represent a large and growing branch of the nonmarket valuation literature. As we move forward and continue to apply and advance the methodology, it is important to look back and take stock on what has been done and the empirical implications of past analyses and modeling decisions. Our meta-analysis attempts to do just that, by systematically and quantitatively reviewing the hedonic property value literature on the price effects of water quality.

With the intention of providing information to assess hedonic models and the estimated price effects, we highlight three key points. First, our meta-regression results are limited by the existing literature, so we encourage researchers to fill in areas where the literature is scarce. In the context of the US, more hedonic property value studies examining water quality in the west and midwest regions, and for rivers and estuaries, are needed. Similarly, more studies including different water quality measures will allow separate meta-regressions to examine whether the impact of methodological choices on estimated property value effects vary across those measures.

Second, we demonstrate, across all meta-regressions, that study design choices and modeling assumptions have a large influence in determining the estimated price effects. To help assess the methodological decisions made in past studies, our study included observations that did not follow current best practices. As more studies are published or made available as working papers, more novel techniques such as quasi-experiments should be added to the meta-dataset.

The fact that methodological choices can have a greater and more systematic influence on the results than characteristics of the housing market and commodity itself is eye-opening and leads to our third and final point. We unequivocally recommend that researchers continue to follow contemporary guidance in hedonic modeling (Bishop et al., 2020). Our meta-regression results suggest that practices currently considered to be subpar in many applications, like using assessed housing values and not accounting for spatial dependence, may increase the tendency for a hedonic analysis to yield the hypothesized result. We caution that the expected result is not necessarily the correct result, and researchers should continue to assess the robustness of their findings against their study design and methodological choices.

Data availability

The authors have shared the link to their data in the manuscript.