Targeting the Poor: Evidence from a Field Experiment in Indonesia

Abstract

This paper reports an experiment in 640 Indonesian villages on three approaches to target the poor: proxy-means tests (PMT), where assets are used to predict consumption; community targeting, where villagers rank everyone from richest to poorest; and a hybrid. Defining poverty based on PPP$2 per-capita consumption, community targeting and the hybrid perform somewhat worse in identifying the poor than PMT, though not by enough to significantly affect poverty outcomes for a typical program. Elite capture does not explain these results. Instead, communities appear to apply a different concept of poverty. Consistent with this finding, community targeting results in higher satisfaction.

I. Introduction

Targeted social safety net programs have become an increasingly common tool to address poverty (Coady, Grosh and Hoddinott, 2004). In developed countries, the selection of the beneficiaries for these programs (“targeting”) is frequently accomplished through means-testing: only those with incomes below a certain threshold are eligible. However, in developing countries, where most potential recipients work in the informal sector and lack verifiable records of their earnings, credibly implementing a conventional means test is challenging.

Consequently, in developing countries, there is an increased emphasis on targeting strategies that do not rely on directly observing incomes. In particular, there are two main types of strategies that we consider in this paper: proxy means tests (PMTs) and community-based targeting.¹ In a PMT, which has been used in the Mexican Progresa/Oportunidades and Colombian Familias en Acción programs, the government collects information on assets and demographic characteristics to create a “proxy” for household consumption or income, and this proxy is in turn used for targeting. In community-based methods, such as the Bangladesh Food-For-Education program (Galasso and Ravallion, 2005) and the Albanian Economic Support safety net program (Alderman, 2002), the government allows the community or some part of it (e.g. local leaders) to select the beneficiaries. Both methods aim to address the problem of unobservable incomes. In the PMTs, the presumption is that household assets are harder to conceal from government surveyors than income; in community-based targeting, the presumption is that wealth is harder to hide from ones’ neighbors than from the government.

The choice between the two approaches is generally framed as a tradeoff between the better information that communities might have versus the risk of elite capture in the community process. By focusing on assets, PMTs capture the permanent component of consumption. In the process, however, they miss out on transitory or recent shocks. For example, a family may fall into poverty because one of its members has fallen ill and cannot work, but because the family has a large house, a PMT may still classify it as non-poor. Neighbors, on the other hand, may know the family's true situation, by regularly observing the way that they live.² If the community perceives that the PMT is wrong, a lack of legitimacy and political instability may ensue.³

However, while community targeting allows for the use of better local information, it also opens up the possibility that targeting decisions may be based on factors beyond poverty as defined by the government. This may be due to genuine disagreements about what “poverty” means: the central government typically evaluates households based on consumption, whereas the utility function used by local communities may include other factors, such as a household's earning potential, non-income dimensions of poverty, or its number of dependants.⁴ Or, the two groups may place a different weight on the same variable when predicting consumption. Moreover, the community process could also favor the friends and relatives of the elites, resulting in a lack of legitimacy with the process.

Given the tradeoffs involved, which method works best is ultimately an empirical question. If elite capture of community targeting is important, then the PMT could dominate community targeting either based on the government's consumption-based metric or a more holistic welfare metric, since the PMT limits the opportunity for capture. If better local information is important, then community targeting could dominate the PMT on both of these metrics. If a different local conception of welfare is empirically important, then the PMT may best match the government's consumption-based metric, while community targeting may work best based on alternative welfare metrics.⁵ In this paper, we use randomized evaluation techniques to compare PMT targeting with methodologies that allow for varying degrees of community inclusion in the decision-making process. We first compare how the methods perform from the perspective of the central government: poverty as measured by per capita expenditures and satisfaction with the targeting process. To understand why the methods produce different results, we then investigate the tradeoffs discussed above along four dimensions: elite capture, the role of effort, differences in information, and different conceptions of poverty.

In 640 villages in Indonesia, we conducted a field experiment in collaboration with the government. The government, through the Central Bureau of Statistics, implemented a cash transfer program that sought to distribute 30,000 Rupiah (about $3) to households that fell below location-specific poverty lines. In a randomly selected one-third of the villages, the government conducted a PMT in order to identify the beneficiaries. In another third of these villages, chosen at random, it employed community targeting (henceforth, the “Community Method”): the community members were asked to rank everyone from richest to poorest during a meeting, and this ranking determined eligibility. In the remaining villages, it used a combination of the two methods (henceforth, the “Hybrid”): communities engaged in the ranking exercise, and then the ranks were used to limit the universe of households whom the government would survey. Eligibility was then determined by conducting PMT on this limited list. This hybrid aimed to utilize the communities’ knowledge, while using the PMT as a check on potential elite capture.

We begin by evaluating the methods from the perspective of the central government, i.e. which method best targeted the poor based on consumption-based poverty and which method produced the highest satisfaction with the beneficiary list. We conducted a baseline survey that collected per capita expenditure data from a set of households prior to the experiment and then defined a household as poor if it fell below the PPP$2 per day cutoff. We find that both the community and hybrid methods perform worse than the PMT on this metric: in both methods, there was a 3 percentage point (10 percent) increase in the error rate based on consumption (which we will call “error rate” from now on for conciseness) relative to the PMT. The community-based strategies actually do as well (if not better) at finding the very poor.

On net, the differences in targeting accuracy across the methods are not large; for example, for a typically-sized transfer program in Indonesia, simulations suggest that these different targeting methods would not yield significantly different effects on reducing the poverty rate in Indonesia. Finally, we find that the results are similar in both urban and rural locations, in villages with greater or less inequality, and with greater or less levels of social connectedness; this suggests that the results may be generalizable along these dimensions.

Despite the somewhat worse targeting outcomes based on consumption, the community methods resulted in higher satisfaction and greater legitimacy of the process along all of the dimensions that we considered. Community targeting resulted in 60 percent fewer complaints than the PMT, and there were many fewer difficulties in distributing the actual funds in the community treatment villages. When asked ex-post about the targeting results, villagers in community treatment villages suggested fewer modifications to the beneficiary list.

We next turn to understanding why the community methods may differ from the PMT. We consider four dimensions: elite capture, community effort, local concepts of poverty, and information. To test for elite capture in the community treatment, we randomly divided the community and hybrid villages so that, in half of these villages, everyone in the community was invited to participate in the ranking meeting, whereas in the other half, only the “elites” (i.e. local community leaders such as the sub-village head, teachers, religious leaders, etc.) were invited. In addition, we gathered data in the baseline survey on which households were related to the local elites. We find no evidence of elite capture. The error rates were the same, regardless of whether only the elites attended the meeting. Moreover, we find no evidence that households that are related to the elites are more likely to receive funds in the community treatments relative to the PMT. In fact, we find the opposite: in the community treatments, elites and their relatives are much less likely to be put on the beneficiary list, regardless of their actual income levels.

To examine the role of effort, we randomized the order in which households were considered at the meetings. This allows us to test whether the effectiveness of community targeting differs between households that were ranked first and those ranked last (when fatigue may have set in). We find that effort matters: at the start of the community meeting, targeting performance is better than in the PMT, but it worsens as the meeting proceeds.

To examine the role of preferences and information, we studied alternative metrics of evaluating perceptions of poverty from our baseline survey. First, we asked every survey respondent to rank a set of randomly chosen villagers from rich to poor (henceforth, “survey ranks”). Second, we asked the head of the sub-village to conduct the same exercise. Finally, and perhaps most importantly, we asked each household that we interviewed to subjectively assess its own welfare level. We find that the community treatment produces a ranking of villagers that is much more correlated with these three alternate metrics than the ranking produced by PMT. In other words, the community treatments moved the targeting outcomes away from a ranking based purely on per-capita consumption and towards the rankings that one would obtain by polling different classes of villagers or by asking villagers to rate themselves.

There are two ways of explaining these findings: either the community has less information about different household's per-capita consumption than the PMT, or the community's conception of poverty is different from that based solely on per-capita consumption. The evidence suggests that the latter theory predominantly drives the results. First, even controlling for all variables in the PMT, the community members rankings of other households in the village contain information about those households’ per-capita consumption, which shows that community members have residual information about consumption beyond that contained in the PMT variables. Second, when we investigate how the survey ranks differ from consumption, we find that communities place greater weight on factors that predict earnings capacity than would be implied by per-capita consumption. For example, conditional on actual per capita consumption, the communities consider widowed households poorer than the typical household. The fact that communities employ a different concept of poverty explains why community targeting performance might differ from the PMT, as well as why it results in greater satisfaction levels.

The paper proceeds as follows. We discuss the empirical design in Section II, and we describe the data in Section III. In Section IV, we compare how each of the main targeting methods fared in identifying the poor. Section V tests for evidence of elite capture, while Section VI aims to understand the role of effort. In Section VII, we test whether the community and the government have different maximands. Section VIII explores the differences in the community's maximand in greater depth. Section IX concludes.

II. Experimental Design

II.A. Setting

This project occurred in Indonesia, which is home to one of the largest targeted cash transfer programs in the developing world, the Direct Cash Assistance (Bantuan Langsung Tunai, or BLT) program. Launched in 2005, the BLT program provided transfers of about US $10 per month to about 19.2 million households during periods of economic crisis. The targeting in this program was accomplished through a combination of community-based methods and proxy-means tests. Specifically, the Central Statistics Bureau (Badan Pusat Statistik, or BPS) enumerators met with neighborhood leaders to create a list of households who could potentially qualify for the program. The BPS enumerators then conducted an asset survey and a PMT for the listed households.

Targeting has been identified by policymakers as one of the key problems in the BLT program. Comparing with the goal of targeting the poorest one-third of households, the World Bank estimates that 45 percent of the funds were incorrectly provided to non-poor households and 47 percent of the poor were excluded from the program in 2005-2006 (World Bank, 2006).⁶ Perhaps more worrisome from the government's perspective is the fact that citizens voiced substantial dissatisfaction with the beneficiary lists. Protests about mis-targeting led some village leaders to resign rather than defend the beneficiary lists to their constituents: over 2,000 village officials refused to participate in the program for this reason.⁷ The experiment reported in this paper was designed and conducted in collaboration with BPS to investigate these two primary targeting issues: targeting performance and popular acceptance of the targeting results.

II.B. Sample

The sample for the experiment consists of 640 sub-villages spread across three Indonesian provinces: North Sumatra, South Sulawesi, and Central Java. The provinces were chosen to represent a broad spectrum of Indonesia's diverse geography and ethnic makeup. Within these three provinces, we randomly selected a total of 640 villages, stratifying the sample to consist of approximately 30 percent urban and 70 percent rural locations.⁸ For each village, we obtained a list of the smallest administrative unit within it (a dusun in North Sumatra and Rukun Tetangga (RT) in South Sulawesi and Central Java), and randomly selected one of these sub-villages for the experiment. These sub-village units are best thought of as neighborhoods. Each sub-village contains an average of 54 households and has an elected or appointed administrative head, whom we refer to as the sub-village head.

II.C. Experimental Design

In each sub-village, the Central Statistics Bureau (BPS) and Mitra Samya, an Indonesian NGO, implemented an unconditional cash transfer program, where beneficiary households would receive a one-time, Rp. 30,000 (about $3) cash transfer. The amount of the transfer is equal to about 10 percent of the median beneficiary's monthly per-capita consumption, or a little more than one day's wage for an average laborer.⁹

Each sub-village was randomly allocated to one of the three targeting treatments that are described in detail below.¹⁰ The number of households that would receive the transfer was set in advance through a geographical targeting approach, such that the fraction of households in a sub-village that would receive the subsidy was held constant, on average, across the treatments. We then observed how each treatment selected the set of beneficiaries.

After the beneficiaries were finalized, the funds were distributed. To publicize the lists, the program staff posted two copies of it in visible locations such as roadside food stalls, mosques, or the sub-village head's house. They also placed a suggestion box and a stack of complaint cards next to the list, along with a reminder about the program details. Depending on the sub-village head's preference, the cash distribution could occur either through door-to-door handouts or at a community meeting. After at least three days, the suggestion box was collected.

Main Treatment 1: PMT

In the PMT treatment, the government created formulas that mapped easily observable household characteristics into a single index using regression techniques. Specifically, it created a list of 49 indicators similar to those used in Indonesia's 2008 registration, encompassing the household's home attributes (wall type, roof type, etc.), assets (TV, motorbike, etc.), household composition, and household head's education and occupation. Using pre-existing survey data, the government estimated the relationship between these variables and household per-capita consumption.¹¹ While it collected the same set of indicators in all regions, the government estimated district-specific formulas due to the high variance in the best predictors of poverty across districts. On average, these regressions had an R² of 0.48 (Appendix Table 1). ¹²

Appendix Table 1.

PMT Regressions by district

District	Humbang Hasundutan	Serdang Bedagai	Pematang Siantar	Purbalingga	Wonogiri	Demak	Kendal	Semarang	Bone	Enrekang	Tana Toraja	Makasar
Indicators
Type of place (1=Urban 0=Others)		−0.086 (0.023)		−0.077 (0.025)	−0.068 (0.025)	−0.095 (0.020)	−0.230 (0.017)				0.112 (0.031)
Percapita Floor			0.004 (0.001)			0.001 (0.001)		0.001 (0.000)	0.001 (0.000)	0.001 (0.001)	0.002 (0.001)	0.003 (0.001)
Type of Floor (1=Not earth 0=Others)	−0.100 (0.024)	0.113 (0.026)	0.149 (0.031)	0.118 (0.018)	0.133 (0.017)	0.111 (0.018)	0.096 (0.015)	0.169 (0.027)		−0.108 (0.021)		0.088 (0.028)
Type of Wall (1=Brick/Cement 0=Others)	0.104 (0.027)		0.053 (0.022)		0.055 (0.018)		0.033 (0.018)		0.119 (0.026)	0.059 (0.028)	0.114 (0.038)
Toilet Facility (1=Private 0=Others)	0.056 (0.015)	0.094 (0.022)	0.184 (0.034)	0.127 (0.017)	0.066 (0.019)	0.094 (0.019)	0.123 (0.015)	0.073 (0.020)	0.103 (0.017)	0.033 (0.014)	0.087 (0.019)	0.140 (0.022)
Drinking Water source (1=Clean 0=Other)	0.035 (0.014)		0.112 (0.036)	0.044 (0.017)			0.064 (0.014)	0.100 (0.019)	−0.047 (0.018)	0.100 (0.013)	0.028 (0.016)	0.117 (0.020)
Electricity (1=PLN 0=Others)		0.113 (0.033)	0.294 (0.074)	0.112 (0.034)	0.177 (0.077)	0.125 (0.043)	0.286 (0.081)	0.286 (0.123)	0.190 (0.021)	0.093 (0.021)
Type of Roof (1=Concrete/Corrugated 0=Others)	0.078 (0.034)	0.085 (0.042)	0.108 (0.046)		−0.208 (0.021)	−0.121 (0.024)	−0.037 (0.018)	−0.075 (0.021)	0.178 (0.057)	0.093 (0.032)	0.095 (0.053)	0.122 (0.028)
Fuel for Cooking (1=Not Firewood 0=Other)	0.178 (0.033)	0.155 (0.021)	0.274 (0.019)	0.188 (0.027)	0.172 (0.038)	0.155 (0.023)	0.168 (0.018)	0.152 (0.018)	0.229 (0.030)	0.145 (0.021)	0.074 (0.036)	0.188 (0.018)
Ownership of house (1=Private 0=Others)	0.060 (0.021)			0.080 (0.042)	0.076 (0.035)		0.102 (0.022)	0.077 (0.019)				0.087 (0.021)
Having Micro Credit		0.129 (0.073)		0.098 (0.036)	0.165 (0.050)	0.209 (0.045)	0.069 (0.023)	−0.106 (0.049)		0.045 (0.022)	0.227 (0.060)	0.304 (0.111)
Household Size	−0.287 (0.018)	−0.247 (0.033)	−0.261 (0.026)	−0.314 (0.025)	−0.330 (0.021)	−0.254 (0.027)	−0.277 (0.025)	−0.378 (0.021)	−0.249 (0.017)	−0.250 (0.015)	−0.209 (0.019)	−0.293 (0.018)
Household Size Squared	0.017 (0.001)	0.012 (0.003)	0.014 (0.002)	0.019 (0.002)	0.021 (0.002)	0.014 (0.003)	0.016 (0.003)	0.026 (0.002)	0.014 (0.001)	0.015 (0.001)	0.012 (0.002)	0.016 (0.002)
Age of the head of household	0.010 (0.004)	0.013 (0.005)	0.004 (0.001)	0.012 (0.004)			0.012 (0.004)		0.009 (0.004)	0.007 (0.003)	0.004 (0.001)	0.007 (0.004)
Age of the head of household Squared	0.000 (0.000)	0.000 (0.000)		0.000 (0.000)	0.000 (0.000)	0.000 (0.000)	0.000 (0.000)	0.000 (0.000)	0.000 (0.000)	0.000 (0.000)		0.000 (0.000)
Head of household gender (1=Male 0=Female)	0.153 (0.037)	0.193 (0.073)	0.109 (0.057)	0.092 (0.026)	0.098 (0.037)	0.171 (0.049)	0.153 (0.040)		0.070 (0.024)	0.145 (0.029)	0.101 (0.038)	0.135 (0.036)
Head of household is Married	0.166 (0.083)				−0.066 (0.034)	−0.086 (0.050)						0.119 (0.068)
Head of household is Male*Married	−0.207 (0.089)	−0.141 (0.071)					−0.074 (0.038)			−0.060 (0.026)	−0.075 (0.035)	−0.230 (0.076)
Sector of HH Head is Agriculture		−0.066 (0.021)			−0.071 (0.022)				−0.080 (0.021)			−0.138 (0.058)
Sector of HH Head is Industry	0.240 (0.073)		−0.060 (0.029)	0.105 (0.020)				−0.121 (0.019)		−0.108 (0.031)	−0.101 (0.044)	−0.110 (0.036)
Sector of HH Head is Service	0.248 (0.028)			0.145 (0.021)	0.098 (0.027)	0.107 (0.020)	0.053 (0.019)			0.089 (0.026)		−0.065 (0.027)
Sector of HH Head is in Formal Sector			0.082 (0.023)			0.036 (0.019)	0.076 (0.018)	0.058 (0.018)	0.056 (0.031)	0.083 (0.031)	0.089 (0.031)	0.040 (0.020)
Sector of HH Head is in Informal Sector	0.045 (0.024)		0.045 (0.023)						0.077 (0.020)	0.051 (0.020)	0.140 (0.020)
Education Attainment of HH Head is Elementary School	0.053 (0.023)	0.152 (0.040)		0.031 (0.028)	0.061 (0.025)	0.183 (0.035)	0.041 (0.018)	0.115 (0.028)	0.053 (0.019)		0.063 (0.032)
Education Attainment of HH Head is Junior School	0.072 (0.027)	0.164 (0.040)	0.198 (0.031)	0.166 (0.032)	0.153 (0.028)	0.164 (0.036)	0.128 (0.026)	0.204 (0.054)	0.099 (0.030)	0.081 (0.016)	0.080 (0.031)	0.156 (0.035)
Education Attainment of HH Head is Senior +	0.096 (0.033)	0.194 (0.045)	0.143 (0.032)	0.140 (0.051)	0.113 (0.036)	0.088 (0.045)	0.198 (0.032)	0.317 (0.035)	0.099 (0.035)	0.129 (0.020)	0.170 (0.036)	0.196 (0.053)
Number of children 0-4	−0.043 (0.011)							−0.078 (0.017)	−0.029 (0.013)	−0.044 (0.010)		−0.028 (0.016)
Number of Children in Elementary School
Number of Children in Junior H.School		0.056 (0.022)		0.056 (0.018)	0.086 (0.020)	0.072 (0.019)	0.045 (0.017)	0.073 (0.020)
Number of Children in Senior H.School		0.076 (0.023)	0.069 (0.017)	0.165 (0.024)	0.150 (0.024)	0.135 (0.024)	0.116 (0.022)	0.167 (0.015)		0.029 (0.014)	0.049 (0.018)	0.109 (0.012)
Highest Education Attainment of HH Members is Elementary School	0.053 (0.023)	0.152 (0.040)			0.061 (0.025)	−0.045 (0.019)	0.105 (0.028)	0.133 (0.054)	0.064 (0.026)		0.063 (0.032)	0.077 (0.031)
Highest Education Attainment of HH Members is Junior School	0.077 (0.025)	0.058 (0.032)	0.113 (0.083)	0.122 (0.034)	0.153 (0.028)	0.164 (0.036)	0.070 (0.030)	0.221 (0.032)	0.120 (0.035)	0.081 (0.016)	0.050 (0.024)	0.170 (0.051)
Highest Education Attainment of HH Members is Senior +	0.110 (0.033)	0.135 (0.044)	0.211 (0.082)	0.317 (0.044)	0.267 (0.042)	0.281 (0.043)	0.133 (0.035)	0.310 (0.054)	0.170 (0.041)	0.129 (0.020)	0.109 (0.032)	0.231 (0.039)
Dependency Ratio		−0.039 (0.018)	−0.034 (0.015)	−0.027 (0.016)		−0.075 (0.017)			−0.034 (0.014)	−0.022 (0.009)	−0.040 (0.011)	−0.074 (0.018)
Distance to District		−0.004 (0.001)	−0.025 (0.010)		−0.003 (0.001)	−0.004 (0.001)	−0.007 (0.001)				−0.004 (0.000)	−0.004 (0.002)
Existence of SD		−0.224 (0.040)	0.183 (0.102)					−1.438 (0.057)			0.093 (0.041)
Existence of SLTP	−0.150 (0.019)		−0.051 (0.028)	−0.088 (0.020)	0.041 (0.017)				0.053 (0.016)
Existence of Puskesmas/Pustu	−0.047 (0.020)	−0.116 (0.024)	0.100 (0.031)	0.032 (0.017)							0.049 (0.020)	0.038 (0.019)
Existence of Polindes	−0.054 (0.017)	−0.114 (0.028)			−0.048 (0.016)					0.029 (0.015)
Existence of Posyandu	−0.062 (0.018)	−0.081 (0.040)			−0.184 (0.073)				0.174 (0.038)		−0.205 (0.038)
Availability of Doctor						−0.050 (0.021)	−0.080 (0.018)	0.092 (0.023)				0.085 (0.025)
Availability of Bidan	0.082 (0.025)	0.089 (0.035)	−0.144 (0.068)		−0.065 (0.027)	0.072 (0.036)	0.093 (0.034)				−0.068 (0.021)	0.084 (0.025)
Road type is Asphalt	0.101 (0.015)	0.132 (0.023)	−0.280 (0.057)	0.137 (0.018)	−0.042 (0.018)	0.053 (0.017)			−0.114 (0.018)	0.057 (0.015)		−0.247 (0.066)
Existence of Semi permanent market place			0.276 (0.098)	0.049 (0.021)			0.065 (0.018)		−0.090 (0.018)		−0.099 (0.034)	0.048 (0.021)
Existence of Credit Facility		0.055 (0.022)						−0.072 (0.018)	−0.040 (0.017)	−0.031 (0.014)	−0.185 (0.022)
Constant	12.839 (0.106)	12.884 (0.150)	12.131 (0.218)	12.119 (0.123)	13.287 (0.127)	12.756 (0.076)	12.344 (0.131)	14.008 (0.149)	12.577 (0.109)	12.852 (0.082)	13.082 (0.087)	13.098 (0.118)
Observations	1920	2239	1824	2112	2208	2208	2208	2496	2016	1824	1920	2208
R-squared	0.606	0.284	0.486	0.457	0.436	0.363	0.471	0.516	0.474	0.556	0.478	0.583

Notes: Each column reports the result from a separate regression for that district. The dependent variable is log per capita consumption. Following standard BPS procedure, for each district, a first regression was run with all variables listed. A second regression was then run retaining only those variables that were statistically significant at the 10% level in the first regression. The results above present the results of this second regression, which were used for the PMT calculation. All variables above are statistically significant at least at the 10% level.

Government enumerators from BPS collected these indicators from all households in the PMT sub-villages by conducting a door-to-door survey. These data were then used to calculate a computer-generated poverty score for each household using the district-specific PMT formula. A list of beneficiaries was generated by selecting the pre-determined number of households with the lowest PMT scores in each sub-village.

Main Treatment 2: Community Targeting

In the community treatment, the sub-village residents determine the list of beneficiaries through a poverty-ranking exercise. To start, a local facilitator visited each sub-village, informed the sub-village head about the program, and set a date for a community meeting. The meeting dates were set several days in advance to allow the facilitator and sub-village head sufficient time to publicize the meeting. Facilitators made door-to-door household visits in order to encourage attendance. On average, 45 percent of households attended the meeting.

At the meeting, the facilitator first explained the program. Next, he displayed a list of all households in the sub-village (from the baseline survey), and asked the attendees to correct the list if necessary. The facilitator then spent 15 minutes helping the community brainstorm a list of characteristics that differentiate the poor households from the wealthy ones in their community.

The facilitator then proceeded with the ranking exercise using a set of randomly-ordered index cards that displayed the names of each household in the sub-village. He hung a string from wall to wall, with one end labeled as “most well-off” (paling mampu) and the other side labeled as “poorest” (paling miskin). Then, he presented the first two name cards from the randomly-ordered stack to the community and asked, “Which of these two households is better off?” Based on the community's response, he attached the cards along the string, with the poorer household placed closer to the “poorest” end. Next, he displayed the third card and asked how this household ranked relative to the first two households. The activity continued with the facilitator placing each card one-by-one on the string until all the households had been ranked.¹³ By and large, the community reached a consensus on the ranks.¹⁴ Before the final ranking was recorded, the facilitator read the ranking aloud so adjustments could be made if necessary.

After all meetings were complete, the facilitators were provided with “beneficiary quotas” for each sub-village based on the geographic targeting procedure. Households ranked below the quota were deemed eligible. Note that prior to the ranking exercise, facilitators told the meeting attendees that the quotas were predetermined by the government, and that all households who were ranked below this quota would receive the transfer. The quota itself was not known by either facilitators or attendees at the time of the meeting. Facilitators also emphasized that the government would not interfere with the community's ranking.

Main Treatment 3: Hybrid

The hybrid method combines the community ranking procedure with a subsequent PMT verification. In this method, the ranking exercise, described above, was implemented first. However, there was one key difference: at the start of these meetings, the facilitator announced that the lowest-ranked households, those ranked 1.5 times below the “beneficiary quotas,” would be independently checked by government enumerators before the list was finalized.

After the community meetings were complete, the government enumerators visited the lowest-ranked households to collect the data needed to calculate their PMT score. Beneficiary lists were then determined using the PMT formulas. Thus, it was possible, for example, that some households could become beneficiaries even if they were ranked as slightly wealthier than the beneficiary quota cutoff line on the community list (and vice versa).

The hybrid treatment aims to take advantage of the relative benefits of both methods. First, as compared to the community method, the hybrid method's additional PMT verification phase may limit elite capture. Second, in the hybrid method, the community is incentivized to accurately rank the poorest households at the bottom of the list, as richer households would later be eliminated by the PMT. Third, as compared to the PMT treatment, the hybrid method's use of the community rankings to narrow the set of households that need to be surveyed may be potentially more cost-effective, in light of the fewer household visits required.

Community Sub-Treatments

We designed several sub-treatments in order to test three hypotheses about why the results from the community process might differ from those that resulted from the PMT treatment: elite capture, community effort, and within-community heterogeneity in preferences.

First, to test for elite capture, we randomly assigned the community and hybrid sub-villages to two groups: a “whole community” sub-treatment and an “elite” sub-treatment. In “whole community” villages, the facilitators actively recruited all community members to participate in the ranking. In the “elite” villages, meeting attendance was restricted to no more than seven invitees that were chosen by the sub-village head. Inviting at least one woman was mandatory and there was some pressure to invite individuals who are usually involved in village decision-making, such as religious leaders or school teachers. The elite meetings are smaller and easier to organize and run. Moreover, the elites may have the legitimacy needed (and possibly even better information) to make difficult choices. However, the danger of the elite meetings is that they will funnel aid to their friends and family (Bardhan and Mookherjee, 2005).

Second, we introduced a treatment to test whether the efficacy of the community approach is limited by a community's ability or willingness to expend effort. Specifically, we randomized the order in which households were ranked in order to compare the accuracy at the start and the end of the meeting.¹⁵ The ranking procedure is tedious: on average, it took 1.68 hours. For a sub-village with the mean number of households (54), even an optimal sorting algorithm would require making 6 pair-wise comparisons by the time the last card was placed. Thus, by the end of the meetings, the community members may be too tired to rank accurately.

The third set of hypotheses concerns the role of preferences. If the community results differ from the PMT results because of preferences, it is important to understand whether these preferences are broadly shared or are simply a function of who attends the meeting. Meeting times were therefore varied in order to attract different subsets of the community. Half of the meetings were randomly assigned to occur after 7:30 pm, when men who work during the day could easily attend. The rest were in the afternoon, when we expected higher female attendance. In addition, we also conducted meetings where we put a focus on “poverty”: in half of the meetings, the facilitator led an exercise to identify the ten poorest households in the sub-village before the ranking exercise began (“10 poorest treatment”).

Randomization Design and Timing

We randomly assigned each of the 640 sub-villages to the treatments as follows (Table 1). In order to ensure experimental balance across the geographic regions, we created 51 geographic strata, where each stratum consists of all villages from one or more sub-districts (kecamatan) and is entirely located in a single district (kabupaten).¹⁶ Then, we randomly allocated sub-villages to one of the three main treatments (PMT, community, or hybrid), stratifying such that the proportion allocated to each was identical (up to integer constraints) within each stratum. We then randomly and independently allocated each community or hybrid sub-village to the sub-treatments, stratified both by stratum and main treatment.

Table 1.

Randomization Design

Community/Hybrid Sub-Treatments			Main Treatments
			Community	Hybrid	PMT
Elite	10 Poorest First	Day	24	23
		Night	26	32
	No 10 Poorest First	Day	29	20
		Night	29	34
Whole Community	10 Poorest First	Day	29	28
		Night	29	23
	No 10 Poorest First	Day	28	33
		Night	20	24
		TOTAL	214	217	209

Notes: This table shows the results of the randomization. Each cell reports the number of sub-villages randomized to each combination of treatments. Note that the randomization of sub-villages into main treatments was stratified to be balanced in each of 51 strata. The randomization of community and hybrid subvillages into each sub-treatment (elite or full community, 10 poorest prompting or no 10 poorest prompting, and day or night) was conducted independently for each sub-treatment, and each randomization was stratified by main treatment and geographic stratum.

From November to December 2008, an independent survey company conducted a census in each sub-village and then collected the baseline data. The targeting treatments and the creation of the beneficiary lists started immediately after the baseline survey was completed (December 2008 and January 2009). Fund distribution, the collection of the complaint form boxes, and interviews with the sub-village heads occurred during February 2009. Finally, the survey company conducted the endline survey in late February and early March 2009.

III. Data

III.A. Data Collection

We collected four main sources of data: a baseline household survey, household rankings generated by the treatments, data on the community meeting process (in community/hybrid treatments only), and data on community satisfaction.

Baseline Data

We conducted a baseline survey in November and December 2008. The survey was administered by SurveyMeter, an independent survey organization. At this point, there was no mention of the experiment to households.¹⁷ We began by constructing a complete list of all households in the sub-village. From this census, we randomly sampled eight households from each sub-village plus the head of the sub-village, for a total sample size of 5,756 households. To ensure gender balance among survey respondents, in each sub-village, households were randomized as to whether the household head or spouse of the household head would be targeted as the primary respondent. The survey included questions on demographics, family networks in the sub-village, participation in community activities, relationships with local leaders, access to existing social transfer programs, and households’ per capita consumption.

The baseline survey also included a variety of measures of the household's subjective poverty assessments. In particular, we asked each household to rank the other eight households surveyed in their sub-village from poorest to richest. Finally, we asked respondents several subjective questions to determine how they assessed their own poverty levels.

Data on Treatment Results

Each of our treatments – PMT, community, and hybrid – produces a rank ordering of all the households in the sub-village (henceforth, the “targeting rank list”). For the PMT treatment, this is the rank ordering of the PMT score, i.e. predicted per capita expenditures. For the community treatment, it is the rank ordering from the community meetings. For the hybrid treatment, it is the final ranked list (where all households that were verified are ordered based on their PMT score, while those that were not are ordered based on their rank from the community meeting). For all treatments, we additionally collected data on which households actually received the transfer.

Data on Community Meetings

For the community and hybrid sub-villages, we collected data on the meetings’ functioning, as well as attendance lists. After each meeting, the facilitators filled out a questionnaire on their perceptions of the community's interest and satisfaction levels.

Data on Community Satisfaction

After the cash disbursement was complete, we collected data on the community's satisfaction level using four different tools: suggestion boxes, sub-village head interviews, facilitator feedback, and household interviews. First, facilitators placed suggestion boxes in each sub-village along with a stack of complaint cards. Each anonymous complaint card asked three yes/no questions in a simple format: (1) Are you satisfied with the beneficiary list resulting from this program? (2) Are there any poor households not included on the list? (3) Are there any non-poor households included on the list? Second, on the day when the suggestion boxes were collected, the facilitators interviewed the sub-village heads.¹⁸ Third, each facilitator filled out feedback forms on the ease of distributing the transfer payments. Finally, in Central Java province, SurveyMeter conducted an endline survey of three households that were randomly chosen from the eight baseline survey households.

III.B. Summary statistics

Table 2 provides sample statistics of the key variables. Panel A shows that average monthly per capita expenditures are approximately Rp. 558,000 (about $50). Panel B provides statistics on the errors in targeting based on consumption. By construction, about 30 percent of the households received the cash transfer. We calculated the per-capita consumption level in each province (separately by urban and rural areas) that corresponded to the percentage of households who were supposed to receive the transfer. This threshold level is approximately equal to the PPP$2 poverty line.¹⁹ We defined the “error rate based on consumption” (from now on “error rate”) to be equal to 1 if either the household's per capita consumption from the baseline survey was below the threshold line and it did not receive the transfer (exclusion errors) or if it was above the threshold line and did receive it (inclusion errors). We further disaggregate these measures by dividing those below the threshold in half into the “very poor” and the “near poor,” with approximately half of the total poor population in each of these two categories. We likewise divide the population above the threshold in half into the “middle income” and “rich.” Based on these metrics, thirty-two percent of the households were incorrectly targeted based on consumption. Twenty percent of the non-poor households received it, while 53 percent of the poor were excluded. Reassuringly, errors were less likely to happen for the rich (14 percent), and most likely to happen for the near poor (59 percent).²⁰

Table 2.

Summary Statistics

Variable	Obs	Mean	Std. Dev.
Panel A: Consumption from baseline survey
Per capita consumption (Rp. 000s)	5753	557.501	602.33
Panel B: Mis-targeting variables:
On beneficiary list	5756	0.30	0.46
Error Rate Based on Consumption	5753	0.32	0.47
Inclusion Error (nonpoor =rich + middle)	3725	0.20	0.40
Exclusion Error (poor =near + very poor)	2028	0.53	0.50
Error Rate Based on Consumption -- rich	1843	0.14	0.35
Error Rate Based on Consumption -- middle income	1882	0.27	0.44
Error Rate Based on Consumption -- near poor	1074	0.59	0.49
Error Rate Based on Consumption -- very poor	954	0.46	0.50
Panel C: Rank correlations between treatment results and...
Per capita consumption	640	0.41	0.34
Community (excluding sub-village head)	640	0.64	0.33
Sub-village Head	640	0.58	0.41
Self-Assessment	637	0.40	0.34

Panel C provides summary statistics for several alternative metrics that can be used to gauge targeting: the rank correlation for each sub-village between one of four different metrics of household well-being and results of the targeting experiment (“targeting rank list”). This allows us to flexibly examine the relationship between the treatment outcomes and various measures of well-being on a comparable scale. First, we compute the rank correlation with per capita consumption, which tells us how closely the final outcome is to the government's metric of well-being. Second, we compute the rank correlation with the ranks provided by the eight individual households during the baseline survey. This allows us to understand how close the targeting rank list is to the community member's individual beliefs about their fellow community members’ well-being. Third, we compute the rank correlation with the ranks provided by the sub-village head in the baseline survey. Finally, we compute the rank correlation with respondents’ self-assessment of poverty from the baseline survey.²¹ This allows us to understand how closely the treatment result matches individuals’ beliefs about their own well-being.

While the targeting rank lists are associated with the consumption rankings, they are more highly associated with the community's rankings of well-being. While the mean rank correlation between the targeting rank lists and the consumption rankings is 0.41, the mean correlation of the targeting rank list with the individual community members’ ranks is 0.64, and the correlation with the sub-village head's ranks is 0.58. Finally, we observe a 0.40 correlation between the ranks from the targeted lists with the individuals’ self assessments.

III.F. Randomization Balance Check

To verify that the randomization for the main treatments generates balance across the covariates, we examined the following five characteristics from the baseline survey prior to obtaining the data from the experiment: ²² per capita expenditures, years of education of the household head, calculated PMT score, the share of households that are agricultural, and the years of education of the sub-village head. We also examined five village characteristics from the 2008 PODES, a census of villages conducted by BPS: log number of households, distance to district center in kilometers, log size of the village in hectares, the number of religious buildings per household, and the number of primary schools per household. The results, presented in Appendix Table 2 and discussed in more detail in the appendix, show that the sub-villages generally appear to be well-balanced.

Appendix Table 2.

Testing Balance Between Treatment Groups

	Means			Differences, No Fixed Effects			Differences, Controlling for Stratum Fixed Effects
	PMT	Community	Hybrid	Community - PMT	Hybrid - PMT	Hybrid - Community	Community - PMT	Hybrid - PMT	Hybrid - Community
	(1)	(2)	(3)	(4)	(5)	(6)	(7)	(8)	(9)
Average per capita expenditure (Rp. 000s)	558.576 [245.845]	550.579 [220.237]	564.295 [337.172]	−7.997 (22.728)	5.719 (28.535)	13.716 (27.416)	−1.331 (20.661)	11.980 (25.973)	13.312 (24.913)
Average years of education of household head among survey respondents	7.360 [2.616]	7.566 [2.644]	7.087 [2.627]	0.206 (0.256)	−0.273 (0.254)	−0.4785* (0.254)	0.219 (0.204)	−0.255 (0.200)	−0.4739** (0.209)
PMT score (calculated from Baseline survey)	12.467 [0.436]	12.519 [0.414]	12.474 [0.423]	0.052 (0.041)	0.007 (0.042)	−0.045 (0.040)	0.053 (0.037)	0.011 (0.037)	−0.043 (0.037)
Pct. of households that are agricultural	45.827 [34.889]	42.887 [33.789]	48.438 [35.038]	−2.940 (3.343)	2.612 (3.391)	5.5515* (3.318)	−3.7806* (2.060)	1.264 (2.096)	5.0442** (2.027)
Years of education of RT head	8.856 [4.018]	8.860 [4.244]	8.604 [3.796]	0.003 (0.402)	−0.253 (0.379)	−0.256 (0.388)	0.033 (0.352)	−0.206 (0.336)	−0.238 (0.335)
Log number of households	3.832 [0.491]	3.895 [0.489]	3.810 [0.460]	0.063 (0.048)	−0.022 (0.046)	−0.0853* (0.046)	0.057 (0.044)	−0.028 (0.043)	−0.0846** (0.041)
Distance to kecamatan in km	0.444 [0.652]	0.416 [0.473]	0.482 [0.431]	−0.028 (0.056)	0.039 (0.054)	0.067 (0.044)	−0.029 (0.050)	0.038 (0.046)	0.0673* (0.037)
Log size of villages in hectares	3.105 [1.278]	3.271 [1.197]	3.282 [1.187]	0.166 (0.121)	0.177 (0.120)	0.011 (0.115)	0.1435* (0.075)	0.1376* (0.075)	−0.006 (0.076)
Religious building per household	0.0070 [0.0050]	0.0060 [0.0050]	0.0060 [0.0050]	−0.0004 (0.0005)	−0.0004 (0.0005)	−0.0001 (0.0005)	−0.0004 (0.0004)	−0.0005 (0.0004)	−0.0001 (0.0003)
Primary school per household	0.0030 [0.0030]	0.0030 [0.0030]	0.0030 [0.0020]	0.0001 (0.0003)	−0.0002 (0.0002)	−0.0003 (0.0002)	0.0000 (0.0002)	−0.0002 (0.0002)	−0.0003 (0.0002)
P-value from joint test				0.275	0.689	0.089	0.165	0.322	0.028

Notes: An observation is a sub-village, and therefore, there are 640 observations. Standard deviations are shown in brackets in columns (1) - (3); robust standard errors are shown in parentheses in columns (4) - (9).

IV. Results on Targeting Performance and Satisfaction

We begin by evaluating the treatments from the government's perspective. Specifically, we examine (1) how the treatments performed in terms of targeting the poor based on per-capita consumption, (2) how the treatments could affect the poverty rate, and (3) how the treatments performed in terms of satisfaction with and legitimacy of the targeting results.

IV.A. Targeting performance based on per-capita consumption

We begin by comparing how the different targeting methods performed based on per-capita consumption levels. Specifically, we compute location-specific poverty lines based on the PPP$2 per day consumption threshold, and then classify a household as incorrectly targeted if its per capita consumption level is below the poverty line and it was not chosen as a beneficiary, or if it was above the poverty line and it was identified as a recipient (Error_ivk). We then examine which method minimized the error rate by estimating the following equation using OLS:

$${\mathrm{ERROR}}_{ivk}=\alpha +{\beta }_1{\mathrm{COMMUNITY}}_{ivk}+{\beta }_2{\mathrm{HYBRID}}_{\mathrm{ivk}}+{\gamma }_k+{\epsilon }_{ivk}$$ (1)

where i represents a household, v represents a sub-village, k represents a stratum, and γ_k are stratum fixed effects.²³ Note that the PMT treatment is the omitted category, so β₁ and β₂ are interpretable as the impact of the community and the hybrid treatments relative to the PMT treatment. Since the targeting methods were assigned at the sub-village level, the standard errors are clustered to allow for arbitrary correlation within a sub-village.

The results, shown in Table 3, indicate that the PMT method outperforms both the community and hybrid treatment in terms of the consumption-based error rate. Under the PMT, 30 percent of the households are incorrectly targeted (Column 1).²⁴ Both the community and hybrid methods increase the error rate by about 3 percentage points —or about 10 percent— relative to the PMT method (significant at the 10 percent level).²⁵

Table 3.

Results of Different Targeting Methods on Error Rate Based on Consumption

	(1)	(2)	(3)	(4)	(5)	(6)	(7)	(8)
		By Income Status		By Detailed Income Status				Per-capita consumption of beneficiaries
Sample:	Full population	Inclusion Error	Exclusion Error	Rich	Middle income	Near Poor	Very Poor
Community treatment	0.031* (0.017)	0.046** (0.018)	0.022 (0.028)	0.028 (0.021)	0.067** (0.027)	0.49 (0.038)	−0.013 (0.039)	9.933 (18.742)
Hybrid treatment	0.029* (0.016)	0.037** (0.017)	0.009 (0.027)	0.020 (0.020)	0.052** (0.025)	0.031 (0.037)	−0.008 (0.037)	−1.155 (19.302)
Observations	5753	3725	2028	1843	1882	1074	954	1719
Mean in PMT treatment	0.30	0.18	0.52	0.13	0.23	0.55	0.48	366

Notes: All regressions include stratum fixed effects. Robust standard errors in parentheses, clustered at the village level. All coefficients are interpretable relative to the PMT treatment, which is the omitted category. The mean of the dependent variable in the PMT treatment is shown in the bottom row. All specifications include stratum fixed effects.

^***p<0.01

^**p<0.05

^*p<01.

Adding a rich household to the list may have different welfare implications than adding a household that is just above the poverty line. To examine this, Figure 1 graphs the log per capita consumption distribution of the beneficiaries (left panel) and non-beneficiaries (right panel) for each targeting treatment. The vertical lines in the graphs indicate PPP$1 and PPP$2 per day poverty lines. Overall, the graphs confirm that all methods select relatively poorer households: for all methods, the mode per-capita consumption for beneficiaries is below PPP$2 per day, whereas it is above PPP$2 per day for non-beneficiaries.

Figure 1.

PDF of log per-capita consumption of beneficiaries and non-beneficiaries, by treatment status

Examining the impact of the treatments, the left panel shows that the consumption distribution of beneficiaries derived from the PMT is centered to the left of the distribution under the community and hybrid methods. Thus, on average, the PMT identifies poorer individuals. However, the community methods select a greater percentage of beneficiaries whose log daily per-capita consumption is less than PPP$1 (the leftmost part of the distribution). Thus, the figures suggest that despite doing worse on average, the community methods may capture more of the very poor. Moreover, the figures suggest that all three methods contain similar proportions of richer individuals (with log income greater than about 6.5). The difference in the error rate across the three treatments is driven by differences in the near poor (PPP$1 to PPP$2) and the middle income group (those above the PPP$2 poverty line, but with log income less than 6.5).

We more formally examine the findings from Figure 1 in the remaining columns of Table 3. In Columns 2 and 3, we examine the error rates separately for the poor (exclusion error) and the non-poor (inclusion error). In Columns 4 and 5, we disaggregate the non-poor into rich and middle, and in Columns 6 and 7, we disaggregate the poor by splitting them into near poor and very poor. The results confirm that much of the difference in the error rate between the community methods and the PMT occurs near the cutoff for inclusion. Specifically, the community and hybrid methods are respectively 6.7 and 5.2 percentage points more likely to misclassify the middle non-poor (Column 5, both statistically significant at 5 percent). They are also more likely to misclassify the near poor by 4.9 and 3.1 percentage points, respectively, although these results are not individually statistically significant. In contrast, we observe much less difference between the methods for the rich and the very poor, and in fact the point estimate suggests that the community method may actually do better among the very poor.

In Column 8, we examine the average per capita consumption of beneficiaries across the three groups. As expected, given that the community treatment selects more of the very poor and also selects more individuals who are just above the PPP$2 poverty line, the average per capita consumption of beneficiaries is not substantially different between the various treatments. This suggests that even though the community treatments are more likely to mis-target the poor as defined by the PPP$2 cutoff, the welfare implications of the three methods appear similar based on the consumption metric.²⁶

Given that the levels of information and capture may be different across localities, we examine the heterogeneity in the relative effectiveness of the different treatments across three dimensions, all of which we specified ex-ante when designing the intervention. First, we hypothesized that the community methods may do worse in urban areas, where individuals may not know their neighbors as well. Our sample was stratified along this dimension to ensure that we had a large enough sample size to test this hypothesis. Second, the level of inequality in the villages could result in important differences between the two techniques. On one hand, community-based targeting may work better in areas with large inequality, since it implies that the rich and the poor are more sharply differentiated. On the other hand, elite capture of community based techniques may be more severe in areas with high inequality if rich elites are powerful enough to exclude the poor from the community decision-making process. Third, we hypothesized that in the areas where many people are related to one other by blood or marriage, they have more information about their neighbors, so the community method should work better.

We present the results of the analysis where we interact the various treatment variables with these three dimensions of heterogeneity in Table 4.²⁷ We find that, in general, the error rate was lower in the community treatment (relative to the PMT) in urban areas, in areas with high inequality, and in areas where many households are related. However, these effects are not significant at conventional levels. In addition, we also test whether the treatments differed in Java and the other provinces, as previous studies (e.g., Dearden and Ravallion,1988) have shown that Java tends to be more egalitarian (note that our sample was also stratified along this dimension). The results of this analysis, presented in Appendix Table 4, show no substantive differences between Java and the other provinces.

Table 4.

Results of Different Targeting Methods on Error - Heterogeneity

	(1)	(2)	(3)	(4)	(5)	(6)	(7)	(8)
		By Income Status		By Detailed Income Status				Per-capita consumption of beneficiaries
Sample:	Full population	Exclusion Error	Inclusion Error	Rich	Middle income	Near Poor	Very Poor
Community treatment	0.069** (0.035)	0.005 (0.039)	0.145** (0.058)	−0.052 (0.050)	0.068 (0.053)	0.218*** (0.079)	0.042 (0.083)	−44.804 (36.192)
Hybrid treatment	0.087** (0.037)	0.017 (0.042)	0.130** (0.060)	0.041 (0.054)	−0.009 (0.054)	0.200** (0.078)	0.092 (0.087)	−22.408 (40.155)
Urban village	−0.010 (0.030)	−0.098*** (0.035)	0.128*** (0.046)	−0.088** (0.043)	−0.113** (0.047)	0.231*** (0.063)	0.035 (0.062)	−20.668 (36.623)
Inequality	−0.004 (0.026)	−0.026 (0.027)	0.057 (0.040)	−0.029 (0.033)	−0.015 (0.039)	0.043 (0.055)	0.091* (0.053)	−3.963 (33.960)
General Connectedness	0.043* (0.026)	−0.010 (0.031)	0.000 (0.039)	0.009 (0.039)	−0.046 (0.039)	0.036 (0.053)	−0.043 (0.056)	−20.957 (29.451)
Urban village x Community treatment	−0.049 (0.036)	0.026 (0.039)	−0.110* (0.060)	0.056 (0.045)	−0.029 (0.058)	−0.184** (0.078)	−0.041 (0.083)	25.062 (41.936)
Urban village x Hybrid treatment	−0.051 (0.036)	0.014 (0.039)	−0.069 (0.059)	−0.001 (0.045)	0.020 (0.055)	−0.149* (0.079)	−0.015 (0.082)	11.794 (47.509)
Inequality x Community treatment	−0.016 (0.035)	0.022 (0.039)	−0.134** (0.055)	0.027 (0.047)	0.052 (0.057)	−0.128* (0.075)	−0.117 (0.076)	38.764 (39.728)
Inequality x Hybrid treatment	−0.021 (0.034)	−0.002 (0.037)	−0.069 (0.054)	−0.036 (0.045)	0.061 (0.054)	0.006 (0.073)	−0.157** (0.077)	13.022 (41.603)
General Connectedness x Community treatment	−0.018 (0.035)	0.022 (0.041)	−0.007 (0.058)	0.069 (0.050)	−0.025 (0.056)	−0.038 (0.078)	0.040 (0.082)	44.665 (39.438)
General Connectedness x Hybrid treatment	−0.050 (0.035)	0.018 (0.040)	−0.081 (0.058)	−0.007 (0.050)	0.034 (0.052)	−0.166** (0.080)	−0.010 (0.078)	17.931 (40.290)
Observations	5753	3725	2028	1843	1882	1074	954	1719
Mean in PMT treatment	0.30	0.18	0.52	0.13	0.23	0.55	0.48	366

Notes: See Notes to Table 3.

^***p<0.01

^**p<0.05

^*p<0.1

Appendix Table 4.

Results of Different Targeting Methods on Error Rate - Heterogeneity for Java/Non-Java

	(1)	(2)	(3)	(4)	(5)	(6)	(7)	(8)
		By Income Status		By Detailed Income Status				Per-capita consumption of beneficiaries
Sample:	Full population	Exclusion Error	Inclusion Error	Rich	Middle income	Near Poor	Very Poor
COMMUNITY	0.038* (0.022)	0.055** (0.023)	0.017 (0.040)	0.048** (0.024)	0.066* (0.035)	0.058 (0.051)	−0.030 (0.062)	26.028 (23.419)
HYBRID	0.021 (0.021)	0.016 (0.020)	−0.001 (0.038)	0.034 (0.023)	−0.001 (0.032)	0.025 (0.051)	−0.024 (0.057)	−3.088 (22.716)
COMMUNITY × Java	−0.016 (0.033)	−0.026 (0.037)	0.012 (0.056)	−0.047 (0.043)	−0.012 (0.053)	−0.016 (0.075)	0.035 (0.080)	−26.834 (36.993)
HYBRID × Java	0.019 (0.032)	0.053 (0.036)	0.016 (0.054)	−0.032 (0.043)	0.127** (0.051)	0.010 (0.075)	0.027 (0.075)	2.673 (37.739)

Notes: All regressions include stratum fixed effects. Robust standard errors are shown in parentheses, adjusted for clustering at the village level. All coefficients are interpretable relative to the PMT treatment, which is the omitted category. The mean of the dependent variable in the PMT treatment is shown in the bottom row. All specifications include stratum fixed effects.

*** p<0.01

^**p<0.05

^*p<0.1

In sum, we do not observe significant differences between the methods based on the four levels of heterogeneity that we considered. This speaks to the external validity of the study, suggesting that the findings may be easier to generalize to other settings.

IV.B. Effects of Targeting Policy on Poverty Rate and Gap

We observe that the community treatment has a 3-percentage point larger error rate based on consumption than the PMT. Given that the differences are largely driven by those near the thresholds, an important question is whether this is large enough to affect real outcomes, such as the headcount poverty rate (the percentage of people who fall below the poverty line) and the poverty gap (the mean distance below the poverty line as a proportion of the line, counting the non-poor as having zero gap). Moreover, given that the community method better targets the very poor, it is possible that the community methods may perform better at reducing the squared poverty gap (which places greater weight on reducing the poverty of the very poor), even if it performs worse in reducing the poverty headcount ratio.

We follow the methods used in Ravallion (2009) and simulate the effects of the different targeting methods on the headcount poverty rate, the poverty gap and squared poverty gap. We provide the results of the simulation for selected transfer sizes (no transfer, Rp 50,000, Rp 100,000, Rp 200,000, and Rp 500,000 per month) in Table 5, and graph out the full results by transfer size in Figure 2. In both Table 5 and Figure 2, we focus on the poor and very poor poverty lines. Note that despite the randomization, there are statistically insignificant differences between the poverty rates in the different treatments as a result of sampling; for the simulations, we assume for all treatments the distribution of consumption from the PMT villages, so that we have exactly the same income distribution across treatments.²⁸

Table 5.

Simulated poverty impacts for different transfer sizes and poverty lines

Transfer Size (Rp. 000s)		Poverty line = Poor			Poverty line = Very poor
		PMT	Community	Hybrid	PMT	Community	Hybrid
No transfer	headcount	33.86	33.86	33.86	15.64	15.64	15.64
	pov. gap	9.45	9.45	9.45	3.55	3.55	3.55
	sq. pov gap	3.73	3.73	3.73	1.21	1.21	1.21
50	headcount	32.81	32.81	33.30	15.04	14.52	14.55
	pov. gap	8.88	8.89	8.87	3.17	3.15	3.16
	sq. pov gap	3.42	3.41	3.39	1.04	1.02	1.02
100	headcount	31.45	32.03	32.01	14.02	13.68	13.77
	pov. gap	8.37	8.40	8.35	2.84	2.82	2.83
	sq. pov gap	3.14	3.13	3.11	0.91	0.88	0.88
200	headcount	29.60	30.03	30.10	12.33	12.13	12.28
	pov. gap	7.47	7.54	7.48	2.37	2.33	2.32
	sq. pov gap	2.71	2.69	2.67	0.74	0.68	0.68
500	headcount	24.22	25.15	25.38	8.85	9.17	9.13
	pov. gap	5.76	5.87	5.84	1.69	1.59	1.61
	sq. pov gap	2.02	1.98	1.97	0.53	0.45	0.46

Notes: Transfers are in thousands of Rupiah (Rp. 1,000 = US $0.10) and assume a uniform transfer for all recipient households. Note that the transfer is per household, whereas poverty is defined as per-capita, so a given transfer has less of an impact on larger households. Headcount and poverty gap are in percent; squared poverty gap is multiplied by 100.

Figure 2.

Simulated poverty impacts for different transfer sizes and poverty lines

The differences in targeting accuracy across the three methods do not result in large differences in the measures of poverty under consideration. For example, Indonesia's BLT program provided transfers of Rp 100,000 per month for a year. If a similar transfer was provided to those below poor poverty line, the PMT would reduce the headcount ratio from 33.86 to 31.45 (Column 1), while the community and hybrid methods would result in ratios of 32.03 and 32.01, respectively (Column 2 and 3).²⁹ The poverty gap for the three methods is also not significantly different (7.47 for PMT, 7.54 for the community method, and 7.48 for the hybrid method). However, given that the community method works better at identifying the very poor, the community treatments actually do better at reducing the poverty headcount (from 15.64 to 13.68 for the community and to 13.77 for the hybrid) than the PMT (14.02) at the very poor poverty line, but this difference is not statistically significant. The squared poverty gap, with its emphasis on the very poorest, gives the best chance for the community method to dominate the PMT. With the very poor poverty line, for a transfer of Rp 100,000 per month, the squared poverty gaps are still quite similar for the PMT (0.91) and the community method (0.88). Doubling the transfer results in the community method doing substantially better-- 0.68 for the community versus 0.74 for the PMT, though the difference is still not statistically significant.

Note that these baseline simulations do not include differences in targeting costs. To account for the costs of targeting, we assume that the targeting is done annually. We therefore divide the targeting costs by the number of households per village who are beneficiaries to provide a per-beneficiary cost, and divide by 12 to obtain a monthly cost. We then reduce the transfer by this amount. Since the targeting costs are small when expressed monthly in this way (the costs are Rp. 7,000 per beneficiary per month for PMT, compared with Rp. 3,100 for community, and Rp. 8,000 for hybrid), they do not qualitatively affect the results above. Appendix Table 5 reproduces Table 5 after subtracting out these targeting costs.

Appendix Table 5.

Simulated poverty impacts after subtracting targeting costs .

Transfer Size (Rp. 000s)		Poverty line = Poor			Poverty line = Very poor
		PMT	Community	Hybrid	PMT	Community	Hybrid
No transfer	headcount	33.86	33.86	33.86	15.64	15.64	15.64
	pov. gap	9.45	9.45	9.45	3.55	3.55	3.55
	sq. pov gap	3.73	3.73	3.73	1.21	1.21	1.21
50	headcount	33.07	32.91	33.41	15.11	14.69	14.74
	pov. gap	8.96	8.92	8.96	3.22	3.17	3.22
	sq. pov gap	3.46	3.42	3.44	1.06	1.03	1.05
100	headcount	31.73	32.11	32.19	14.17	13.78	13.89
	pov. gap	8.43	8.43	8.42	2.88	2.84	2.88
	sq. pov gap	3.18	3.14	3.15	0.93	0.89	0.90
200	headcount	29.66	30.09	30.32	12.46	12.13	12.44
	pov. gap	7.53	7.56	7.54	2.40	2.34	2.35
	sq. pov gap	2.74	2.70	2.70	0.75	0.68	0.69
500	headcount	24.24	25.24	25.53	8.86	9.19	9.25
	pov. gap	5.78	5.89	5.87	1.70	1.59	1.62
	sq. pov gap	2.03	1.99	1.98	0.54	0.45	0.46

Note that these results are very similar to the main targeting results because the differences in targeting costs are small relative to the transfers considered here. Note that the transfer sizes above are monthly transfers in Rp. 000s. Assuming that targeting is done once per year, so the one-time cost of targeting is amortized over twelve monthly transfers, the costs of targeting per beneficiary are Rp. 7,000 for PMT, Rp. 3,100 for community, and Rp. 8,000 for hybrid.

IV.C. Satisfaction

In Table 6, we study the impacts of the treatments on the communities’ satisfaction levels and the legitimacy of the targeting. Panel A presents data from the endline household survey. Panel B presents data from the follow-up survey of sub-village heads. Panel C presents the results from the anonymous comment box, the community's complaints to the village head, and the facilitator comments on the ease of distributing the transfer payments.³⁰

Table 6.

Satisfaction

	(1)	(2)	(3)	(4)	(5)	(7)
Panel A: Household Endline Survey
	Is the method applied to determine the targeted households appropriate? (1=worst,4=best)	Are you satisfied with P2K08 activities in this sub-village in general? (1=worst,4=best)	Are there any poor HH which should be added to the list? (0=no, 1 = yes)	Number of HH that should be added from list	Number of HH that should be subtracted from list	P-value from joint test
Community treatment	0.161*** (0.056)	0.245*** (0.049)	−0.189*** (0.040)	−0.578*** (0.158)	−0.554*** (0.112)	<0.001
Hybrid treatment	0.018 (0.055)	0.063 (0.049)	0.020 (0.042)	0.078 (0.188)	−0.171 (0.129)	0.762
Observations	1089	1214	1435	1435	1435
Mean in PMT treatment	3.243	3.042	0.568	1.458	0.968
Panel B: Sub-village Head Endline Survey
	Is the method applied to determine the targeted households appropriate? (0=no, 1=yes)	In your opinion, are villagers satisfied with P2K08 activities in this sub-village in general? (1=worst,4=best)	Are there any poor HH which should be added to the list? (0=no, 1=yes)	Are there any poor HH which should be subtracted from the list? (0=no, 1=yes)
Community treatment	0.378*** (0.038)	0.943*** (0.072)	−0.169*** (0.045)	−0.010 (0.020)		<0.001
Hybrid treatment	0.190*** (0.038)	0.528*** (0.071)	−0.065 (0.043)	−0.019 (0.019)		<0.001
Observations	636	629	640	640
Mean in PMT treatment	0.565	2.456	0.732	0.057
Panel C: Comment forms and fund disbursement results
	Number of comments in the comment box	Number of complaints in the comment box	Number of complaints received by sub-village head	Did facilitator encounter any difficulty in distributing the funds? (0=no, 1=yes)	Fund distributed in a meeting (0=no, 1=yes)
Community treatment	−0.944 (0.822)	−1.085*** (0.286)	−2.684*** (0.530)	−0.062*** (0.023)	0.082** (0.038)	0.0014 (0.177)
Hybrid treatment	−0.364 (0.821)	−0.554** (0.285)	−2.010*** (0.529)	−0.045* (0.026)	0.051 (0.038)
Observations	640	640	640	621	614
Mean in PMT treatment	11.392	1.694	4.34	0.135	0.579

Notes: All estimation is by OLS with stratum fixed effects. Using ordered probit for multiple response and probit models for binary dependent variables produces the same signs and statistical significance as the results shown. These results are available from the authors upon request.

Individuals are much more satisfied with the community treatment than with the PMT or hybrid treatments (Panel A). For example, in the community treatment, respondents wish to make fewer changes to the beneficiary list; they would prefer to add about 1/3 fewer households to the list of beneficiaries (Column 4) and subtract about 1/2 as many households (Column 5) than in the PMT or the hybrid treatments. Individuals in the community treatment are more likely to report that the method was appropriate (Column 1) and are also more likely to state that they are satisfied with the program (Column 2). A joint test of these dependent variables indicates that the community treatment differences are jointly statistically significant (p-value < 0.001).

Sub-village heads are also much more satisfied (Panel B). The sub-village head was 38 percentage points more likely to say that the targeting method was appropriate when community-based targeting was used and 17 percentage points less likely to name any households that should be added to the list. The higher levels of satisfaction were also manifested in fewer complaints (Panel C). There were on average 1.09 fewer complaints in the comment box in the community sub-villages relative to the PMT sub-villages, and 0.55 fewer complaints in the hybrid sub-villages relative to the PMT (Column 2). The sub-village head also reported receiving 2.68 and 2.01 fewer complaints in the community and hybrid treatment, respectively (Column 3).

The higher satisfaction levels in the community treatment led to a smoother disbursal process. First, the facilitators who distributed the cash payment were 4-6 percentage points less likely to experience difficulties while doing so in sub-villages assigned to the community or hybrid method (Panel C, Column 4). Second, the sub-village heads could choose for the cash disbursals to happen in an open community meeting or, if the head felt that they would encounter problems in the village, the facilitator could distribute the transfer door-to-door. Facilitators were 8 percentage points more likely to distribute the cash in an open meeting in the sub-villages assigned to the community treatment (Panel D, Column 5).³¹

IV.C Understanding the differences between PMT and community targeting

The findings present an interesting puzzle. The results on the error rates suggest that the community-based methods actually do somewhat worse at identifying the poor, although this does not significantly impact the poverty rate. However, the community method results in greater satisfaction levels. The following sections explore alternative explanations of why the PMT and the community methods differ: elite capture, community effort problems, heterogeneity in preferences within the villages, and differences in information.

V. Elite Capture

In Table 7, we test for elite capture by examining whether elite connected households are more likely to be beneficiaries when the elites potentially have more control of the process (i.e. in the elite-only meetings). We first verify that the elite-only meetings had an impact on attendance, and then test for whether the elite-only treatment affected the error rate. We re-estimate equation 1 (with both attendance measures and the error rate as outcome variables), but now include a dummy for the ELITE sub-treatment. As expected, elite meetings have lower participation: we find that 48 percent of households attended the meetings in the whole community treatment, compared to 18 percent in the elite sub-treatment (Column 1).³² However, the error rate was not significantly different across the two treatments (Column 2).

Table 7.

Elite Treatments

	(1)	(2)	(3)	(4)	(5)	(6)
	Attendance (Survey Data)	Full Sample Error Rate		Full Sample Error Rate		On Beneficiary List
Community treatment	0.367*** (0.038)	0.029 (0.018)	0.033 (0.023)	0.048* (0.025)	0.042* (0.025)	0.054* 0.028)
Hybrid treatment	0.370*** (0.037)	0.027 (0.018)	0.024 (0.022)	0.008 (0.024)	0.025 (0.022)	0.012 (0.023)
Elite sub-treatment	−0.301*** (0.034)	0.004 (0.016)	0.016 (0.020)	−0.013 (0.029)	−0.015 (0.021)	−0.039 (0.032)
Elite × hybrid				0.062 (0.041)		0.051 (0.043)
Elite connectedness			−0.025 (0.021)	−0.025 (0.021)	−0.063*** (0.021)	−0.063*** (0.021)
Elite connectedness × community treatment			−0.015 (0.035)	−0.013 (0.038)	−0.067** (0.033)	−0.078** (0.036)
Elite connectedness × hybrid treatment			0.010 (0.033)	0.010 (0.035)	−0.013 (0.033)	−0.001 (0.035)
Elite connectedness × elite treatment			−0.029 (0.031)	−0.034 (0.047)	0.041 (0.030)	0.064 (0.042)
Elite connectedness × elite treatment x hybrid				0.003 (0.063)		−0.047 (0.060)
Observations	287	5753	5753	5753	5756	5756
Mean in PMT treatment	0.11	0.30	0.30	0.30	0.28	0.28

Notes: In columns (1), an observation is a village and the dependent variable is the share of households surveyed in the endline survey where at least 1 household member attended a targeting meeting. The PMT mean in column (3) is not zero, because the question was worded generically to be about any targeting meeting, not just meetings associated with our project. The dependent variable in column (2) - (4) is the dummy for error in targeting based on consumption, as in column (1) of Table 3. Dependent variable in columns (5) and (6) is a dummy for being a beneficiary of the program. All specifications in Columns (3) - (6) include dummies for the community, hybrid, and elite treatment main effects, as well as stratum fixed effects; columns (4) and (6) also include a dummy for elite × hybrid. Robust standard errors in parentheses, and standard errors are adjusted for clustering at the village level in columns (2) - (6). All specifications include stratum fixed effects.

^***p<0.01

^**p<0.05

^*p<0.1

While the evidence presented in Column 2 is consistent with no elite capture, it is also consistent with the elite dominating the whole community meeting, leading to the result that both types of meetings reflect their preferences.³³ To test this, we examine whether the elites and their relatives were more likely to be selected in both the whole community and elite meetings relative to the PMT. Specifically, we estimate the following equation:

$${\mathrm{ERROR}}_{ivk}=\alpha +{\beta }_1{\mathrm{COMMUNITY}}_{ivk}+{\beta }_2{\mathrm{HYBRID}}_{ivk}+{\beta }_3{\mathrm{ELITE}}_{ivk}+{\beta }_4{\mathrm{CONN}}_{\mathrm{ivk}}+{\beta }_5({\mathrm{COMMUNITY}}_{\mathrm{ivk}}\times {\mathrm{CONN}}_{ivk})+{\beta }_6({\mathrm{HYBRID}}_{ivk}\times {\mathrm{CONN}}_{ivk})+{\beta }_7({\mathrm{ELITE}}_{ivk}\times {\mathrm{CONN}}_{ivk})+{\gamma }_k+{\epsilon }_{ivk}$$ (2)

where CONN_ivk is an indicator that equals one if the household is related to any of the sub-village leaders/elites, or is one of the leaders themselves.³⁴ Columns 3 and 4 examine the error rate as the dependent variable, and columns 5 and 6 examine whether a household received the transfer as the dependent variable. We find little evidence of elite capture. In fact, the point estimates suggest the opposite: the elite connected households are less likely to be mis-targeted in the community and elite treatments, although the effect is not significant at conventional levels. In fact, we find that the elites are actually penalized in the community meetings: elites and their relatives are about 6.7 to 7.8 percent less likely to be on the beneficiary list in the community meetings relative to PMT meetings.³⁵ The point estimates suggest that the elite treatment undoes this penalty to some degree, but on net in elite versions of the community meetings, elites are still 2.6 percentage points less likely to receive transfers than in the PMT treatment (though the combined effect is not statistically significant from 0). These findings suggest that elite capture is not the reason that the mis-targeting is worse under the community method.

VI. Problems with Community Effort

The community-based ranking process requires human effort: ranking 75 households would require making at least 363 pair-wise comparisons. Thus, the worse targeting in the community methods could result simply from fatigue as the ranking progresses. To investigate this, we randomized the order in which households were ranked.

Figure 3 graphs the relationship between the error rate and the randomized rank order from a non-parametric Fan regression, with cluster-bootstrapped 95 percent confidence intervals shown as dashed lines. The error rate is lowest for the first few households ranked, but then rises sharply by the 20^th percentile of households. The magnitude is substantial – the point estimates imply that error rates are between 5-10 percentage points lower for the first household than for households ranked in the latter half of the meeting.

Figure 3.

Effect of order in ranking meeting on mis-target rate

Table 8 reports results from investigating these issues in a regression framework. Column 1 reports the results from estimating the relationship between the error rate and the randomized rank order, which varies from 0 (household was ranked first) to 1 (the household was ranked last). The point estimate is positive, indicating a higher error rate for households ranked later, but it is not statistically significant. In Column 2, we interact the order with the hybrid treatment. The results show that in the community treatment, there is substantially more error at the end of the process: the first household ranked is 5.9 percentage points less likely to be incorrectly targeted than the last household ranked; these results are marginally significant with a p-value of 0.11. On net, the community treatment actually does slightly better than the PMT in the beginning, but substantially worse towards the end. This effect is completely undone in the hybrid, where the random rank order and the error rate appear unrelated. Columns 3 and 4 examine how the rank order affects whether a household receives the transfer. On average, households ranked at the end of the meeting are 4.9 percentage points more likely to be on the beneficiary list than those ranked at the start (significant at the 10 percent level). The additional error from being late in the list thus comes largely from richer households ranked toward the end of the process being more likely to be on the list.

Table 8.

Effort

	(1)	(2)	(3)	(4)
	Mis-target dummy		On beneficiary list dummy
Household order in ranking (percentile)	0.029 (0.026)	0.056 (0.037)	0.049* (0.026)	0.048* (0.029)
Household order in ranking × hybrid		−0.053 (0.052)		0.001 (0.028)
Observations	3784	3784	3785	3785

Notes: All specifications are limited to community and hybrid villages. Columns (1) and (2) include a hybrid dummy and stratum fixed effects; columns (3) and (4) include stratum fixed effects since the total number of beneficiaries is constant in all treatments. The dependent variable in columns (1) and (2) is the mis-target dummy for the full sample, as in column (1) of Table 4. The dependent variable in columns (3) and (4) is a dummy for being chosen as a recipient.

*** p<0.01

** p<0.05

^*p<0.1

VII. Does the Community Have a Different Maximand?

A third reason why the community produced a different outcome than the PMT is that the community is doing its best to identify the poor, but has a different concept of poverty. We thus explore whether the community's views on poverty differ from that of per-capita consumption.

VII.A. Alternative welfare metrics

We begin by examining how the targeting outcomes compare not just against the government's metric of welfare (captured by r_g,, the ranking based on per-capita consumption), but also against alternative welfare metrics. In our baseline survey, we asked eight randomly chosen members of the community to confidentially rank each other from poorest to richest. We average the ranks to construct each household's wealth rank according to the other community members, denoted r_c. To capture welfare as measured from an elite perspective, denoted r_e, we examine how the sub-village head ranked these eight other households. To measure how people assess their own poverty, denoted r_i, we asked all respondents to rate their own poverty level on a scale of 1 to 6. We computed the percentile rank of each measure to put them on the same scale.³⁶

To assess the poverty targeting results against these alternative welfare metrics, we compute the rank correlation between the targeting rank list derived from the experiment and each of four welfare metrics. We then examine the effectiveness of the various targeting treatments against these different measures of well-being by estimating:

$${\mathrm{RANKCORR}}_{vkR}=\alpha +{\beta }_1{\mathrm{COMMUNITY}}_{vk}+{\beta }_2{\mathrm{HYBRID}}_{vk}+{\gamma }_k+{\epsilon }_{vkR}$$ (3)

where RANKCORR_vkR is the rank correlation between the targeting rank list and the well-being measure R in sub-village v. Stratum fixed effects (γ_k) are included. The results are reported in Table 9. As the data is aggregated to the village level, each regression has 640 observations.

Table 9.

Assessing targeting treatments using alternative welfare metrics

	(1)	(2)	(3)	(4)
	Consumption (rg)	Community survey ranks (rc)	Sub-village head survey ranks (re)	Self-Assessment (rs)
Community treatment	−0.065** (0.033)	0.246*** (0.029)	0.248*** (0.038)	0.102*** (0.033)
Hybrid treatment	−0.067** (0.033)	0.143*** (0.029)	0.128*** (0.038)	0.075** (0.033)
Observations	640	640	640	637
Mean in PMT treatment	0.451	0.506	0.456	0.343

Notes: The dependent variable is the rank correlation between the treatment outcome (i.e., the rank ordering of households generated by the PMT, community, or hybrid treatment) and the welfare metric shown in the column, where each observation is a village. Robust standard errors are shown in parentheses.

^***p<0.01

^**p<0.05

* p<0.1

The results provide striking evidence that per capita consumption as we measure it does not fully capture what the community calls welfare. Column 1 confirms the results that are shown in Table 3: both the community and hybrid treatment result in lower rank correlations with per-capita consumption than the PMT. Specifically, they are 6.5-6.7 percentage points, or about 14 percent, lower than the rank correlations obtained with PMT. However, they move away from consumption in a very clear direction – the community treatment increases the rank correlation with r_c by 24.6 percentage points, or 49 percent above the PMT level. The hybrid also increases the correlation with r_c but the magnitude is about half that of the community treatment. Thus, the verification in the hybrid appears to move the final outcome away from the community's perception of well-being. These differences are statistically significant at the 1 percent level. Results using the rank list obtained in the survey from the sub-village head (r_e) are virtually identical to those from the community (r_c) (significant at the 1 percent level). This provides further evidence that the community and the elite broadly share the same assessments of welfare.

Perhaps most importantly, we find that the community treatment increases the rank correlation between the targeting outcomes and the self-assessments of own poverty (r_s) by 10.2 percentage points, or about 30 percent of the level in the PMT (significant at 1 percent). The hybrid treatment also does so by 7.5 percentage points. The community targeting methods are thus more likely to conform with individual's self-identified welfare status.

VII.B. Are these preferences broadly shared?

The results above suggest that the ranking exercise moves the targeting process towards a welfare metric identified by community members. An important question is the degree to which this reflects the view of one group within the community. One experimental sub-treatment was designed precisely to get at this question. In Table 10, we report the effect of changing the composition of the meeting by holding the meeting during the day, when women are more likely to be able to attend. We also consider the other sub-treatments (elite and 10 poorest) in this analysis, as they could also plausibly have affected the welfare weights of those at the meeting.

Table 10.

Do community meetings reflect broadly shared preferences?

	(1)	(2)	(3)	(4)	(5)	(6)	(7)	(8)
					Rank Correlations with:
	Attend Meeting (Meeting Data)	Attend Meeting (HH Data)	Female Attends (Meeting Data)	Mis-target	Consumption	Community (excl sub-village head)	Sub-village Head	Self-Assessment
Community treatment		0.349*** (0.042)		0.028 (0.021)	−0.089** (0.045)	0.232*** (0.040)	0.180*** (0.052)	0.072 (0.044)
Hybrid treatment	0.020 (0.029)	0.353*** (0.041)	0.008 (0.017)	0.026 (0.021)	−0.089*** (0.044)	0.130*** (0.039)	0.064 (0.051)	0.046 (0.044)
Day meeting treatment	−0.021 (0.029)	0.013 (0.033)	0.104*** (0.017)	0.008 (0.016)	0.019 (0.033)	0.004 (0.029)	0.055 (0.038)	0.014 (0.033)
Elite treatment	−0.064** (0.029)	−0.300*** (0.033)	−0.085*** (0.017)	0.005 (0.016)	−0.004 (0.033)	−0.023 (0.029)	0.034 (0.038)	−0.017 (0.033)
10 Poorest treatment	0.022 (0.029)	0.023 (0.034)	−0.010 (0.018)	−0.006 (0.016)	0.031 (0.033)	0.047 (0.029)	0.044 (0.038)	0.062* (0.032)
Observations	431	287	428	5753	640	640	640	637
Mean in PMT treatment		0.110		0.300	0.451	0.506	0.456	0.343

Notes: For column (3), the dependent variable is the percentage of households in the village in which a female attends the meeting, using data collected from the meeting attendance lists.

^***p<0.01

^**p<0.05

^*p<0.1

We begin by investigating the impact of having a daytime meeting on attendance. This treatment does not change the share of households in the village that attend (Columns 1 and 2). However, the percentage of households that are represented by women is about 10 percentage points (for a total of 49 percent) higher in the day than during the evening meetings (Column 3).

Although the day meeting treatment affected the gender composition of the meetings, Columns 4 - 8 show that it did not affect the targeting outcomes. The elite treatment also did not affect the rank correlations with any of the various welfare metrics. Interestingly, the only sub-treatment that affected the rank correlations was the 10 poorest treatment, which increased the correlation of the treatments with ranks from self assessments. Overall, there seems to be no evidence that the identity of the subgroup doing the ranking mattered.

VIII Understanding the Community's Maximand

The evidence so far suggests that the community has a systematic, broadly shared notion of welfare that is not based on per-capita consumption, and that the community targeting methods reflect this different concept of welfare. This raises several key questions: Is the community simply mis-measuring consumption? Or does it value something other than consumption?

VIII.A. Does the Community Lack Information to Evaluate Consumption?

There is no definitive way to prove that the community has all the information that is available in the PMT.³⁷ However, the fact that those ranked early in the process were ranked at least as well as in the PMT suggests that information is not the main constraint. We can also test whether the community has information about consumption beyond that in the PMT. To do so, we estimate:

$${\mathrm{RANKIND}}_{ijvk}=\alpha +{\beta }_1{\mathrm{RANKCONSUMPTION}}_{ivk}+{\beta }_2{\mathrm{RANKPMTSCORE}}_{jvk}+v_j+{\epsilon }_{ijvk}$$ (4)

where RANKIND_ijvk is household j's rank of household i (all ranks are in percentiles), RANKCONSUMPTION_ivk is the rank of household i's per capita consumption in village v, and RANKPMTSCORE_ivk is the rank of household i's PMT score that is computed using the baseline data. Fixed effects for the individual providing the ranking are included (ν_j), and standard errors are clustered at the village level. The results of this analysis are presented in Column 1 of Table 11. In Column 2, we instead include all of the variables that enter the PMT score separately rather than including the rank of the PMT score.

Table 11.

Information

	Survey rank (1)	Survey rank (2)		Survey rank (2 continued)
Rank per capita consumption within village in percentiles	0.132*** (0.014)	0.088*** (0.012)
Rank per capita consumption from PMT within village in percentiles	0.368*** (0.014)
Household floor area per capita		0.001*** 0.000	Has this Household ever got credit?	0.027** (0.011)
Not earth floor		0.060*** (0.010)	Number of children 0-4	0.000 (0.006)
Brick or cement wall		0.065*** (0.007)	Number of Children in Elementary School	0.003 (0.005)
Private toilet		0.047*** (0.008)	Number of Children in Junior High School	0.007 (0.007)
Clean drinking water		0.008 (0.009)	Number of Children in Senior High School	0.022*** (0.008)
PLN electricity		0.064*** (0.008)	Highest Education Attainment within HH is Elem. School	0.007 (0.016)
Concrete or corrugated roof		0.027* (0.014)	Highest Education Attainment within HH is Junior School	0.01 (0.016)
Cooks with firewood		0.031*** (0.008)	Highest Education Attainment within HH is Senior High or higher	0.051*** (0.017)
Own house privately		0.034*** (0.008)	Total Dependency Ratio	0.004 (0.006)
Household size		0.004 (0.006)	AC	0.049** (0.023)
Household Size Squared		−0.001 (0.001)	Computer	0.045*** (0.011)
Age of head of household		0.011*** (0.002)	Radio / Cassette Player	0.001 (0.006)
Age of head of household squared		−0.000*** 0.000	TV	0.043*** (0.010)
Head of household is Male		0.047** (0.019)	DVD/VCD player	0.017** (0.007)
Head of household is married		0.119*** (0.022)	Satellite dish	0.021* (0.011)
Head of household is male and Married		−0.043* (0.026)	Gas burner	0.030*** (0.008)
Head of household works in agriculture sector		−0.006 (0.041)	Refrigerator	0.069*** (0.008)
Head of household works in industry Sector		−0.043 (0.042)	Bicycle	−0.004 (0.007)
Head of household works in service Sector		−0.018 (0.042)	Motorcycle	0.078*** (0.007)
Head of household works in formal sector		0.071 (0.045)	Car / Mini−bus / Truck	0.116*** (0.012)
Head of household works in informal sector		0.048 (0.045)	HP	0.014* (0.007)
Education Attainment of HH Head is Elementary School		0.008 (0.008)	Jewelry	0.034*** (0.006)
Education Attainment of HH Head is Junior School		0.036*** (0.010)	Chicken	−0.001 (0.006)
Education Attainment of HH Head is Senior High School or higher		0.041*** (0.011)	Caribou / Cow	0.065*** (0.012)
Observations	40398	38336

Table 11 illustrates that the community has residual information. Consumption is still highly correlated with individuals’ ranks of other households from the baseline survey even after we control for the rank from the PMT (e.g. all the information that is contained in the PMT): a one percentile increase in consumption rank is associated with a 0.132 percentile increase in individual household ranks of the community (Column 1). This is significant at the 1 percent level. In the more flexible specification (Column 2), the correlation between consumption rank and survey rank remains positive (0.088) and significant at the 1 percent level.

The findings in Table 11 suggest that the community has residual information about consumption beyond that contained in the PMT score or even in the PMT variables. Moreover, the fact that almost all the PMT variables enter into the community ranks with plausible magnitudes suggests that the community has most of the information in the PMT as well, but chooses to aggregate it in different ways. While we cannot completely rule out that the community lacks some information that is present in the PMT, the evidence here suggests that differences in information are not the primary drivers of the different results.

VIII.B. A Different View of Individual Welfare

Table 12 explores the relationship between the welfare metrics (community survey rank r_c, elite survey rank r_e, and self-assessment rank r_s), the targeting results in PMT, community, and hybrid villages, and a variety of household characteristics that might plausibly affect either the welfare functions or the social welfare weights used in targeting. In Columns 1 - 3, we present results of specifications where the dependent variable is the within-village rank of each household in the baseline survey according to different survey-based welfare metrics. In Columns 4 – 6, the dependent variable is the treatment rank, put on a corresponding metric where the lowest ranked (poorest) household in the dataset in each village is ranked 0 and the highest ranked (richest) household in the dataset in each village is ranked 1.³⁸ We control for the log of per capita consumption in all regressions, and therefore the coefficients can be interpreted as conditional on per-capita consumption. Thus, we identify where the community rankings deviate from a ranking based on consumption. We examine four dimensions on which villages may deviate: household demographic composition, ability to smooth shocks, discrimination against minorities or other marginal community members, and earning ability.

Table 12.

What is the community maximizing?

	Rank according to welfare metric...			Targeting Rank List in...
	Community survey ranks (rc)	Sub-village head survey ranks(re)	Self-Assessment (rs)	PMT villages	Community villages	Hybrid villages
	(1)	(2)	(3)	(4)	(5)	(6)
Log per-capita consumption	0.176*** (0.008)	0.145*** (0.008)	0.087*** (0.004)	0.132*** (0.013)	0.197*** (0.014)	0.162*** (0.014)
Panel A: Household demographics
Log HH size	0.164*** (0.011)	0.134*** (0.010)	0.073*** (0.006)	−0.028 (0.019)	0.154*** (0.019)	0.078*** (0.021)
Share kids	−0.125*** (0.021)	−0.094*** (0.021)	−0.037*** (0.012)	−0.296*** (0.035)	−0.068* (0.041)	−0.141*** (0.039)
Panel B: Ability to smooth shocks
Elite connected	0.092*** (0.008)	0.044*** (0.009)	0.025*** (0.005)	0.062*** (0.016)	0.051*** (0.015)	0.043*** (0.015)
Total connectedness	−0.039*** (0.010)	−0.021** (0.009)	−0.015*** (0.005)	−0.016 (0.017)	−0.019 (0.017)	−0.054*** (0.019)
Number of family members outside sub-village	0.012*** (0.004)	0.010*** (0.003)	0.006*** (0.002)	0.020*** (0.006)	0.001 (0.006)	0.001 (0.006)
Participation through work to community projects	0.002 (0.011)	0.021** (0.010)	0.005 (0.006)	0.000 (0.018)	0.010 (0.019)	0.003 (0.019)
Participation through money to community projects	0.061*** (0.009)	0.041*** (0.009)	0.024*** (0.005)	0.056*** (0.016)	0.058*** (0.016)	0.034* (0.018)
Participation in religious groups	0.027*** (0.010)	0.033*** (0.010)	0.014** (0.006)	0.033** (0.016)	0.012 (0.017)	0.029 (0.017)
Total savings	0.000 (0.000)	0.000 (0.000)	0.000 (0.000)	0.000 (0.000)	0.000 (0.000)	0.000 (0.000)
Share of savings in a bank	0.096*** (0.011)	0.069*** (0.010)	0.052*** (0.006)	0.121*** (0.018)	0.103*** (0.021)	0.075*** (0.020)
Debt as share of consumption	0.005*** (0.001)	0.004*** (0.001)	0.002*** (0.000)	0.002 (0.002)	0.007*** (0.001)	0.008*** (0.001)
Panel C: Discrimination against minorities?
Ethnic minority	−0.024* (0.014)	−0.019 (0.014)	−0.003 (0.008)	0.012 (0.026)	−0.051** (0.025)	−0.011 (0.024)
Religious minority	0.012 (0.018)	−0.007 (0.017)	−0.014* (0.008)	−0.018 (0.030)	0.025 (0.032)	0.012 (0.033)
Panel D: Correcting for earnings ability
HH head with primary education or less	−0.028*** (0.009)	−0.025*** (0.009)	−0.037*** (0.005)	−0.108*** (0.017)	−0.011 (0.018)	−0.066*** (0.017)
Widow	−0.104*** (0.014)	−0.083*** (0.014)	−0.012 (0.008)	0.009 (0.027)	−0.108*** (0.024)	−0.026 (0.028)
Disability	−0.045*** (0.016)	−0.037*** (0.014)	−0.026*** (0.008)	−0.079*** (0.027)	0.009 (0.026)	0.012 (0.027)
Death	−0.041* (0.025)	−0.031 (0.025)	−0.010 (0.015)	−0.111*** (0.042)	−0.013 (0.048)	−0.059 (0.043)
Sick	−0.038*** (0.011)	−0.041*** (0.011)	−0.028*** (0.006)	0.007 (0.018)	−0.018 (0.019)	−0.044** (0.019)
Recent shock to income	−0.001 (0.009)	−0.005 (0.009)	−0.013** (0.005)	−0.019 (0.016)	0.009 (0.016)	−0.012 (0.017)
Tobacco and alcohol consumption	−0.0002*** (0.000)	−0.0002*** (0.000)	−0.0001*** (0.000)	−0.0002*** (0.000)	−0.0002*** (0.000)	−0.0001*** (0.000)
Observations	5337	4680	5724	1814	1876	1889

Notes: Note that the children and household head education variables are explicitly included in the PMT regression (see Table 12). The PMT regression also includes dummies for the household head being male, married, and male * married, which together will be closely correlated with the widow variable.

First, in Panel A, we examine whether villages make adjustments for equivalence scales. In our setting, the PMT is explicitly defined using per-capita consumption. Thus, it makes no adjustment for economies of scale in the household. By contrast, all of the community welfare functions (Columns 1-3) reveal that people believe that there are household economies of scale, so that conditional on per-capita consumption, those in larger households are considered to have higher welfare (as in Olken, 2005). Likewise, the same is true for the community ranking, which assigns almost an identical household size premium (Column 5). Interestingly, for a given household size and consumption, all methods rank households with more kids as poorer, even though children generally cost less than adults (Deaton, 1997).

Second, the community may know more about other households’ ability to smooth shocks. Conditional on current consumption, the household that is better able to smooth shocks may be at a higher long-run expected utility level and therefore may need transfers less. For example, if two families have the same per capita consumption, the one that is more elite connected may worry less about bad shocks because it can expect to get help from rich relatives. The community might therefore feel that elite connected households are richer than their consumption indicates. Whether or not this is the correct theory, it aligns perfectly with what we find. In Panel B, we show that the community survey ranks put about a 9 percentage point premium on being elite connected, and even the elite and self-assessed survey ranks place a 4.4 and 2.5 percentage point premium, respectively. The community treatment ranks place a 5.1 percentage point premium on elite connectedness.

Similarly, there appears to be a premium for being better connected to the financial system. While total savings does not affect the rank, households that have a greater share of savings in a bank are classified as richer in both the individual surveys (Column 1-3) and the community meeting (Column 5). Households with family outside the village (who can presumably send remittances), are also ranked as less poor in terms of individual ranks, sub-village head ranks and the self-assessment, though not in the community meetings.

Third, in Panel C we test for discrimination against minorities or other marginal community members. We find no evidence of this: ethnic minorities are more likely to be ranked as poor in the community treatment, suggesting perhaps that even extra care is paid to them in the interest of social harmony (Column 5). In addition, we find no evidence of favoring families that are more engaged with the community. Contributing labor to village projects does not affect a family's status. However, those who contribute money are viewed as rich (Column 1-3), though they are also likely to be ranked as richer by the PMT (Column 4).

Finally, in Panel D we find suggestive evidence that communities may try to provide the “right” incentives to households. For example, in a standard Mirlees (1971) framework, one would ideally like to target on ability to earn, rather than actual earnings, so as not to disincentivize households. To test whether this is what communities are doing in practice, we first look at the education level of the household head. Households where the household head has a primary education or less rank 2-4 percentage points poorer, conditional on their actual consumption. Similarly, households headed by a widow, those with a disability, and those where there is a serious illness are all rated poorer, conditional on actual consumption. The adjustment for widowhood is also reflected in the community treatment ranking, but not the disability and serious illness adjustments (Column 5).³⁹ Finally and rather interestingly, the village does not penalize those who spend a lot of money on smoking and drinking. Families with these attributes are actually ranked lower both in community surveys (Column 1-3), and community meetings (Column 5), suggesting that the village treats these preferences as problems for the family as a whole rather than as behaviors that should be punished.

IX. Conclusion

The debate regarding decentralization in targeting is usually framed in terms of the benefits of utilizing local information versus the costs of some form of malfeasance, such as elite capture. While we started with an experiment that took both of these ideas very seriously, our results point to a third factor as being very important: the community seems to have a widely shared objective function other than per-capita consumption, and implementing this objective is a source of widespread satisfaction in the community. Moreover, this objective function does not differ based on elite capture. Rather, these preferences appear to be informed by a better understanding of factors that affect a household's earning potential or vulnerability, such as the returns to scale within the family, as compared to relying purely on consumption.

Given these findings, if targeting the poor based on consumption is the only objective, the PMT does perform somewhat better than the community methods, though the difference between the methods in terms of the ultimate poverty impact for a typically-sized program is not significantly different. Especially given the relatively small differences in ultimate poverty outcomes between the alternative treatments, it is not evident that there is a strong enough case to overrule the community's preferences in favor of the traditional consumption metric of poverty, especially given the gain in satisfaction and legitimacy. On the other hand, what is clear is that there is no case for the intermediate hybrid method: it resulted in both worse targeting performance and low legitimacy. This may be because its main theoretical advantage— preventing elite capture—was not important in our setting. It is possible that perhaps alternative hybrid designs (e.g. using a PMT process in the first stage but then allowing the community to add some very poor households to the resulting beneficiary list) might perform better than those where the selection process is ultimately determined strictly by the PMT survey results, as the community does better at identifying the very poor.

The findings in this paper raise several interesting questions. First, while we found little evidence of elite capture, it is possible that this might change over time as individuals learn to better manipulate the system. Manipulation over time has been shown to occur in some kinds of PMT systems (Camacho and Conover, 2011), but whether it would occur when the per-village allocation is fixed, and whether it would be more or less severe in community-targeted systems, are important open questions. Second, given how well the community outcomes match individual self-assessments, an important question is whether a self-targeting system (perhaps connected to an ordeal mechanism as in Nichols and Zeckhauser (1982)) could provide a more cost-effective method. We regard these as important questions for future research.

Appendix Table 3.

Results of Different Targeting Methods on Error Rate - Time elapsed between survey and targeting

	(1)	(2)	(3)	(4)	(5)	(6)	(7)	(8)
		By Income Status		By Detailed Income Status				Per-capita consumption of beneficiaries
Sample:	Full population	Inclusion Error	Exclusion Error	Rich	Middle income	Near Poor	Very Poor
Community treatment	0.088 (0.072)	0.098 (0.074)	0.042 (0.129)	0.090 (0.086)	0.102 (0.111)	0.127 (0.170)	−0.072 (0.178)	68.008 (78.501)
Hybrid treatment	0.018 (0.072)	0.074 (0.071)	−0.226* (0.125)	0.023 (0.081)	0.117 (0.108)	−0.252 (0.166)	−0.227 (0.176)	5.139 (90.750)
Time elapsed	0.000 (0.001)	−0.000 (0.001)	0.001 (0.002)	−0.001 (0.002)	−0.000 (0.002)	0.004 (0.003)	−0.002 (0.003)	0.759 (1.552)
Time elapsed x Community Treatment	−0.001 (0.002)	−0.001 (0.002)	−0.001 (0.003)	−0.001 (0.002)	−0.001 (0.003)	−0.003 (0.004)	0.002 (0.004)	−1.358 (1.852)
Time elapsed x Hybrid Treatment	0.000 (0.002)	−0.001 (0.002)	0.005 (0.003)	0.000 (0.002)	−0.001 (0.003)	0.005 (0.004)	0.005 (0.004)	−0.322 (2.049)
Observations	5595	3617	1978	1791	1826	1052	926	1687
Mean in PMT treatment	0.30	0.18	0.52	0.13	0.23	0.55	0.48	366

Notes: All regressions include stratum fixed effects. Robust standard errors are shown in parentheses, adjusted for clustering at the village level. All coefficients are interpretable relative to the PMT treatment, which is the omitted category. The mean of the dependent variable in the PMT treatment is shown in the bottom row. All specifications include stratum fixed effects.

*** p<0.01

** p<0.05

^*p<0.1

Appendix Table 6.

Are elite results driven by social connections?

	(1)	(2)	(3)	(4)
	Error		On beneficiary list dummy
Elite connectedness	−0.034 (0.021)	−0.034 (0.021)	−0.078*** (0.023)	−0.078*** (0.023)
Connectedness	0.041* (0.023)	0.041* (0.023)	0.067*** (0.022)	0.067*** (0.022)
Elite connectedness × community treatment	−0.010 (0.035)	−0.002 (0.039)	−0.064* (0.034)	−0.075** (0.037)
Elite connectedness × hybrid treatment	0.003 (0.034)	−0.004 (0.036)	−0.022 (0.034)	−0.010 (0.037)
Elite connectedness × elite treatment	−0.032 (0.031)	−0.050 (0.047)	0.040 (0.031)	0.062 (0.043)
Elite connectedness × elite treatment × hybrid		0.030 (0.064)		−0.050 (0.061)
Connectedness × community treatment	−0.002 (0.038)	−0.026 (0.043)	0.008 (0.036)	0.019 (0.041)
Connectedness × hybrid treatment	0.041 (0.037)	0.064 (0.041)	0.055 (0.035)	0.042 (0.037)
Connectedness × elite treatment	−0.000 (0.035)	0.043 (0.051)	−0.004 (0.032)	−0.029 (0.047)
Connectedness × elite treatment × hybrid treatment		−0.090 (0.071)		0.050 (0.065)
Observations	5753	5753	5756	5756

Notes: All specifications include dummies for the community, hybrid, and elite treatment main effects, as well as stratum fixed effects; columns (2) and (4) also include a dummy for elite x hybrid. Robust standard errors in parentheses, adjusted for clustering at the village level. Dependent variable in columns (1) and (2) is the mis-target dummy for the full sample, as in column (1) of Table 4. Dependent variable in columns (3) and (4) is a dummy for being a beneficiary of the program.

^***p<0.01

^**p<0.05

^*p<0.1

Appendix Table 7.

Rank correlation matrix of alternative welfare metrics

	(1)	(2)	(3)	(4)
	Consumption (rg)	Community survey ranks (rc)	Sub-village head survey ranks (re)	Self-Assessment (rs)
Consumption (rg)	1.000
Community survey ranks (rc)	0.376	1.000
Sub-village head survey ranks(re)	0.335	0.746	1.000
Self-Assessment(rs)	0.264	0.447	0.407	1.000

Notes: This table reports the correlation matrix between the within-village ranks of the four variables listed. All correlations are statistically significantly different from 0 at the 1% level.

Appendices: Not For Publication

Appendix A: Checking for randomization balance

To check whether the randomization is balanced, we chose ten variables for this check prior to obtaining the data from the experiment. Specifically, we examined the following characteristics from the baseline survey: per capita expenditures, years of education of the household head, calculated PMT score, the share of households that are agricultural, and the years of education of the sub-village head. We also examined five village characteristics from the 2008 PODES, a census of villages conducted by BPS: log number of households, distance to district center in kilometers, log size of the village in hectares, the number of religious buildings per household, and the number of primary schools per household.

The results from this analysis are shown in Appendix Table 2. In Columns 1, 2, and 3, we present the mean of each variable for the sub-villages assigned to the PMT, community, and hybrid treatments, respectively. Standard deviations are listed below the means in brackets. We present the difference in means between the community and the PMT groups in Column 4, between the hybrid and the PMT in Column 5, and between the hybrid and the community in Column 6. In Columns 7 – 9, we replicate the analysis shown in Columns 4-6, but additionally control for stratum fixed effects. Robust standard errors are shown in parentheses in Columns 4 – 9. All variables are aggregated to the sub-village level; thus each regression includes 640 observations. In the final row of Table 3, we provide the p-value of a test of joint significance of the difference across each of the outcome variables.

The sub-villages appear to be generally well-balanced across the ten characteristics. Out f the sixty individual differences presented, three are statistically significant at the 5 percent level – precisely what one would expect from random chance. All of these significant differences are in Column 9, which compares the community and hybrid methods, controlling for stratum fixed effects. Specifically, controlling for stratum fixed effects, households in community locations have less education and are less likely to be agriculturists than households in the hybrid treatment, and hybrid villages have 8 percent fewer households than community villages. Looking at the joint significance tests across all ten variables considered, without stratum fixed effects, the only jointly significant difference is between the hybrid and the community (Column 6, p-value 0.089); with stratum fixed effects (Column 9), the p-value is 0.028. All results in this paper are robust to specifications that include these additional ten control variables.