Fair voting outcomes with impact and novelty compromises? Unravelling biases in electing participatory budgeting winners

Abstract

Participatory budgeting, as a paradigm for democratic innovations, engages citizens in the distribution of a public budget to projects, which they propose and vote for implementation. So far, voting algorithms have been proposed and studied in social choice literature to elect projects that are popular, while others prioritize a proportional representation of voters’ preferences, for instance, the rule of equal shares. However, the anticipated impact and novelty in the broader society by the winning projects, as selected by different algorithms, remains totally under-explored, lacking both a universal theory of impact for voting and a rigorous unifying framework for impact and novelty assessments. This article tackles this grand challenge towards new axiomatic foundations for designing effective and fair voting methods. This is through new and striking insights derived from a large-scale analysis of biases over 345 real-world voting outcomes, characterized for the first time by a novel portfolio of impact and novelty metrics. We find strong causal evidence that equal shares comes with impact loss in several infrastructural projects of different cost levels that have been so far over-represented. However, it also comes with a novel, yet over-represented, impact gain in welfare, education and culture. We discuss the broader implications of these results and how impact loss can be mitigated at the stage of campaign design and project ideation.

This article is part of the theme issue ‘Co-creating the future: participatory cities and digital governance’.

1. Introduction

Participatory budgeting [1–3] is a revolutionary approach to democratic governance, whereby citizens and local governments are actively involved in the decision-making process on the spending of public funds, which are usually carried out through rounds of deliberation alongside voting processes. Since its inception in the city of Porto Alegre, Brazil in 1989 and after more than 11,000 cases worldwide [4] with annual investments in the scale of €137M by countries such as Poland [5], participatory budgeting initiatives are gaining traction as an effective means to realize local grassroots democratic movements. In a participatory budgeting process [6–9], citizens initially propose their project ideas with estimated costs for implementation, given a total budget made available by the city. Initially, participatory budgeting processes involved internal evaluations and selection of proposed projects by the city council after deliberation rounds with citizen representatives. However, by now, recent participatory budgeting designs involve a voting process, where citizens decide on the set of projects to implement. Thus, citizens express their preferences using an input voting method (e.g. approval, cumulative, Knapsack [10]) and the final winning projects are calculated using a ballot aggregation method (e.g. utilitarian greedy [11], Phragmèn’s sequential voting [12] and equal shares [13]).

Fair voting outcomes with equal shares: Recently, the method of equal shares has been tested in the real world, for instance, the City Idea project in Aarau, Switzerland [14,15] or the projects in Wieliczka (Green Million) and Swiecie in Poland. This creates a significant momentum for key democratic innovations [16–18] and a breakthrough for fairer voting outcomes. In contrast to the standard ballot aggregation method of utilitarian greedy that simply selects the next most popular project (most received votes) as long as the available budget is not exhausted, the method of equal shares selects projects that aim for a fairer and more proportional representation of all voters, by assigning for every voter decision power for an equal part of the budget. The full explanation of axiomatic and algorithmic approaches to fair and proportional voting methods is out of the scope of this article and it can be found in earlier work [13,19,20]. Nevertheless, in practice equal shares results in voting outcomes that may sacrifice a large costly project that can be highly popular (among the ones with the top received votes), to ‘replace’ it with several smaller low-cost projects so that more voters are satisfied (yet, not to the same extent of satisfaction). For instance, in the participatory budgeting campaign of Aarau, the method of equal shares selected 10 more projects than utilitarian greedy (17 versus 7 out of 33), while strikingly, the third most popular project is sacrificed [14,15]. Figure 1 demonstrates the sacrifice of the top-4 most popular (as well as costly) projects by equal shares when applied to earlier empirical evidence of real-world election instances (Pabulib repository data [21]).

Figure 1.

Equal shares is likely to sacrifice projects among the top-4 most popular ones that would otherwise get elected with the utilitarian greedy method, motivating the study of a potential impact loss. Likewise, the most popular projects have a higher share of very expensive projects compared to other cost levels. Nevertheless, compared to utilitarian greedy, equal shares is likely to select (i) more projects that are (ii) less popular and (iii) less costly. The plot shows the cost distribution rate (bars) of the top-20 most popular projects, and the selection rate (Y-axis) of such top-20 most popular projects (X-axis) by utilitarian greedy and equal shares over 811 voting instances collected in Pabulib [21]. These are 613 approval, 103 cumulative and 95 ordinal voting instances counting the scores or votes that the proposed projects receive.

Do fair voting outcomes come with a price? A large-scale systematic investigation of these particular effects is the motivation and focus of this article. In particular, we hypothesize that the fairness of equal shares results in potential compromises of the anticipated impact and novelty of the projects in the voting outcomes. This critical inquiry is not made to claim any ineffectiveness or faulty design of equal shares on these aspects, although alternative aggregation methods that optimize for other qualities are plausible. It is also not made to claim that impact loss is a necessary condition for fairness. Instead, with this inquiry we aim to provide new insights to citizens, city authorities and designers of participatory budgeting campaigns about the proposed projects and their selection to be put for voting such that voting outcomes by equal shares are even more legitimate, impactful and mitigate for any potential (cost) biases (refer to §3). These insights, missing so far, are critical and timely, and they are expected to significantly strengthen the adoption of methods that promote fairness, such as equal shares. As Cho points out, political fairness is a complex phenomenon, and requires compromise and balancing competing interests so that members of all groups are represented [22]. For instance, infrastructure planning and investments to meet net-zero targets in cities can be costly [23,24]. To what extent is it likely for equal shares to disadvantage such projects because of their inherently costly nature, resulting in biases affecting sustainability? Or is it likely that multiple such type of projects have more chances to materialize when they have a more localized scope and collectively build, in a more bottom-up way, a sustainability impact at city scale? Do such biases over impact areas also affect in turn the voting turnout within different groups? For instance, Stewart et al. have highlighted the case of participatory budgeting held in Chicago’s 49th ward on how a biased voter turnout resulted in the selection of low-priority projects favouring only specific population demographics [25]. All these are some key questions that this study puts under scrutiny for the first time and which come with significant policy implications and merit.

On the challenge of measuring the impact of voting outcomes: Measuring the impact and novelty of voting outcomes is by itself a complex long-standing challenge, and one that this article addresses. The absence of a standard for impact evaluations on the outcomes of participatory budgeting processes is expected given the inherently ambiguous nature of the term impact. Impact is a multi-faceted concept and can be assessed differently by different stakeholders. The European Commission underscores the importance of conducting impact assessments for any public initiatives expected to yield substantial economic, social or environmental modifications [26]. Campbell et al. have conducted a scoping review on the impact of participatory budgeting processes on health and well being [27]. Beuermann & Amelina have presented their experimental findings of a participatory budgeting model carried out in Russia, reporting an increased citizens’ engagement in public decision making and raised revenue in the local tax collection [28]. Hajdarowicz conducted a qualitative analysis on the empowerment of women from participatory budgeting processes [29]. Cabannes presents case studies of participatory budgeting instances in four cities across Latin America on the initiatives to involve children and young people in the local governance of their cities [30]. There is a significant body of literature assessing the impact of participatory processes [31–34], however, no existing work evaluates the anticipated impact loss or gain by the winning outcomes of different ballot aggregation methods. This means that our work is distinguished from earlier efforts that focus on measuring the actual impact of the implemented projects. Instead, we assess the anticipated impact by the projects that constitute the voting outcome of a participatory budgeting process. In this regard, our approach does not replace existing efforts on impact assessment but rather complements them with a novel focus on anticipated impact determined by project selections of a different nature. Moreover, distinguishing impact areas of elected participatory budgeting projects is limited to measuring the popularity of impact areas on voters, including how well voters are represented by the voting outcomes [21,35]. Moreover, this earlier work is limited to 4 impact areas and 76 election instances [21]. Similarly, the recent empirical work by Nelissen on the winning outcomes by equal shares and utilitarian greedy focuses on 35 participatory budgeting instances in Amsterdam [36]. Our study makes a significant advancement in the field by providing stronger validity, an analysis of larger scale and novel universal insights on assessing impact based on new measurements that are not covered in earlier work.

A framework for measuring impact and novelty: We introduce a novel framework of impact and novelty measurements on voting outcomes and voters’ ballots for participatory budgeting (refer to figure 2). The framework introduces a number of impact and novelty metrics applied to different impact areas (and project beneficiaries) measured in terms of costs, number of projects and popularity (votes). It consists of the following three elements: (i) the ballot aggregation method that forms the voting outcomes. For this study, we focus on the utilitarian greedy (popularity-oriented) and the equal shares (fairness-oriented) methods. (ii) A number of anticipated impact areas (and beneficiaries) that characterize a proposed project. For instance, a new park with a playground is likely to create impact on the urban greenery and public space development, in particular for children and families. (iii) A number of metrics that characterize the impact areas within the sets of winning projects, proposed projects and both together. By bringing these three elements together, it becomes possible to assess a number of impact metrics for the voting outcomes of a ballot aggregation method. We also distinguish for a certain impact area the exclusive winning projects of a particular aggregation method; these are, for instance, the projects of this impact area selected by equal shares but not by utilitarian greedy, and vice versa. Based on this notion of exclusivity, we introduce novelty metrics that can be applied to the number, cost and popularity of projects belonging to an impact area. For both impact and novelty metrics, the loss (or gain) of equal shares over utilitarian greedy is measured as their difference (refer to table 6).

Figure 2.

Impact and novelty assessment framework for voting outcomes by different ballot aggregation methods in participatory budgeting. For each impact area, the metrics characterize the winning set (share), the proposal set (representation) and both the winning and proposal set (proportionality). These metrics are measured in terms of cost, number of projects or popularity (votes). These metrics are calculated both at the voting outcome and ballot level. The novelty metrics capture the exclusive winning projects by a ballot aggregation method and are distinguished into within-impact-area and between-impact-areas measurements (refer to §4 for more information).

Impact metrics: To measure the prevalence of an impact area at the level of a voting outcome or a ballot, we introduce three calculations, each with 3 units of measurement, making a total of 2 levels × 3 calculations × 3 units of measurement = 18 metrics to assess impact. Figure 2 shows a visual rationale of these impact calculations using Venn diagrams, while mathematical formulations are given in §4. The figures for each impact and novelty measurement can be found for reference in table 2. We also introduce a numerical toy example of all impact and novelty calculations in the electronic supplementary material, table S2.

The calculations include the following: (i) share—the prevalence of an impact area out of all impact areas in the voting outcome or the ballot. This metric characterizes the winning projects; (ii) representation—the prevalence of an impact area in the set of winning projects out of the prevalence of this impact area in the set of proposed projects. This metric characterizes the proposed projects; (iii) proportionality—the ratio between the prevalence of an impact area out of all impact areas in the voting outcome or the ballot, and this over the prevalence of this impact area out of all impact areas in the proposed projects. This metric characterizes both the winning and proposed projects.

Each of the three impact calculations measures the prevalence of the impact area in the set of winning or proposed projects as follows: (i) number of projects—this counts the projects that belong in the impact area; (ii) cost—this is the monetary value of the projects that belong in the impact area; and (iii) popularity—this is the voters’ support on the projects that belong in the impact area. Depending on the adopted input voting method, support is counted with the number of approvals or total score assigned to the projects.

Novelty metrics: These metrics characterize how unique the winning projects of a ballot aggregation method are. Two notions of novelty are distinguished: within-impact-area novelty that measures the exclusive winning projects in an impact area by a ballot aggregation method, out of the total winning projects in this impact area. In contrast, between-impact-areas novelty measures the exclusive winning projects in an impact area by a ballot aggregation method, out of the exclusive winning projects in all impact areas. In both cases, the unit of measurement can be the number, the cost or the popularity of projects in an impact area. Novelty measurements can be made for both voting outcomes and ballots. The latter reflects the novelty of project choices that voters make in an impact area. Mathematical formulations are given in §4.

Analysis of biases in 345 participatory budgeting elections: To acquire evidence for any potential impact and novelty loss by the ballot aggregation methods, we apply the proposed framework to the data of 345 out of 810 participatory budgeting elections instances, which are collected from the Pabulib [22] repository. These participatory budgeting instances are the ones that contain information about the impact areas and beneficiaries of the proposed projects. Each project has one or more labels for the following nine impact areas: education, health, welfare, culture, public transit and roads, public space, urban greenery, environmental protection and sport. Likewise, each project comes with one or more labels for the following eight beneficiaries: families with children, students, disabled people, children, adults, animals, youth and elderly. To control for the factor of cost when assessing the impact and novelty loss, the following additional labels are assigned to each project based on the cost quartile to which it belongs: very cheap, cheap, expensive or very expensive. Controlling for the cost of projects allows us to distinguish between (i) biases on project costs on which the method of equal shares relies for its calculations, and (ii) biases on the project impact area. We also conduct a conjoint analysis [37] to explore causal evidence and explain how the impact areas represented in the winning projects explain the voting outcomes by different ballot aggregation methods.

Real-world case studies of equal shares for impact loss mitigation: All the studied participatory budgeting elections used the utilitarian greedy method. It is likely that if another ballot aggregation method was used, the projects and even the choices of voters would be different, for instance, lower-cost projects would be proposed, and likely, in different impact areas. To address this threat of validity, we pick up the public voting outcomes [14,15,38] of two real-world participatory budgeting processes designed to run with the method of equal shares: (i) Green Million (Wieliczka) and (ii) City Idea (Aarau). We study these voting outcomes as case studies and hypothesize that they mitigate part of the overall impact and novelty loss.

2. Results

Four key results are illustrated in this article:

1. Equal shares results in voting outcomes with an impact loss in several infrastructural and sustainable development projects, which have shown over-representation. It also results in impact gain in welfare, education and culture, which have been under-represented. Strikingly, impact loss for such infrastructural and sustainable development projects is more frequent, while impact gain in welfare, education and cultural projects is larger in scale.

2. Equal shares results in novelty gain in terms of unique winning projects within each impact area. It also results in novelty gain for the unique winning projects of welfare, education and culture out of all unique winning projects. In all other cases, equal shares shows novelty loss.

3. Equal shares results in impact loss that originates from both high- and low-cost projects for the impact areas of infrastructural and sustainable development projects. Strikingly, high-cost sport projects show impact loss, while low-cost ones show impact gain. Equal shares results in impact gain for unpopular projects and impact loss for popular and high-cost projects.

4. The application of equal shares in two real-world participatory budgeting campaigns mitigates impact loss in public space, urban greenery, sport and, for one of the campaigns, in public transit. Culture and education show lower impact gain than anticipated.

(a). Impact loss and gain of equal shares

Equal shares has an impact loss in infrastructural and sustainable development projects: Figure 3 shows the impact loss from utilitarian greedy to equal shares measured with the six metrics distinguished by different impact areas. Projects related to infrastructural and sustainable development such as public space, public transit, urban greenery and environmental protection show higher impact loss. For instance, the highest mean loss of cost share is 7% for public space, 6% for urban greenery and 5% for public transit projects. A similar mean loss is observed for projects and popularity share as well as cost representation, however, the projects and popularity representation show impact gain. This means that although these impact areas are not so prevalent in the winning set, they remain well represented. The voting instances with the cost share losses are 54%, 36% and 38%, respectively, for public space, urban greenery and public transit. For the metrics of projects and popularity share, these are 57%, 60%, 48% and 58%, 58%, 47%, respectively, for these impact areas. The voting instances with losses in these impact areas are 56%, 40%, 41%, respectively, for cost representation, 8%, 15%, 14% for projects representation and 25%, 27% and 30% for popularity representation.

Figure 3.

Equal shares results in voting outcomes with a more frequent impact loss in infrastructural and sustainable development projects, while an impact gain of larger magnitudes is observed in welfare, education and culture projects. For the depicted metrics, (a) cost share, (b) projects share, (c) popularity share, (d) cost representation, (e) projects representation and (f) popularity representation, positive loss (Y-axis, UG − ES) in an impact area (coloured lines) show outperformance by utilitarian greedy (UG), while negative loss shows outperformance by equal shares (ES). The X-axis denotes election instances sorted according to impact loss. A circular marker is placed for each line to easily distinguish the number of voting instances with positive/negative loss. The four numbers next to each impact area denote the (i) % of voting instances with positive loss; (ii) the mean `$\sim$'; (iii) mean positive `+'; and (iv) mean negative `−' impact loss. Two additional numbers with the prefixes '+' and '−' placed on each of the metrics signify the overall mean positive and negative impact loss respectively across all impact areas.

Equal shares has an impact gain for culture, education and welfare: Unlike infrastructural and sustainable development projects, there is a net impact gain (as signified by negative values of mean loss) for culture, education and welfare projects across all six impact metrics in figure 3. For instance, the mean loss of cost representation is −25%, −20% and −15% for welfare, culture and education, respectively. Similar impact gain is observed for projects representation and popularity representation, which is significantly higher than the impact gain for the metrics of cost/projects/popularity share. This means a large portion of the proposed projects in these impact areas are elected. Moreover, the number of voting instances with impact losses is as low as 4%, 9%, 16% respectively for both metrics of cost share and cost representation in such impact areas. Similarly, the voting instances with losses in projects share and projects representation are 14%, 14%, 13% and 1%, 1% and 2%, respectively, which is the lowest among other impact areas. The popularity share and popularity representation show a similar pattern.

Equal shares balances between under-represented and over-represented impact areas: Figure 4 illustrates the performance of equal shares and utilitarian greedy for projects across different impact areas in terms of projects proportionality. We find that winning outcomes by equal shares are more proportionally represented compared to utilitarian greedy. A value of projects proportionality greater than 1 for an impact area means that the impact area is over-represented in the winning outcome, while a value less than 1 signifies under-representation. Particularly, equal shares increases the proportionality of under-represented impact areas such as education, culture and welfare, while it decreases the over-representation of urban greenery and environmental protection. These insights also align with those for the cost proportionality (electronic supplementary material, figure S5) and popularity proportionality (electronic supplementary material, figure S6).

Figure 4.

Equal shares shows impact gain in projects proportionality for education, welfare, culture and health, with an impact loss in all other impact areas. Equal shares over-represents the impact areas of culture and education, while it under-represents the area of public transit. In contrast, utilitarian greedy over-represents urban greenery, environmental protection and public space, while under-representing welfare projects. For each impact area, a projects proportionality value (Y-axis) of `1' represents a proportional representation of the impact areas in the winning outcome with respect to the proposed projects. The X-axis denotes voting instances sorted according to projects proportionality. The numbers in the parentheses next to each impact area denote the impact loss measured by the relative mean difference of projects proportionality between utilitarian greedy and equal shares with respect to the value of utilitarian greedy, i.e. (UG − ES)/UG.

Equal shares better represents project beneficiaries: The equal shares method also improves the projects representation of any demographic beneficiary. Table 1 shows that across all beneficiaries, the projects representation by equal shares increases significantly compared to utilitarian greedy, i.e. the winning outcome of equal shares includes projects from which all population groups benefit to a higher extent compared to utilitarian greedy. The increase in projects representation for population beneficiaries and a more proportional representation of various impact areas comes at the expense of sacrificing a few more costly projects for several less costly ones (refer to electronic supplementary material, figure S2).

Table 1.

Equal shares results in the selection of projects that increases projects representation across all beneficiaries compared to utilitarian greedy. The table shows the overall number of proposed projects across all 345 instances targeted towards specific beneficiaries. It also shows the corresponding values for projects representation by utilitarian greedy (UG) and equal shares (ES), including the relative loss in representation with respect to utilitarian greedy.

beneficiaries	total projects	UG projects representation (%)	ES projects representation (%)	relative loss (%)
disabled people	4266	28.3	61.9	−119
youth	6943	33	65.5	−98
elderly	6239	33.8	65.7	−94
children	7062	33.7	65	−93
adults	7656	35.3	66.6	−89
students	366	44.3	72.1	−63
families	2625	47.6	66	−39
animals	313	67.1	80.8	−20

How impact gains and losses align at the ballot and outcome level: Figure 5 illustrates for each impact area (a) the projects share and (b) the projects representation at the ballot level by equal shares and utilitarian greedy. The projects share at the ballot level comes with a loss for equal shares in the impact areas that also show a loss at the outcome level. This is 44%, 52%, 50% and 48% for public space, urban greenery, environmental protection and public transit, respectively. On the other hand, the ones with impact gain at the outcome level, that is, education, culture, welfare and health, come along at the ballot level with higher projects representation gain of 51%, 58%, 98% and 49%, respectively. The difference is particularly prominent for projects representation levels of $<0.6$. In contrast, for the impact areas with an impact loss at the outcome level, such as environmental protection, public space, public transit and urban greenery, representation at the ballot level remains similar for the two ballot aggregation methods.

Figure 5.

Equal shares preserves the voters’ satisfaction levels for projects with impact gain, while the satisfaction levels are reduced under projects with impact loss. Equal shares shows increased voters’ representation levels for projects with impact gain, while the representation levels are not influenced under projects with impact loss. For each impact area, the fraction of voters (Y-axis) for different levels of (a) projects share and (b) projects representation is shown along the X-axis for utilitarian greedy and equal shares. A total of 345 approval voting instances are counted, with the value of $N$ denoting the number of election instances with at least one proposed project belonging to the corresponding impact area. The numbers in the parentheses next to each impact area denote the impact loss measured by the relative mean difference of projects representation at the ballot level between utilitarian greedy and equal shares with respect to the value of utilitarian greedy, i.e. (UG − ES)/UG.

(b). Novelty loss and gain of equal shares

Novelty gain of equal shares: Figure 6 illustrates the within- and between-novelty loss (and gain) across different impact areas. Within-novelty gain is observed across all impact areas. The within-novelty gain is prominent for the impact areas of education, culture and welfare. For instance, in terms of cost and number of projects, the within-novelty loss is −39%, −52%, −75% and −48%, −56%, −76%, respectively. This means that out of the costs allocated to these impact areas, an additional 39%, 52% and 75% of costs can be attributed to exclusively winning projects by equal shares, which ultimately results in a net impact gain. Equal shares also exhibits between-novelty gain but of lower magnitude for the same impact areas of education, culture and welfare, i.e. in terms of costs and number of projects, the between-novelty losses are −30%, −28%, −13% and −36%, −33% and −13%, respectively.

Figure 6.

Equal shares shows within-novelty gain in almost all impact areas and between-novelty gain for education, culture and welfare. In all other impact areas, equal shares shows loss of between novelty. Within novelty comes with larger gains than losses, while between novelty comes with larger losses than gains. The figure shows different impact areas (Y-axis) and the corresponding novelty metric values (X-axis). The three columns compare the novelty gain and loss of equal shares over utilitarian greedy in different units: cost, number of projects and popularity.

Novelty loss of equal shares: Equal shares shows between-novelty losses for the impact areas of public transit, public space, urban greenery, sport and environmental protection. For instance, in terms of costs, the between-novelty losses for urban greenery, public space, public transit and environmental protection are 24%, 21%, 19% and 19%, respectively. Likewise, in terms of number of projects, the between-novelty losses for the same impact areas are 29%, 28%, 23% and 20%, respectively.

(c). Disentangling the interactions of cost, popularity and impact

Impact gain and loss when controlling for project cost: To disentangle the interactions between cost of projects and observed impact gain/loss, we control for the cost of the projects. The proposed projects within a single voting instance are classified into four cost levels—very cheap, cheap, expensive and very expensive—based on the quartile distribution of their costs. Figure 7 illustrates the average impact performance of equal shares and utilitarian greedy, and consequently the impact loss or gain of equal shares for projects in different impact areas across these cost levels.

Figure 7.

The impact loss of equal shares for public transit originates from very expensive projects in terms of cost share and projects share. In contrast, the impact loss of equal shares for environmental protection, urban greenery and public space originates from very cheap and cheap projects for the same metrics. Strikingly, very expensive sport projects show impact loss, while cheap and slightly expensive such projects show impact gain. In terms of cost and projects representation, impact loss is observed for very expensive projects but results in impact gain for all other cost levels across all impact areas. Impact values (Y-axis) of equal shares and utilitarian greedy across different impact areas for (a) cost share, (b) projects share, (c) popularity share, (d) cost representation, (e) projects representation and (f) popularity representation segregated at different cost levels (X-axis) are shown. The last row depicts the respective impact loss values (UG − ES).

Despite the promotion of low-cost projects by equal shares, very cheap urban greenery projects exhibit an average impact loss of 5% for cost share and projects share. However, equal shares achieves an impact gain of 4% for cost and projects representation. For very expensive urban greenery projects, the impact loss of equal shares in cost and projects share is minimal (0% and 1%, respectively), while loss in cost and projects representation is more significant (7% and 5%, respectively). Similarly, for very cheap environmental protection projects, the impact loss in cost and projects share by equal shares is 6% and 7%, respectively. However, equal shares achieves an impact gain of 5% in cost and projects representation.

On the other hand, for very cheap educational projects, impact is gained by equal shares for all metrics, i.e. cost share, projects share, cost and projects representation by 7%, 7%, 23% and 21%, respectively. For very cheap culture projects, the impact gained for these metrics is 4%, 4%, 15% and 14%, respectively. However, for very expensive educational projects, the impact gain of equal shares is 6% for cost share and projects share, respectively, while a 3% and 2% of impact loss is observed in cost and projects representation, respectively. Despite the very expensive projects in culture and welfare, equal shares results in an impact gain of 5%, 2% in cost share and 4%, 2% in projects share, respectively. However, impact is lost by a marginal value of 1% for such very expensive projects in culture and welfare in terms of cost and projects representation.

Impact gain and loss when controlling for popularity: The impact loss and gain of equal shares is assessed by controlling for different quartile levels of projects popularity: unpopular, quite popular, popular and very popular. In the representation metrics, equal shares shows a mean impact gain of 5% (cost representation), 8% (projects representation) and 9% (popularity representation) for unpopular projects but an impact loss of 3% (projects representation), 3% (popularity representation) and 8% (cost representation) for very popular projects (refer to electronic supplementary material, figure S4).

How project cost interacts with project popularity: For the winning projects in utilitarian greedy, the cost share and popularity share show a high correlation of 0.78, 0.79 and 0.81 (p-values = $1.01\times {10}^{-61}$, $4.17\times {10}^{-76}$,$1.05\times {10}^{-74}$) for urban greenery, public space and public transit, respectively. However, for such winning outcomes by utilitarian greedy, welfare and culture projects show relatively lower correlations of 0.75 (p‐value = $6.98\times {10}^{-36}$) and 0.69 (p‐value = $7.73\times {10}^{-47}$) that are significant (refer to electronic supplementary material, figure S3).

How impact areas and cost levels explain voting outcomes: a conjoint analysis: We conduct a conjoint analysis to explore the causal relationship in determining the budget utilization rate of equal shares and utilitarian greedy using the attributes of impact areas and cost levels. Eight independent variables are used (4 variables for popular combinations of impact areas × 2 variables for cost levels). Table 4 shows the most frequent combinations used that cover the range of impact areas. For instance, we observe that two out of four of these combination groups are (i) sustainable infrastructure projects and (ii) culture & educational projects. For each of these four independent variables, we further segregate them into two cost levels—low cost and high cost—resulting in a total of eight independent variables.

Based on the presence or absence of winning projects with these combinations at two different cost levels in the winning outcomes of equal shares and utilitarian greedy, we model a choice-based conjoint analysis problem to predict the budget utilization (overall cost share). The designed model results in a good fit for both equal shares and utilitarian greedy with $R^2=0.88$ and $R^2=0.82$, respectively, and significant p-values for all independent variables, i.e. p-value < 0.05 (refer to electronic supplementary material, table S7).

The relative importance (part-worth utilities) of different impact areas and cost levels by equal shares and utilitarian greedy is shown in figure 8. The conjoint analysis further reinforces the findings of our results. The figure illustrates that projects with the combination of education, culture at low cost contribute the most to budget utilization by both equal shares and utilitarian greedy (relative importance of 71.6% and 52.5%, respectively). However, the same combination of projects at higher cost levels contributes negatively to the budget utilization rate in both aggregation methods. Likewise, projects with combinations of environmental protection, public space and urban greenery across both cost levels contribute positively to the budget utilization in the case of utilitarian greedy, whereas such projects contribute negatively in the case of equal shares. This observation is congruent with results that suggest impact is lost by equal shares for sustainable infrastructure projects, while impact is gained for projects related to culture, education and welfare.

Figure 8.

The impact areas of culture and education with low cost have the highest importance in both equal shares and utilitarian greedy, while for high costs, both of these impact areas come with negative importance. This pattern is also observed for the combinations of health, public space, sport and public space, public transit. In contrast, for the impact area of environmental protection, public space and urban greenery, the ballot aggregation method is more important than the cost: positive importance for both cost levels by utilitarian greedy, negative importance for both cost levels by equal shares. The most frequent combinations of impact areas (X-axis) are shown across cost levels and their relative importance (Y-axis, part-worth utilities) contributing to the budget utilization by equal shares and utilitarian greedy.

(d). Equal shares in real world: the cases of City Idea & Green Million

Does equal shares mitigate impact loss in real world? Here, we explore the impact loss and gain of equal shares across different impact areas for the recently implemented participatory budgeting instances in Aarau (City Idea Project) and Wieliczka (Green Million), in which winning outcomes were determined using a variant of the equal shares method (add1 using integral endowments for Green Million and add1u for City Idea [39]). The impact loss (or gain) on the winning outcomes by equal shares for these two cases along with the mean values of all voting instances is shown in figure 9.

Figure 9.

The application of equal shares in real world mitigates expected impact loss in public space and urban greenery for cost share and projects share, while impact loss is observed in culture and education and impact gain in sport. A significant impact gain for public transit is observed in the Green Million campaign in Poland. Welfare in the City Idea campaign in Switzerland shows impact gain for cost share and projects share. The impact loss (Y-axis) for different impact metrics across different impact areas (X-axis) for (i) all 345 voting instances (mean); (ii) the Green Million campaign in Wieliczka, Poland; and (iii) the City Idea campaign in Aarau, Switzerland is shown.

The winning outcomes of equal shares for the Green Million participatory budgeting instance are different from the expected winning outcomes of equal shares based on the past voting instances. Projects under the impact areas of public transit, public space, urban greenery and environmental protection achieve an additional impact gain instead of the expected impact loss for such projects. For instance, public space projects achieve an impact gain of 6% in cost share instead of an expected impact loss of 7%. Likewise, environmental protection projects show an impact gain of 8% in projects share instead of an expected impact loss of 6%. This is because the Green Million Project, as its name suggests, was particularly aimed at promoting eco-friendly ‘green’ projects such that most of the proposed projects are related to environmental protection, urban greenery and public space. Likewise, due to the absence of projects related to education, culture and welfare for this campaign, an unexpected and significant impact loss is observed for such impact areas in all metrics. For instance, educational projects have a cost share loss of 5% rather than the expected gain of 10%, while welfare projects have a projects share loss of 6%. In case of the City Idea campaign in Aarau, the winning outcomes by equal shares exhibit the expected impact gains for welfare (an average of 17% across all metrics) and the expected impact loss for public transit (8% projects share loss and 4% popularity share loss) and environmental protection (18% cost share loss, 19% projects share loss, 8% popularity share loss and 16% cost representation loss). However, unlike the expected impact gains of equal shares for projects related to education, a significant impact loss for educational projects (8% cost share loss, 5% projects share loss and 7% projects representation loss) is observed. Moreover, a significant impact gain is observed for projects related to urban greenery (9% projects share gain and 9% popularity representation gain).

3. Discussion

The findings and key results come with some significant implications. Equal shares gains momentum with radically different voting outcomes in participatory budgeting: a larger number of lower-cost projects are elected (refer to electronic supplementary material, figure S2), often ‘replacing’ expensive popular projects, creating a fundamentally different impact for the society as a result of higher proportional representation of voters’ preferences. We show that there are some opportunities and risks here. In particular, infrastructural projects, especially ones related to sustainability, may be frequently disadvantaged by equal shares, as a result of their inherently costly nature but also beyond this (refer to figure 7). On the other hand, equal shares strongly favours novel welfare, education and cultural projects, which is likely a result of their lower cost in the context of participatory budgeting.

The risk of a democratic deficit towards the endeavour of sustainability and net zero requires attention and mitigation. Apparently, letting infrastructural and sustainability projects be planned in a top-down way is a major threat to legitimacy [40] by creating vulnerabilities for corruption and even green washing by large corporate organizations with influence in governments. Preserving capacity in participatory budgeting campaigns to materialize infrastructural and sustainability projects that fairly benefit the whole population is a key challenge to address in the future. The new findings here suggest that participatory budgeting campaigns with projects of high discrepancies of cost may be more effective when they are broken down into multiple campaigns with more local and synergistic effects in their implementation across different localities, for instance, low-cost incremental expansions of bike lanes across a city, instead of centrally deciding the creation of a large multi-million bike lane network. The possibility of such substitution or complementarity effects among proposed projects is also discussed by Jain et al. [41]. Furthermore, the insights of this study are also pertinent to national elections when considering the political agendas of different candidates and in which impact areas their priorities lie, for instance, climate change and social welfare.

We also show evidence from the first recent real-world participatory budgeting campaigns conducted using the method of equal shares that the mitigation of this impact loss is feasible. Awareness about how the ballot aggregation method works and what it prioritizes results in different proposed projects, with different (lower) costs and, eventually, different preferences that yield impact recovery for public transit and sport (refer to figure 9).

This study provides new insights for several beneficiaries: for citizens and communities, the results unravel new pathways for democratic social innovations and the preservation of the capacity to innovate as such. They also provide further empirical evidence and understanding of more complex voting methods such as equal shares, and its effectiveness. For policymakers, the findings of this work can support a more effective design of participatory budgeting campaigns that mitigate for potential impact losses already at the stage of project proposals and feasibility checks. They also support them to align ambitious net-zero policy agendas with citizens’ participation in their implementation and democratic legitimacy [34]. For the academic community of (computational) social choice and beyond, the insights of our study motivate for new axiomatic foundations to assess fairness, proportionality and voting methods. Last but not least, for industry, a more localized, distributed and synergistic implementation of infrastructural and sustainability projects with direct citizens’ engagements suggests the need for new business models aligned with democratic values.

4. Methods

This section outlines the empirical dataset used to apply the impact assessment framework as well as the mathematical formulations of all impact and novelty metrics. We also outline how we account for threats to validity. Table 2 outlines the metrics and the figures that illustrate the respective impact assessment results.

Table 2.

Metrics [impact & novelty]—results mapping.

dimension	# of projects		cost		popularity
	outcome	individual ballot	outcome	individual ballot	outcome
share	figure 3	figure 5	figure 3	electronic supplementary material, figure S7	figure 3
representation	figure 3	figure 5	figure 3	electronic supplementary material, figure S8	figure 3
proportionality	figure 4	electronic supplementary material, figure S9	electronic supplementary material, figure S5	electronic supplementary material, figure S10	electronic supplementary material, figure S6
between-novelty and within-novelty impact areas = figure 6

(a). Empirical dataset

The data on participatory budgeting elections were collected in August 2023 from the Pabulib [22] open repository. The repository consists of multiple files which signify the details of a particular participatory budgeting voting instance defined by a standard .pb file. Such files contain details on project costs, votes received, project description and the total available budget. In reality, for all except two of the voting instances from the collected dataset, the winning outcomes were determined using utilitarian greedy. To determine the hypothesized winning outcomes by equal shares, we used the pabutools [39] library. While calculating the winning outcomes by equal shares, we added the conditions of arbitrary budget increments and utilitarian termination (add1u [39]) to ensure high utilization of budget.

The dataset consists of 810 participatory budgeting instances. Out of these, only 345 instances have information on project impact areas and population beneficiaries. Table 3 shows the overall and average distribution of projects across the nine different impact areas and eight different beneficiaries. The distribution of number of projects, their costs as well as their popularity across different impact areas is shown in electronic supplementary material, figure S1. Table 4 shows the most frequent combination of project impact areas from the observed dataset used in the conjoint analysis.

Table 3.

Distribution of proposed projects across different impact areas and beneficiaries.

impact areas / beneficiaries	total projects	avg. number of projects
education	4336	13.3
health	1408	4.6
culture	3019	9.4
sport	2760	8.4
public transit	1973	6.3
welfare	783	4.2
urban greenery	2534	8.6
public space	5512	16.4
environmental protection	2093	7.5
families	2625	7.7
students	366	1.1
disabled People	4266	12.4
children	7062	20.5
adults	7656	22.3
elderly	6239	18.1
animals	313	0.9
youth	6943	20.2

Table 4.

Most frequently observed mutually exclusive combination of labels for project impact areas.

project tags combination	number of projects
culture, education	1227
environmental protection, public space, urban greenery	966
public space, public transit and roads	755
—	—
health, public space, sport	261

(b). Impact and novelty metrics

Table 5 outlines the list of mathematical symbols and their meanings, which define the impact and novelty metrics. They are assessed on the voting outcomes and the individual voters’ ballots. In the following sections, we rigourously introduce how the impact and novelty metrics are calculated at the voting outcome level and the ballot level. For both the impact and novelty metrics, the loss (or gain) value of equal shares relative to the utilitarian greedy approach is quantified as the difference between the two values, as presented in table 6. Furthermore, a toy voting example with 11 voters and three different impact areas shows how the proposed impact and novelty metrics are calculated (refer to electronic supplementary materials, tables S1 and S2).

Table 5.

List of mathematical symbols.

symbol	interpretation
${\displaystyle o}$	voting outcome
$b$	individual ballot
${\displaystyle l}$	an impact area
$L$	set of impact areas
${\displaystyle p}$	a proposed project
$P$	set of proposed projects
${\displaystyle c_p}$	cost of a project ${\displaystyle p}$, where ${\displaystyle p}$ ${\displaystyle \in }$ ${\displaystyle P}$
$P_l$	set of proposed projects belonging to an impact area ${\displaystyle l}$, where ${\displaystyle l\in L}$
$V$	set of voters
${\displaystyle B_v}$	set of approved projects for a voter ${\displaystyle v\in V}$
$v_p$	votes received by a project ${\displaystyle p}$, where ${\displaystyle p}$ ${\displaystyle \in }$ ${\displaystyle P}$
${\displaystyle u}$	utilitarian greedy
$e$	equal shares
${\displaystyle f}$	ballot aggregation method, i.e.${\displaystyle f=\{e,u\}}$
$W_f$	set of proposed projects that are winners in the outcome of an aggregation method ${\displaystyle f}$
${\displaystyle {\hat{W}}_f}$	set of proposed projects that are exclusive winners in the outcome of an aggregation method ${\displaystyle f}$
$W_{l,f}$	${\displaystyle P_l\cap W_f}$
${\widehat{W}}_{l,f}$	${\displaystyle P_l\cap {\hat{W}}_f}$
$W_{l,v,f}$	${\displaystyle P_l\cap B_v\cap W_f}$
${\widehat{W}}_{l,v,f}$	${\displaystyle P_l\cap B_v\cap {\hat{W}}_f}$
$r_{c,l}$	ratio of total cost of projects in the impact area ${\displaystyle l}$ over the total cost of proposed projects, i.e.${\displaystyle \frac{\sum _{p=1}^{|P_l|}{c}_p}{\sum _{p=1}^{|P|}{c}_p}}$
${\displaystyle r_{n,l}}$	ratio of number of proposed projects in the impact area ${\displaystyle l}$ over the total number of proposed projects, i.e.${\displaystyle \frac{|P_l|}{|P|}}$
${\displaystyle r_{v,l}}$	ratio of total votes received by projects in the impact area ${\displaystyle l}$ over the total votes received by proposed projects, i.e.${\displaystyle \frac{\sum _{p=1}^{|P_l|}{v}_p}{\sum _{p=1}^{|P|}{v}_p}}$
$i$	impact metric
${\displaystyle \omega }$	within-novelty metric
$\beta$	between-novelty metric

Table 6.

Loss or gain calculation for impact and novelty metrics.

impact	impact loss (cost)		impact loss (projects)		impact loss (popularity)
	outcome	individual ballot	outcome	individual ballot	outcome
share	${\displaystyle S_{\text{o,c,l}}^{\text{i,u}}-S_{\text{o,c,l}}^{\text{i,u}}}$	${\displaystyle S_{\text{b,c,l}}^{\text{i,u}}-S_{\text{b,c,l}}^{\text{i,u}}}$	${\displaystyle S_{\text{o,n,l}}^{\text{i,u}}-S_{\text{o,n,l}}^{\text{i,u}}}$	${\displaystyle S_{\text{b,n,l}}^{\text{i,u}}-S_{\text{b,n,l}}^{\text{i,u}}}$	${\displaystyle S_{\text{o,v,l}}^{\text{i,u}}-S_{\text{o,v,l}}^{\text{i,u}}}$
representation	${\displaystyle R_{\text{o,c,l}}^{\text{i,u}}-R_{\text{o,c,l}}^{\text{i,u}}}$	${\displaystyle R_{\text{b,c,l}}^{\text{i,u}}-R_{\text{b,c,l}}^{\text{i,u}}}$	${\displaystyle R_{\text{o,n,l}}^{\text{i,u}}-R_{\text{o,n,l}}^{\text{i,u}}}$	${\displaystyle R_{\text{b,n,l}}^{\text{i,u}}-R_{\text{b,n,l}}^{\text{i,u}}}$	${\displaystyle R_{\text{o,v,l}}^{\text{i,u}}-R_{\text{o,v,l}}^{\text{i,u}}}$
proportionality	${\displaystyle P_{\text{o,c,l}}^{\text{i,u}}-P_{\text{o,c,l}}^{\text{i,u}}}$	${\displaystyle P_{\text{b,c,l}}^{\text{i,u}}-P_{\text{b,c,l}}^{\text{i,u}}}$	${\displaystyle P_{\text{o,n,l}}^{\text{i,u}}-P_{\text{o,n,l}}^{\text{i,u}}}$	${\displaystyle P_{\text{b,n,l}}^{\text{i,u}}-P_{\text{b,n,l}}^{\text{i,u}}}$	${\displaystyle P_{\text{o,v,l}}^{\text{i,u}}-P_{\text{o,v,l}}^{\text{i,u}}}$

novelty	novelty loss (cost)		novelty loss (projects)		novelty loss (popularity)
	outcome		outcome		outcome
between novelty	${\displaystyle S_{\text{o,c,l}}^{\beta \text{,u}}-S_{\text{o,c,l}}^{\beta \text{,u}}}$		${\displaystyle S_{\text{o,n,l}}^{\beta \text{,u}}-S_{\text{o,n,l}}^{\beta \text{,u}}}$		${\displaystyle S_{\text{o,v,l}}^{\beta \text{,u}}-S_{\text{o,v,l}}^{\beta \text{,u}}}$
within novelty	${\displaystyle S_{\text{o,c,l}}^{\omega \text{,u}}-S_{\text{o,c,l}}^{\omega \text{,u}}}$		${\displaystyle S_{\text{o,n,l}}^{\omega \text{,u}}-S_{\text{o,n,l}}^{\omega \text{,u}}}$		${\displaystyle S_{\text{o,v,l}}^{\omega \text{,u}}-S_{\text{o,v,l}}^{\omega \text{,u}}}$

Impact metrics calculated at the voting outcome level: The prevalence of the impact areas is measured in terms of their share, representation and proportionality in the voting outcome $o$ of an aggregation method $f$. For different voting aggregation methods, each of the impact metrics is measured in terms of (a) cost, (b) number of projects and (c) popularity to assess the impact on the voting outcome.

Share: This metric quantifies the winning rate of projects from a specific impact area with respect to the winning outcome. For an aggregation method $f$, share is calculated as the fraction of the total cost, number or popularity of projects in the winning outcome that belongs to a given impact area $l$:

cost share	projects share	popularity share
${\displaystyle S_{o,c,l}^{i,f}=\frac{\sum _{p=1}^{|W_{l,f}|}{c}_p}{\sum _{p=1}^{|W_f|}{c}_p}}$	${\displaystyle S_{o,n,l}^{i,f}=\frac{|W_{l,f}|}{|W_f|}}$	${\displaystyle S_{o,v,l}^{i,f}=\frac{\sum _{p=1}^{|W_{l,f}|}{v}_p}{\sum _{p=1}^{|W_f|}{v}_p}}$

Representation: This metric quantifies the representation rate of a specific impact area in the winning set with respect to the set of proposed projects in that impact area. For an aggregation method $f$, representation is calculated as the fraction of the total cost, number or popularity of projects in a given impact area $l$, that belongs to the winning set:

cost representation	projects representation	popularity representation
${\displaystyle R_{o,c,l}^{i,f}=\frac{\sum _{p=1}^{|W_{l,f}|}{c}_p}{\sum _{p=1}^{|P_l|}{c}_p}}$	${\displaystyle R_{o,n,l}^{i,f}=\frac{|W_{l,f}|}{|P_l|}}$	${\displaystyle R_{o,v,l}^{i,f}=\frac{\sum _{p=1}^{|W_{l,f}|}{v}_p}{\sum _{p=1}^{|P_l|}{v}_p}}$

Proportionality: This metric characterizes the proportional representation of impact areas across both the winning and proposal sets. Given that a certain impact area constitutes a specific fraction out of all proposed projects, the metric signifies if a proportional fraction is maintained in the winning outcome for that impact area. For an aggregation method $f$, proportionality is calculated as the ratio of winning shares for an impact area $l$ over the initial fraction of that impact area across all proposed projects:

cost proportionality	projects proportionality	popularity proportionality
${\displaystyle {\mathit{\text{P}}}_{o,c,l}^{i,f}=\frac{S_{o,c,l}^{i,f}}{r_{c,l}}}$	${\displaystyle {\mathit{\text{P}}}_{o,n,l}^{i,f}=\frac{S_{o,n,l}^{i,f}}{r_{n,l}}}$	${\displaystyle {\mathit{\text{P}}}_{o,v,l}^{i,f}=\frac{S_{o,v,l}^{i,f}}{r_{v,l}}}$

Impact metrics calculated at the ballot level.: In this case, share, representation and proportionality metrics across different impact areas are computed for each voter to understand how voters’ choice relate to the voting outcomes, and how impact gain or loss aligns at the individual voters’ ballots and outcome level. Popularity is not accounted at the individual ballot level, because popularity is formulated from the collective set of individual ballots.

Share: For an aggregation method $f$, share at the ballot level is calculated as the fraction of the total cost or number of projects in the winning outcome that belongs to a given impact area $l$ which is also approved by voter $v$:

cost share	projects share
${\displaystyle S_{b,c,l}^{i,f}=\frac{\sum _{p=1}^{|W_{l,v,f}|}{c}_p}{\sum _{p=1}^{|W_f|}{c}_p}}$	${\displaystyle S_{b,n,l}^{i,f}=\frac{|W_{l,v,f}|}{|W_f|}}$

Representation: For an aggregation method $f$, representation at the ballot level is calculated as the fraction of the total cost or number of projects in a given impact area, $l$, that belongs to the winning set which is also approved by the voter $v$:

cost representation	projects representation
${\displaystyle R_{b,c,l}^{i,f}=\frac{\sum _{p=1}^{|W_{l,v,f}|}{c}_p}{\sum _{p=1}^{|P_l|}{c}_p}}$	${\displaystyle R_{b,n,l}^{i,f}=\frac{|W_{l,v,f}|}{|P_l|}}$

Proportionality: For an aggregation method $f$, proportionality at the ballot level is calculated as the ratio of winning shares for an impact area $l$ which is also approved by voter $v$ over the initial fraction of that impact area across all proposed projects in terms of cost and number of projects:

cost proportionality	projects proportionality
${\displaystyle P_{b,c,l}^{i,f}=\frac{S_{b,c,l}^{i,f}}{r_{c,l}}}$	${\displaystyle P_{b,n,l}^{i,f}=\frac{S_{b,n,l}^{i,f}}{r_{n,l}}}$

Within-novelty metrics calculated at the voting outcome level: The novelty $\omega$ within an impact area $l$ for an aggregation method $f$ is calculated as the fraction of cost, number or popularity of projects that are exclusively winning within the impact area $l$ with respect to the cost, number or popularity of winning projects in that impact area:

cost within novelty	projects within novelty	popularity within novelty
${\displaystyle S_{o,c,l}^{\omega ,f}=\frac{\sum _{p=1}^{|{\hat{W}}_{l,f}|}{c}_p}{\sum _{p=1}^{|W_{l,f}|}{c}_p}}$	${\displaystyle S_{o,n,l}^{\omega ,f}=\frac{|{\hat{W}}_{l,f}|}{|W_{l,f}|}}$	${\displaystyle S_{o,v,l}^{\omega ,f}=\frac{\sum _{p=1}^{|{\hat{W}}_{l,f}|}{v}_p}{\sum _{p=1}^{|W_{l,f}|}{v}_p}}$

Within-novelty metrics calculated at the voters’ ballot level: The novelty $\omega$ within an impact area $l$ for an aggregation method $f$ at the ballot level is calculated as the fraction of cost or number of projects that are exclusively winning within the impact area $l$, which is also approved by the voter $v$ with respect to the cost and number of all winning projects in that impact area:

cost within novelty	projects within novelty
${\displaystyle S_{b,c,l}^{\omega ,f}=\frac{\sum _{p=1}^{|{\hat{W}}_{l,v,f}|}{c}_p}{\sum _{p=1}^{|W_{l,f}|}{c}_p}}$	${\displaystyle S_{b,n,l}^{\omega ,f}=\frac{|{\hat{W}}_{l,v,f}|}{|W_{l,f}|}}$

Between-novelty metrics calculated at the voting outcome level: The novelty between-impact-areas $\beta$ for an aggregation method $f$ is calculated as the fraction of cost, number or popularity of exclusively winning projects for an impact area $l$ with respect to the cost, number or popularity of all exclusively winning projects:

cost between novelty	projects between novelty	popularity between novelty
${\displaystyle S_{o,c,l}^{\beta ,f}=\frac{\sum _{p=1}^{|{\hat{W}}_{l,f}|}{c}_p}{\sum _{p=1}^{|{\hat{W}}_f|}{c}_p}}$	${\displaystyle S_{o,n,l}^{\beta ,f}=\frac{|{\hat{W}}_{l,f}|}{|{\hat{W}}_f|}}$	${\displaystyle S_{o,v,l}^{\beta ,f}=\frac{\sum _{p=1}^{|{\hat{W}}_{l,f}|}{v}_p}{\sum _{p=1}^{|{\hat{W}}_f|}{v}_p}}$

Between-novelty metrics calculated at the voters’ ballot level: The between-impact-areas novelty $\beta$ for an aggregation method $f$ at the ballot level is calculated as the fraction of cost or number of projects that are exclusively winning for an impact area $l$, which is also approved by a voter $v$ with respect to the cost or number of all exclusively winning projects:

cost between novelty	projects between novelty
${\displaystyle S_{b,c,l}^{\beta ,f}=\frac{\sum _{p=1}^{|{\hat{W}}_{l,v,f}|}{c}_p}{\sum _{p=1}^{|{\hat{W}}_f|}{c}_p}}$	${\displaystyle S_{b,n,l}^{\beta ,f}=\frac{|{\hat{W}}_{l,v,f}|}{|{\hat{W}}_f|}}$

(c). Threats to validity

The 345 real-world participatory budgeting instances are retrieved from the Pabulib repository and cover voting scenarios at city, district and municipal level. These instances feature a broad spectrum of projects, from 5 to 220. The number of voters spans from 200 to 40 000, while the available budget expands in the range of 10–900 K in PLN (Polish Zlotych). This diversity provides strong empirical foundation to scale up and generalize the insights derived from the impact and novelty analysis.

To account for the different nature of the voting instances, the measurements illustrated in the figures of this article show the distribution over all voting instances, rather than only the mean values. Moreover, each proposed project in a given election instance is independently labelled based on the quartile it belongs within that particular instance irrespective of the cost of other proposed projects in different election instances. This normalization allows us to make comparisons among different voting instances that may come with costs of different level.

For the participatory budgeting instances of (i) City Idea, Aarau and (ii) Green Million, Wieliczka, the impact area labels for the proposed projects were absent. As such, the impact areas labels for the proposed projects in these instances were assigned independently by multiple individuals to cross-validate the classification.

Ethics

This work did not require ethical approval from a human subject or animal welfare committee.

Data accessibility

Relevant dataset and source code used for the analysis used in this paper is made available at [42].

Supplementary material is available online [43].

Declaration of AI use

We have not used AI-assisted technologies in creating this article.

Authors’ contributions

S.M.: data curation, formal analysis, investigation, methodology, visualization, writing—original draft; S.M.: conceptualization, formal analysis, investigation, resources, validation, writing—review and editing; E.P.: conceptualization, methodology, project administration, supervision, validation, writing—review and editing.

All authors gave final approval for publication and agreed to be held accountable for the work performed therein.

Conflict of interest declaration

We declare we have no competing interests.

Funding

This work is funded by a UKRI Future Leaders Fellowship (MR/W009560/1): Digitally Assisted Collective Governance of Smart City Commons–ARTIO. All data used for this study are in the public domain.