The effectiveness of forecast‐based humanitarian assistance in anticipation of extreme winters: a case study of vulnerable herders in Mongolia

Abstract

This paper presents evidence relating to a forecast‐based cash and non‐food item distribution among vulnerable herder households during the 2017–18 dzud (extreme winter) season in Mongolia, and analyses the results of a quasi‐experimental study evaluating its impacts. An innovative approach in disaster risk reduction, forecast‐based financing (FbF) can have short‐ and long‐term benefits to vulnerable households but remains understudied. The paper contributes information on a multimodal FbF programme offering one‐off cash grants and in‐kind veterinary kits. The data found significant effects of reduced mortality and increased offspring survival in some types of livestock, and that the timing of FbF assistance is crucial, as reported early assistance correlated to positive outcomes in terms of reduced animal mortality. These findings can be used to design more effective FbF interventions, to understand better the appropriateness of FbF designs, and to use early warnings and early actions to help people prepare and withstand disasters such as dzuds.

Introduction

As the effects of global climate change become ever clearer, including more devastating droughts and floods and more intense storms and heatwaves (IPCC, ²⁰¹⁹), disaster risk reduction (DRR) interventions to assist those confronting such events continue to be refined. Most interventions centre on either post‐disaster response and reconstruction or long‐term DRR; very few are short‐term initiatives undertaken in the often narrow window of time between the issuance of evidence‐based early warnings and the climate‐related happening itself (Kellett, Caravani, and Pichon, 2013; Costella et al., ²⁰¹⁷). If targeted appropriately, actions at this time can potentially lead to a reduction in vulnerability before a disaster (Suarez and Tall, 2010). Some can lead to increased preparation for disaster response and can yield a large return despite the short‐term nature of the measure itself (Coughlan de Perez et al., 2015).

One of the newest approaches within DRR is forecast‐based financing (FbF). Such mechanisms frequently comprise a tripartite process entailing matching forecasts with actions, disbursing money from an established preparedness fund once a forecast is issued, and developing standard operating procedures by the organisation(s) involved (Coughlan de Perez et al., 2015). The number of FbF programmes is rapidly rising, with more than 50 countries currently setting up anticipatory action systems. This boosts both the value and the necessity of sharing good practices and lessons learnt to heighten the effectiveness of FbF (IFRC, ²⁰¹⁸, p. 68).

This paper examines a specific FbF activation in the case of Mongolia, which experiences extreme winters known as dzuds, posing a major threat to pastoralist livelihoods. A dzud comprises extreme winter conditions preceded by a hot and dry summer, which diminishes the availability of pasture and in turn, can augment herd mortality rates. Once herds go below given thresholds, they can become non‐viable, leading to what some researchers call a ‘poverty trap’ (Barrett and Carter, 2013), and illustrating the importance of mechanisms that support herd retention for resilience. Often as an effect of the reduction of herd size due to dzuds, herders may give up their pastoral livelihoods entirely and undertake destitution migration to urban areas (Field et al., 2012, p. 502), presenting challenges for development. There have been indications in recent decades of an increase in the frequency of dzuds (Fernandez‐Gimenez, Batkhishig, and Batbuyan, 2012), making the development of effective DRR tools to address this phenomenon ever more urgent.

It is highly desirable to integrate forecast‐based action mechanisms into institutionalised social protection programmes, yet this work is still in its early stages (Costella et al., 2018) and is not currently occurring in the context of Mongolia. The country does, however, have several significant social protection programmes, including the universal Child Money Programme and allowances for the elderly and those with disabilities. The Government of Mongolia, together with the World Bank, has instituted an index‐based livestock insurance programme that, since 2006, increasing numbers of herders have been able to use (Ahmed, 2020). Furthermore, there is a social security system that is extended to herders and other self‐employed or informal workers on a voluntary basis. In reality, though, significant numbers of herders and informal and self‐employed workers lack sufficient income security because of weak administration and delivery services (ILO, ²⁰¹⁶). Additional support is needed, particularly in the face of extreme weather.

This paper analyses the results of a quasi‐experimental study to evaluate the FbF intervention in Mongolia. The second section provides some context and an overview, and the third section presents the methodology employed. The fourth section sets out the findings, including on the impact of the action, as well as in relation to variation in the perceived timing of when assistance was received. The fifth section discusses lessons learnt and proposes some next steps for research and practice.

Context and overview

Livestock is the main livelihood in Mongolia and makes up more than 80 per cent of the gross value of agricultural production (Sutti, Reynolds, and Batello, 2005). The most common livestock in Mongolia are Bactrian camels, cattle, goats, horses, sheep, and yaks. Traditional livestock are all well adapted to the harsh climate; they can regain condition and build up fat reserves rapidly during the short growing season. Camels, cashmere goats, and yaks develop winter down among their coats, which helps to minimise heat loss. In normal years, all can survive outdoors throughout the winter with little to no shelter or supplementary feed. When climate conditions are average, herders manage risks to animals using their knowledge and skills, keeping livestock mortality low (less than 4.5 per cent of herds).

However, dzuds can present a significant threat to livestock, which is the only source of income for more than 36 per cent of Mongolian households (Sutti, Reynolds, and Batello, 2005). Vulnerable herders have insufficient access to animal feed and animal care supplies in years when there is increased stress on animals, owing to a dzud, for instance. Access to necessary supplies is limited due to a lack of availability or inflated prices in the market. This can lead to increased livestock mortality, which can have huge ramifications for herders' lives and livelihoods (Lehmann‐Uschner and Kraehnert, 2018).

In 2017, 80 per cent of Mongolia had experienced drought‐like conditions (FAO, ²⁰¹⁸, p. 34), and wheat production was more than 40 per cent less than it had been in the previous five years (FAO, ²⁰¹⁷, p. 23). In anticipation of a dzud, and the associated challenges posed, the Mongolian Red Cross Society (MRCS) established a FbF mechanism in 2018 to help herder households cope. The anticipated impacts of a dzud are based on forecast information and a risk map. Fourteen indicators establish the dzud threat level, including changes in rainfall, the risk of drought, and the regional temperature (IFRC, ²⁰¹⁸, p. 65).

The National Agency for Meteorology and Environmental Monitoring (NAMEM) released the dzud risk map on 26 November 2017. This seasonal impact‐based forecast revealed that 50 per cent of the country's territory was at high risk and 30 per cent was at medium risk, especially central and western areas of Mongolia. In response, the FbF system was activated, initiating a combination of unrestricted cash grants and animal care kits, distributed to herder households. This programme was implemented by the MRCS, with the financial support of the British Red Cross and technical advice by the Red Cross Red Crescent Climate Centre.

The MRCS targeted the 40 most‐at‐risk soums (districts) in 12 provinces; large parts of their territory registered the highest level of risk on the map. Herders with 50–200 livestock who fulfilled one of the following criteria were considered to be eligible for the intervention:

• families with multiple children (more than five under the age of 16);

• families with a disabled member;

• elderly families aged 60 or more with no guardian;

• single‐headed households with three or more children under the age of 16; or

• herders with up to 400 livestock who live in high dzud risk areas identified by Local Emergency Management Agencies (LEMAs).

The intervention assisted 2,000 herder households, ² supplying unrestricted cash grants of MNT 240,000 (approximately USD 100) in December 2017, sent to bank accounts, and animal care kits in January 2018. Pastoralists in Mongolia have widespread access to banks, and those who receive benefits are used to getting them via a bank transfer. The veterinary kits included fish oil, hoof ointment, and mineral blocks, supplies that do not require veterinary training to administer and are well‐known and used by herders. The items were recommended by livestock specialists within the Food and Agriculture Organization (FAO) of the United Nations and the Civil Service of Mongolia. These ‘actions’ were identified as ‘essential’ to decrease the effect of a dzud, and could be administered in the time between forecast and potential impact. ³ The intention was to provide these essential resources before winter conditions became most extreme and animals were weak. It was not clear whether veterinary kits would be available in the local market, which strengthened the decision to offer them on an in‐kind basis.

The combination of cash and in‐kind assistance is gaining in popularity as a useful form of multimodal programming. It enables perceived ‘essentials’ to be supplied directly while still offering beneficiaries freedom and choice and expanding their purchasing power (The World Bank Group, 2016, pp. 15–16; WFP, ²⁰²⁰). Although not appropriate in all settings and at all times (Bailey and Harvey, 2015), evidence demonstrates that unconditional cash transfers can enable men and women in rural areas to accumulate productive assets (Asfaw et al., 2014; Barca et al., ²⁰¹⁵; Natali et al., ²⁰¹⁶). Qualitative research conducted by the FAO in Ghana, Kenya, and Zambia found that cash transfers allowed rural men and women to enlarge their existing small trade businesses and improve their income base (Barca et al., 2015), enabling them to become more resilient to disasters. While some of the assets accumulated as an effect of transfers can be at risk during disasters, climate adaptation and DRR strategies can aid planning on how to manage and protect them in such circumstances. Indeed, when cash transfers are made as an anticipatory or preventative social protection step in the face of a disaster, the advantages may be enhanced in terms of adequacy, predictability, scalability, and timeliness (Costella et al., 2017).

At the same time, evidence demonstrates that the regularity and timing of cash transfers plays an important role in effectiveness (Bastagli et al., 2016, p. 32), and that one‐off cash transfers may not be as effective in creating resilience over time (Ulrichs, Slater, and Costella, 2019, p. 377). However, the provision of emergency cash transfers as an anticipatory initiative is very different to the provision of recurrent social protection cash payments in a development context (Costella et al., 2018). It is to this newer area of research and practice that this paper aims to contribute.

One‐off cash grants were offered in Mongolia for many of the positive reasons cited above, and under the assumption that many beneficiaries would use their funds to buy hay for their livestock—it was deemed to be too logistically complicated to distribute hay to remote areas and a cash intervention avoided a perceived risk of corruption. Grants of MNT 240,000, equivalent to approximately USD 100 at the time, and to the monthly minimum wage in Mongolia, were provided. The amount was chosen based on a joint agreement among international agencies working in Mongolia in an effort to streamline cash interventions. The dzud of 2017 was the third harsh winter in a row in Mongolia, meaning that international organisations had gained experience from earlier interventions. Multiple agencies had offered different amounts of cash to their respective beneficiaries during the previous two winters, creating a perceived disparity among recipients and generating confusion and frustration. Picking the average minimum wage as the amount for a cash transfer was a way to offer assistance without risking disruptions to the local market, and a means of helping as many people as possible within a limited budget.

The next section outlines the methodology employed by the study to test the impact of this FbF intervention of unconditional cash and in‐kind assistance.

Methodology

To assess the effects of providing forecast‐based cash on the livelihoods and well‐being of herder households, the MRCS developed a quasi‐experimental study design to collect representative sample survey data from FbF beneficiaries and comparison households that were equally vulnerable and exposed to the same extreme winter conditions. ⁴ It allowed the researchers to compare the effects of FbF on the experiences and socioeconomic well‐being of beneficiary households with the counterfactual of comparable households that did not receive FbF assistance. The same eligibility criteria were used as for the FbF intervention to obtain a random sample of 223 beneficiaries (out of 2,000 reached by FbF support) and 223 comparison households. Experimental and quasi‐experimental research designs have been widely used to assess the impacts of cash transfers (Bastagli et al., 2019) and other development interventions. ⁵

The intervention targeted the most vulnerable households. Similar to a regression discontinuity design, the comparison households (non‐FBF‐assisted households) were just above the eligibility threshold. The means comparison of key sample characteristics (see supplementary material relating to Table 2 in the Appendix) shows that they were slightly less vulnerable, a difference accounted for here through the use of Propensity Score Matching (PSM) and a bias‐corrected matching estimator. The PSM method ensures that only the most comparable households from both groups are retained in the sample, thereby reducing bias and increasing the accuracy of analysis of the effects of FbF assistance, or lack thereof. The PSM procedure effectively reduced the sample differences between the groups except in the proportion of households with a disabled family member, where the proportion of FbF‐assisted households remained double that of non‐assisted families.

To test the robustness of the findings, the average treatment effect (ATE) was also estimated using a nearest‐neighbour, bias‐corrected matching estimator (Abadie et al., 2004; Abadie and Imbens, 2006, 2011). The matching estimator applies a vector of distances to the covariates between an FbF‐assisted household and each non‐assisted potential match to impute a counterfactual outcome for each sample household across the intervention and comparison groups. Avoiding a decrease in sample size as introduced by the PSM, we draw on the strengths of this approach for relatively small sample studies and match with replacement so that each household can be used in more than one match.

Lastly, to measure the levels of stress experienced by the participants, the survey respondents were asked a series of questions about their feelings and thoughts during the one month preceding the survey. This adhered to the standardised Perceived Stress Scale (PSS) (Cohen, Kamarck, and Mermelstein, 1983), an internationally and widely used self‐assessment gauge of personal stress.

Findings

Survey data were collected from 446 herder households across four provinces and 10 soums in May 2018. The following subsections detail findings pertaining to important areas of the intervention, namely the identification and assistance of vulnerable households, household actions and socioeconomic impacts, loans and money, and livestock impacts.

Identification and assistance of most vulnerable households

Four key findings reveal that the intervention group appears to be slightly more vulnerable than the comparison group based on several socioeconomic indicators, including family size and per capita livestock (for more information, see supplementary material relating to Table 2 in the Appendix). First, FbF‐assisted households (intervention group) were more likely to have a larger family, with a greater number of children between the ages of 5 and 16, and a significantly higher number of family members with a disability than in non‐FBF‐assisted households (control group). Second, the mean herd size, in sheep head units (SHU), ⁶ while insignificantly larger in the intervention group, is significantly smaller when calculated in proportion to family size (per capita SHU). Third, while a higher proportion of FbF‐assisted households reported additional non‐herding income, this is mainly due to a greater number of intervention households receiving child and disability social protection benefits. Fourth, the average monthly income from non‐herding sources was significantly lower for FbF‐assisted households (approximately USD 59 equivalent), by almost one‐quarter as compared to non‐assisted households (approximately USD 75 equivalent). This difference is mostly accounted for by a substantially higher pension income in the comparison group.

Household actions and socioeconomic impacts

The majority of herders in both groups (85 per cent each) indicated that they knew in advance that the winter conditions of 2017–18 would be extreme and that a dzud would occur. Approximately one‐half of the respondents in both groups cited their own experience as a source of early warning knowledge, as well as television or radio broadcasts, whereas one‐quarter of households were informed by their friends or neighbours and by government administrative units (aimag (province) or soum). The Red Cross did not feature prominently as a source of early warning, with only around one per cent of households having received dzud‐related information from the MRCS.

Given their early knowledge, most households in both groups (96 per cent in FbF‐assisted and 92 per cent in comparison) implemented early actions to prevent the impacts of dzud hazards. As anticipated, virtually all households bought hay or animal feed; the absence of FbF cash assistance does not appear to have affected the comparison group's ability to buy animal fodder in preparation for the dzud. The differences in the average prices of hay and animal treatment are not statistically significant.

The only highly significant difference in the early actions taken by the two groups is the sale of livestock: 43 per cent of FbF beneficiaries indicated selling some livestock, as compared to only 25 per cent of households in the comparison group. All said that they had to sell these animals to cope with the ramifications of the dzud. However, while a larger proportion of beneficiary households sold some animals, the number of animals sold was significantly smaller (just one‐half) than the amount of livestock sold by comparison households. The destocking of animals can also be regarded as a positive early step to avoid livestock deaths later in the season, but the prices that they can fetch can be very low and thus may be a necessary but not necessarily advantageous precautionary strategy of herders.

FbF cash spending

Within the intervention group the overwhelming majority of beneficiaries (91.9 per cent) spent at least part of their FbF cash on buying hay or feed for their livestock (see Figure 1). One‐third of respondents used the FbF cash to purchase food for household consumption and one‐fifth met ‘other’ expenses, a category mainly composed of buying fuel/gasoline. If herders used the fuel to move to better pastures or to less dzud‐affected areas, this may indicate positive coping, but this could not be confirmed by the study. The median amount used for buying hay or feed was MNT 240,000 (approximately USD 100) or 100 per cent of FbF assistance, making this by far the most important expense category—the average amounts spent are shown in Figure 2. The animal care kits lasted for a median 56 days per beneficiary household, or almost two months, and were given to about 60 per cent of the livestock (in SHUs) in beneficiary herder households.

Figure 1.

Use of FbF cash by spending category (multiple mentions possible)

Figure 2.

Average FbF cash amount spent per category (thousands of MNT, USD equivalent shown in parentheses)

Source: authors.

FbF assistance does not appear to have influenced the extent to which households were able to afford basic necessities during the dzud period. Approximately one‐third of both groups said that they were not able to afford some necessary items; for most, this related to more animal hay or fodder, food for the household, health expenses, transport, and other items. There were also no discernible effects of FbF cash on the food consumption of beneficiary households.

It is notable that no herder family had to abandon completely their herding life and move to an urban area owing to the severity of the dzud, and only around two per cent of respondents indicated that at least one family member who normally tends to animals had to give up herding and relocate to the city.

Psychosocial stress

A longstanding body of research demonstrates the negative psychological effects of disasters triggered by natural hazards, including post‐traumatic stress disorder (Neria, Nandi, and Galea, 2008) and depression (Bell et al., 2017). One aim of the intervention was to reduce such impacts through anticipatory support. In addition, post‐traumatic stress can be a proxy for the magnitude of the shock that a household experiences, with high stress correlating with high impact; these data may thus serve as an extra indicator of whether the results were as hypothesised. The FbF intervention does not appear to have made a significant difference to helping beneficiary households cope with the dzud‐associated stresses. Households in both groups demonstrated moderate stress levels on average, with the FbF‐assisted group having a statistically significant but only slightly higher total stress score (19.1 out of 40) than the comparison group (17.6).

Loans and money

A slightly higher proportion of FbF beneficiary households (43 per cent) took out new loans between December 2017 and when the survey was conducted in May 2018 as compared to non‐FbF‐assisted herder families (37 per cent), although this difference is not statistically significant. About two‐thirds of respondents in both groups said that they had obtained loans specifically because of the dzud (see supplementary material relating to Table 1 in the Appendix). Practically all loans came from banks at low rates of interest (reported to be less than two per cent).

One should note that the median loan sum in both groups (MNT 2–2.5 million, approximately USD 1,000) was around 10 times the amount of FbF cash assistance (MNT 240,000, approximately USD 100). Ninety‐eight per cent of loan amounts were well above the FbF cash level, with most in the MNT 1.5–3 million bracket (approximately USD 700–1,500). FbF assistance does not appear to have influenced the extent to which households took out or paid back their loans, perhaps because of the comparatively low amount provided. Between 86 and 93 per cent of respondents across both groups had at least some portion of their 2017–18 dzud season loans still outstanding at the time of the survey, admittedly within a very short time period, with an average of around 90 per cent of the loan sum yet to be repaid.

As illustrated in Figure 3, buying hay or feed was the highest spending category in both groups and attracted by far the most expenditure. The large ‘other’ category includes a diverse range of answers, such as buying fuel and firewood, repairing cars or houses, building fences, or purchasing electricity generators.

Figure 3.

Proportion of respondents who spent loan money, by spending category; FbF‐assisted and comparison households (multiple mentions possible; no significant differences)

Source: authors.

Most loans were acquired relatively early in the dzud season: the median borrowing date for FbF beneficiaries was 22 January 2018, two days after the median date of having received FbF cash. In the comparison group, the median borrowing date was 15 February 2018, three weeks later than the beneficiary group.

Livestock impacts

Households in both survey groups witnessed a deterioration in the body conditions of 46 per cent of their animals because of the dzud of 2017–18, with all species similarly affected. FbF assistance appears to have had a significant bearing on helping beneficiary households to reduce the mortality of their horses during the period by about 50 per cent as compared to the households that did not receive FbF cash or care kits (see Figure 4). The differences for other animal species are not significant. This outcome is particularly important, though, because horses are the household's most valuable animal in terms of SHU (7:1 sheep to horse).

Figure 4.

Animal mortality rates by species, comparison of FbF‐assisted and comparison households

FbF assistance also had a significant positive effect on the offspring survival rates in the herds of beneficiary households (see Figure 5). The survival rate of goats was increased by 11 per cent in the FbF‐assisted group, and the survival of young sheep rose by 7 per cent as compared to the group that did not receive FbF cash or animal care kits. Given the importance of livestock to the beneficiary households, these decreased mortality odds can be huge in terms of positive impact, particularly given that keeping animals alive can have long‐ and short‐term positive effects. ⁷

Figure 5.

Offspring survival rates by species, FbF and comparison households

Note: asterisks indicate significant differences; see supplementary material relating to Figures 4 and 5 in the Appendix for significance levels.

Source: authors.

Column A of Table 1 shows that the effect of FbF assistance on reducing horse mortality rates by approximately five percentage points (see Figure 4) is robust and statistically significant, even when controlling for a range of covariates, including household demographics, geographic location, and dzud severity (models A, B and C) (see Table 1 note for details). Columns B and C show that the positive effect of FbF assistance on the survival rates of goat and sheep offspring respectively is robust and statistically significant in its direction and magnitude.

Table 1.

The effect of receiving FbF assistance on livestock mortality and offspring survival rates during the dzud of 2017–18 (bias‐corrected matching estimator; significant coefficients in bold)

	Outcome variables
	a. Mortality rate: horses	b. Offspring survival rate: goats	c. Offspring survival rate: sheep
A. Minimal set of covariates FbF assistance (cash + animal care kits)	‐0.043 ∗∗ (0.021)	0.071 ∗∗ (0.030)	0.048 ∗ (0.026)
B. Core set of covariates FbF assistance (cash + animal care kits)	‐0.039 ∗ (0.021)	0.067 ∗∗ (0.031)	0.049 ∗ (0.027)
C. Maximal set of covariates FbF assistance (cash + animal care kits)	‐0.037 ∗ (0.021)	0.070 ∗∗ (0.027)	0.048 ∗ (0.026)
N	338	382	398

Notes:

^∗p<0.1;

^∗∗p<0.05;

^∗∗∗p<0.01; see supplementary information relating to Table 1 in the Appendix.

Source: authors.

Table 1 provides the estimated coefficients for the average treatment effect (ATE). The minimal set of covariates (A) includes the household size, total number of livestock, soum‐level livestock mortality as a proxy for the severity of dzud conditions, and aimag‐level fixed effects. The core set of covariates (B) adds the distance of the household to the soum centre, the educational attainment of the household head, and total non‐herding income, including government transfers. The maximal covariates set (C) also includes the age of the household head and the number of persons living with a disability in the household, as well as minimum monthly average temperatures (degrees Celsius) and maximum monthly total precipitation (millimetres) at the soum level as crude indications of dzud severity. Unfortunately, more granular data on dzud severity was not available for the FbF intervention and survey sample area.

The ATE was estimated instead of the Average Treatment Effect on the Treated (ATT), which should be identical in an ideal randomisation scenario, since the intervention targeted highly vulnerable households. The sample of survey respondents was randomly drawn from the soum population, which is highly similar in terms of vulnerability status, as the sample means comparison shows. The treatment effect is expected, therefore, to be similarly applicable to all sample households.

Effects of (reported) early versus late assistance

Interestingly, while all of the respondents in the FbF‐assisted group indicated that they received cash and care kits, the time periods in which the households reported having received the assistance vary widely. For cash, the dates fluctuate between December 2017 and as late as March 2018, and for animal care kits between December 2017 and April 2018. This stands in contrast to all records and testimonies of those in charge of the distribution, who state that the distribution occurred in the intended window of December 2017 for unrestricted cash grants of USD 100 and January 2018 for animal care kits. As is discussed later in the paper, however, this discordance merits further research.

To assess whether the survey respondents' perception of ‘early’ versus ‘late’ assistance is associated with differences in effectiveness, the study analysed animal mortality and offspring survival rates depending on whether the beneficiaries reported having received FbF cash and care kits before or after 31 January 2018. This may seem to be a somewhat arbitrary cut‐off point, but Mongolian livestock census data ⁸ show that animal mortality typically rises sharply after the first quarter of the year when the compounded effects of a preceding dry grazing season and extreme winter take their toll on already weakened livestock.

Figure 6 shows that the beneficiary‐reported timing of FbF assistance has a sizeable, statistically significant effect on the mortality rates of all livestock species except cattle. The effect is most pronounced for horses: mortality rates increased almost fivefold if assistance was obtained late. Figure 6 demonstrates that the significant difference in horse mortality between the FbF‐assisted and the comparison group, as indicated above, is accounted for almost entirely by those households that reported getting early assistance, or what they perceived to have been early.

Figure 6.

Animal mortality rates by species; comparison by whether FbF cash and animal care kits were received before (early) or after (late) 31 January 2018

Notes: asterisks indicate significant differences; see supplementary material relating to Figure 6 in the Appendix for p‐values.

Source: authors.

To test the robustness of the positive effect of reported early assistance, and conversely the negative effect of reported late assistance, bias‐corrected matching estimators were calculated for the average treatment effect of beneficiaries reporting to have received FbF cash and animal care kits before 31 January 2018, controlling for other potentially influential covariates in models A, B and C (see table notes for details). Columns A and B of Table 2 confirm respectively the positive effect of reported early assistance in reducing horse mortality rates and improving goat offspring survival rates, although column C reveals that the FbF impact cannot be corroborated for sheep offspring.

Table 2.

The effect of receiving early FbF assistance (on or before 31 January 2018) on livestock mortality and offspring survival rates during the dzud of 2017–18; intervention group only (bias‐corrected matching estimator; significant coefficients in bold)

	Outcome variables
	a. Mortality rate: horses	b. Offspring survival rate: goats	c. Offspring survival rate: sheep
A. Minimal set of covariates Early FbF assistance (cash + animal care kits)	‐0.079 ∗∗∗ (0.030)	0.068 (0.052)	‐0.003 (0.052)
B. Core set of covariates Early FbF assistance (cash + animal care kits)	‐0.094 ∗∗∗ (0.029)	0.080 ∗ (0.047)	0.003 (0.045)
C. Maximal set of covariates Early FbF assistance (cash + animal care kits)	‐0.107 ∗∗∗ (0.030)	0.118 ∗∗ (0.050)	‐0.039 (0.047)
N	176	195	200

Notes:

^∗p<0.1;

^∗∗p<0.05;

^∗∗∗p<0.01; see supplementary information relating to Table 2 in the Appendix for model specifications.

Source: authors.

Discussion

This paper has examined the impact of a FbF intervention with herder households in Mongolia led by the MRCS in 2017. The key aims were to reduce livestock mortality in herds belonging to vulnerable herder households and to reduce the psychosocial stress of households. Although the overall results are mixed, the level of success in primary areas points towards the value of particular components of this intervention and the ability of FbF to affect positively vulnerable populations experiencing disasters such as a dzud. As such, this case study contributes further evidence to exploration of FbF as a valuable form of disaster risk prevention and reduction.

The findings demonstrate that the FbF intervention in Mongolia succeeded in identifying the most vulnerable households (see the ‘context and overview’ section above for the criteria), and that the survey was relatively successful in drawing together a comparable sample of similar herder families that did not receive FbF assistance. The aid appears to have had a strong, statistically significant effect on helping beneficiaries to diminish the mortality of horses, among their most valuable animals (in SHUs), by approximately 50 per cent as compared to households that were not part of the FbF intervention. The survey data also reveal a significant positive effect of FbF assistance on the survival rates of goat and sheep offspring, thereby helping herders to secure the future of their livelihoods.

Results such as these may mean that, in the short term after a dzud, herders are able to maintain their livelihoods without significant disruption. In the long term, interventions such as the FbF examined here can help to buffer herder livelihoods at scale during high‐risk years of covariate shock. This may lessen the level of destitution migration to the city and reduce the number of herders falling below the poverty line. Institutionalising FbF so that warnings of extreme climate events trigger anticipatory action may interrupt trajectories of consecutive major losses in herd numbers, increasing, in turn, the resilience of herder households overall.

The findings appear to be robust, based on good quality data from a representative sample of beneficiary and comparison households, despite the slightly greater vulnerability of the FbF‐assisted households. ⁹

FbF cash

The data do not show significant effects of FbF assistance on the ability of households to afford basic necessities during the dzud period, as well as on their overall food intake or experience of psychosocial stress. This may be due in part to the comparatively small amount of cash provided, which does not appear to have been large enough to cover all necessities. As USD 100 was the amount agreed by multiple agencies working in the country, there is a need to rethink the logic driving how cash levels are determined, as the cash component of the intervention had weak outcomes (see Figure 1). Although evidence concerning cash transfers demonstrates that it can be an effective and appropriate response, as noted earlier, a key caveat is that this depends on the setting, the needs of beneficiaries, and the specific benefits under review (Bailey and Harvey, 2015). In the case of Mongolia, it appears that more research on the specific herder context should have been conducted when determining the amount of cash to supply. Despite being the average monthly minimum wage in the country at the time, USD 100 was possibly too low to achieve the desired impact. Partially because of this finding, uses of FbF cash are examined in more depth below and further recommendations made.

The overwhelming majority of beneficiaries used the largest share of their FbF cash to buy additional hay or feed for their livestock. This may demonstrate the disproportionate need that preserving livestock represents. Hence, it may be worth exploring whether a bulk procurement and distribution of hay and feed may be more cost‐efficient for similar FbF interventions in the future, and for beneficiaries accessing these goods. However, cost‐efficiency may not necessarily outweigh the benefits of cash fungibility in the eyes of beneficiaries.

The FbF intervention does not appear to have made a significant difference in helping beneficiary households to cope with the stresses of the dzud experience. There are three principal explanations for this, although none that could be confirmed by the data. First, as discussed elsewhere, it is likely that the cash provided was simply not large enough to reduce notably financial constraints and thus did not have a large effect on diminishing related stress. As mentioned, this was the third dzud experienced in a row, and it was not as immediately devastating as previous ones had been, although the cumulative impacts of multiple bad winters can affect resilience capacities. The winter of 2017, though, may have been comparatively less stressful for many households than the ones prior. Second, very high levels of stress manifest commonly when herders are at the point of needing to sell off their animals and undertake destitution migration to cities; however, the households included in this intervention were not at this point of vulnerability. ¹⁰ Consequently, while the dzud was difficult, it does not appear to have pushed participants into extreme and highly stressful decision‐making regarding their livelihoods. Third, owing to the length of time between the event and the evaluation, there may be some level of recall bias.

Timing

A key point of reflection and learning vis‐à‐vis FbF interventions more broadly is derived from the widely varying dates on which FbF beneficiaries indicated to have received FbF cash or animal care kits. The findings suggest that the timing of FbF assistance is crucial. The difference in the reported timings centres on preventative assistance offered in the face of disaster versus ‘coping’ assistance provided during one. That reported early assistance had a significantly more positive effect reinforces the logic of FbF interventions, which are driven by the understanding that early warnings and actions can influence people's ability to prepare and withstand disasters such as dzuds.

Potential explanations for this observed variance include: the management of the distribution process, including the time it took beneficiaries to collect their assistance from a distribution point; confusion among beneficiaries about what constitutes FbF cash and FbF animal care kits and who provided assistance (although there does not appear to be a statistical relationship between the two); different cash withdrawal dates for recipients, as money was transferred to bank accounts; and challenges in recalling the dates on which assistance was received—the survey was carried out in May 2018, whereas according to project records, the distributions took place between December 2017 and January 2018. Subjective perceptions of FbF assistance having been received early or late may also point to the ideal time for this intervention, since the information may show when beneficiaries made use of their FbF assistance; this cannot be corroborated, however, without further research. ¹¹

Regardless, the results suggest the importance of (i) verifying distribution schedules and logistics, (ii) validating the conceptual understanding of what beneficiaries consider to be intervention assistance, and (iii) qualitatively validating with beneficiaries at what point in time FbF assistance may be subjectively seen as early or late. This, in turn, represents an opportunity to identify the ‘ideal’ time according to beneficiaries either to receive or utilise assistance; knowledge that can shape interventions in the future.

Conclusion

Extreme weather and climate change are directly related (IPCC, ²⁰¹⁹). Studies that scrutinise interventions to prevent and mitigate vulnerability in the face of events such as dzuds are thus increasingly necessary, not only for the specific context assessed, but also to amass lessons for wider application.

FbF, which was viewed until recently as an innovative instrument, is growing in importance as a tool to prevent and reduce extreme weather‐related disasters (Gros et al., 2019). Although a combination of cash and in‐kind assistance was the preferred intervention in the case of Mongolia, multiple other early actions can be selected based on effectiveness (Costella et al., 2017, p. 35). The results of this study point to several considerations for the design of future FbF systems, including:

• evaluating the timing of early actions—earlier support might be more beneficial;

• assessing the size of cash grants in comparison to other often‐accessed sources of funding, such as loans; and

• conducting qualitative interviews to shed light on the complexity of agricultural interventions, such as the impacts on different types of livestock.

Furthermore, implementation capacities and procedures are very important and require careful planning. While the implementation of the intervention was positive overall, the fact that it was the first time this mechanism was tested may have affected outcomes, such as the differing perceptions of assistance timing. Most new FbF projects include, therefore, a ‘simulation’ exercise as part of the system setup so that procedures and implementation arrangements are tested before actual activation.

Mongolia is representative of multiple middle‐income countries that due to their relative poverty and geographical location are vulnerable to extreme weather events, yet countries in the industrialised North are also not exempt from such phenomena, making this an important issue for all (Kraehnert and Kemfert, 2018). As illustrated in this case study of a Red Cross intervention in the face of a Mongolian dzud, FbF can have positive short‐ and long‐term effects on vulnerable households and as such, can make an important contribution to disaster preparedness and risk prevention and mitigation.

Appendix. Online supplementary material

Supplementary material relating to Table 1. Borrowing behaviour, loan amounts and payback rates; means comparison of FbF‐assisted and comparison households (no significant differences)

	FbF	Comparison	p‐value
Taken out any new loans since December 2017	43.4	37.0	0.23
Loans taken specifically because of dzud	72.0	68.8	0.68
Loan amount (median, thousands MNT)	2,500	2,000	0.14
Loan not yet fully paid back	93.3	85.9	0.25
Proportion of loan outstanding at the time of the survey	88.0	90.2	0.56
Has dzud affected your ability to pay back the loan?
Forced me to delay loan payback	70.0	63.6	0 .13
Caused me to default on my loan	12. 9	16.4	0.59
N	75	60	–

Note:

^∗p<0.1;

^∗∗p<0.05;

^∗∗∗p<0.01 (p‐values adjusted for multiple comparisons using Holm's correction within categories; medians tested using the Brown–Mood median test).

Source: authors.

Supplementary material relating to Table 2. Comparison of sample characteristics (statistically significant differences in bold)

Variables	a. Before PSM			b. After PSM
	FbF	Comparison	p‐value	FbF	Comparison	p‐value
Province
Khentii (%)	20.2	19.7	1.00	20.2	19.1	1.00
Tuv (%)	30.0	30.5	1.00	3 0 .1	31.2	1.00
Uvs (%)	29.6	30.0	1.00	30.6	28.3	1.00
Zavkhan (%)	20.2	19.7	1.00	19 .1	21.4	1.00
Household demographics
Age of household head	45.5	46.4	0.54	46.2	44.8	0.40
Woman‐headed household (%)	23.8	21.1	1.00	23.7	20.8	0.61
Number of household members	3.95	3.24	0.00 ∗∗∗	3.5	3.4	0.60
Any children under 5 (%)	43.0	33.6	0.17	39.3	37.6	0.83
Any children 5–16 (%)	50.7	33.2	0.00 ∗∗∗	41.0	37.0	0.53
Any elderly (over 65) (%)	15.2	22.0	0.18	18.5	18.5	1.00
Any household members with a disability (%)	21.5	11.7	0.04 ∗∗	22.0	11.6	0.01 ∗∗
Distance of livestock winter shelter from soum centre (median kilometres)	25.0	27.5	0.64	27	25	0.52
Herd size
Sheep head units (SHU)	292	277	0.42	274	276	0.93
Per capita SHU	90	110	0.02 ∗∗	96	106	0.43
Households with small herd size (<= 200 SHU) (%)	36.8	36.3	1.00	41.6	38.7	1.00
Households with medium herd size (201–400 SHU) (%)	42.6	48	0.67	41.0	44.5	1.00
Households with large herd size (> 400 SHU) (%)	20.6	15.7	0.67	17.3	16.8	1.00
Herding experience (median years)	23	23	1.00	22	22	1.00
Non‐herding household income
Households with any non‐herding income (%)	85.7	78.5	0.06 ∗	83.8	76.9	0.14
Total non‐herding income (thousands MNT)	155.2	197.7	0.03 ∗∗	172.6	187.8	0.51
Non‐FbF assistance received to cope with dzud impacts: government (soum/aimag)	26.9	17.5	0.16	23.1	20.8	0.70
Duration of extreme winter conditions in household area (weeks)	19.2	19.9	0.22	19.1	19.5	0.57
N	223	223	–	173	173	–

Source: authors.

Supplementary material relating to Figures 4 and 5. Animal body conditions, mortality and new‐born survival rates; FbF and comparison households (significant differences in bold)

	FbF	Comparison	p‐value
Share of animals with deteriorated body conditions because of dzud (% of total SHUs)	46.4	46.5	0.99
Animal mortality rates
Goat	0.17	0.19	0.74
Sheep	0.13	0.13	0.74
Cattle	0.16	0.18	0.53
Camel	–	–	–
Horse	0.07	0.12	0.04 ∗∗
Total (SHUs)	0 .14	0.15	0.53
New‐born survival rates
Goat	0.78	0.70	0.02 ∗∗
Sheep	0.82	0.76	0.07 ∗
Cattle	0.93	0.89	0.25
Camel	–	–	–
Horse	0.90	0.86	0.47
Total (SHUs)	0.81	0.77	0.18
N	173	173	–

Notes:

^∗p<0.1;

^∗∗p<0.05;

^∗∗∗p<0.01.

Source: authors.

Supplementary material relating to Figure 6. Effects of receiving FbF cash and animal care kits before (early) or after (late) 31 January 2018 on animal mortality and survival (significant differences in bold)

	Early	Late	p‐value
Goat mortality rate	0.133	0.184	0.06 ∗
Sheep mortality rate	0.089	0.144	0.02 ∗∗
Cattle mortality rate	0 .191	0.148	0.39
Horse mortality rate	0.017	0.083	0.00 ∗∗∗
Total (SHUs)	0.102	0.147	0.01 ∗∗
Goat offspring survival rate	0.816	0.763	0.27
Sheep offspring survival rate	0.834	0.811	0.59
N	42	131	–

Note:

^∗p<0.1;

^∗∗p<0.05;

^∗∗∗p<0.01 (p‐values adjusted for multiple comparisons using Holm's correction within categories; medians tested using the Brown–Mood median test).

Source: authors.

Acknowledgements

The authors would like to thank the MRCS under the leadership of Madame Bolormaa Nordov, Secretary General, whose staff and volunteers implemented the FbF intervention, supported the data collection underlying this study—especially Davaajargal Batdorj and Nyamkhuu Chuluunkhuu—and engage tirelessly in relief and recovery efforts to assist vulnerable populations in Mongolia. In addition, thanks are extended to lecturers at the Department of Management, School of Economics and Business, Mongolian University of Life Sciences for their involvement in data gathering, as well as to the respondents who shared their experiences. The authors gratefully acknowledge, too, the British Red Cross for establishing and supporting the FbF project in Mongolia and for supporting this research financially. Moreover, Heather Fehr and Michelle Yeo at the British Red Cross provided helpful comments during development of the initial analysis. The authors also very much appreciate the insightful remarks and suggestions made on an earlier draft of this paper by Dr Kati Krähnert and members of her research seminar at the Potsdam Institute for Climate Impact Research.

Data availability statement

The data that support the findings of this study are available on request from the corresponding author. The data are not publicly available due to privacy or ethical restrictions.