Building safety in humanitarian programmes that support post‐disaster shelter self‐recovery: an evidence review

Abstract

The humanitarian sector is increasingly aware of the role that good quality evidence plays in the underpinning of effective and accountable practice. This review addresses the need for reliable evidence by evaluating current knowledge about the intersection of two key outcome targets of post‐disaster shelter response: supporting shelter self‐recovery and building back safer. Evidence about post‐disaster shelter programmes that aim to improve hazard resistance while supporting shelter self‐recovery has been systematically analysed and evaluated. Technical support, especially training in safer construction techniques, was found to be a central programme feature, but the impact of this and other programme attributes on building safety was largely not ascertainable. Programme reports and studies lack sufficient detail, especially on the hazard resistance of repaired houses. Accounts of shelter programmes need to include more reliable reporting of key activities and assessment of outcomes, in order to contribute to the growing evidence base in this field.

Introduction

For humanitarian action to be effective and fully accountable, good quality evidence on what works, and how, is essential. As well as being crucial for successful decision‐making on humanitarian programming, it is increasingly necessary to demonstrate programme impact and value for money, as well as for securing future funding (Hofmann et al., 2004; Roberts and Hofmann, 2004; Ashdown, 2011; Gerdin et al., 2014). The demand for greater accountability in this sector has led to a surge in research studies that seek to analyse the evidence for the impact and effectiveness of humanitarian interventions.

The methodological approach that is often employed in these studies, and which is at the heart of evidence‐based policymaking (Anderson and Shemilt, 2010; Blanchet et al., 2017), is the systematic review. This involves the comprehensive collection, assessment, and synthesis of all of the available evidence on a specific topic. The methodology originated in the fields of health and medicine (Bero and Drummond, 1995), before being widely adopted for the evaluation of interventions in social science domains such as education and social care (Davies, Nutley, and Smith, 2000). Most recently, systematic reviews have begun to be utilised for research on international development (Snilstveit, 2012; Cameron, Mishra, and Brown, 2016) and humanitarian aid (Bangpan et al., 2017; Blanchet et al., 2017; Juillard et al., 2017). The common goal of these reviews has been to make available high‐quality, reliable information that can support evidence‐based decision‐making, and ultimately enhance the effectiveness of humanitarian interventions.

So far, the quality of the evidence available to these studies has been remarkably limited (Hofmann et al., 2004; Duffield et al., 2005; Warren et al., 2015; Blanchet et al., 2017). Apart from health and nutrition, the majority of areas in the humanitarian sector suffer from a paucity of evidence, especially of studies that show a causal relationship between assistance provided and outcome targets; shelter has been identified as an area in particular need of robust evidence (Clarke et al., 2014). Accordingly, there have been calls for better evidence on what works in post‐disaster shelter support. Although the volume of literature on this subject is increasing, much of it is still composed of practitioner experience and opinion pieces, or project evaluation reports that provide little evidence about programme outcomes or impact (Knox Clarke and Darcy, 2014; Maynard, Parker, and Twigg, 2017).

This review aims to address the need for better evidence in the humanitarian shelter field by examining the current state of knowledge at the intersection of two important areas of post‐disaster shelter programming:

• supporting shelter self‐recovery; and

• building back safer.

Both have been identified as key outcome targets of post‐disaster shelter response in recent years. In so doing, the review intends to contribute to the burgeoning evidence base for humanitarian shelter assistance.

Post‐disaster shelter programmes: supporting self‐recovery and building back safer

Between 1994 and 2013, more than 116 million homes were damaged or destroyed by disasters triggered by natural hazards (CRED, 2015). The frequency of climate‐related disasters in particular (droughts, floods, and storms) has been rising over recent decades, and this trend is expected to continue (Schilderman and Lyons, 2011; UNISDR, 2012; Schilderman, 2014a). The number of people at risk of losing their home, or being injured or killed as a result of unsafe construction, will continue to rise unless disaster risk reduction measures in vulnerable communities drastically improve (Davis, 2016).

The process of repairing and rebuilding homes is a priority after any disaster, as it is essential for ensuring the health, safety, and livelihood restoration of affected communities (Duyne Barenstein, 2006; Peacock, Dash, and Zhang, 2007; The Sphere Project, 2011). Shelter programmes delivered by governmental agencies and humanitarian organisations may be initiated quickly, but they are able to address only a relatively small proportion of the shelter needs of a disaster‐affected population (Parrack, Flinn, and Passey, 2014). The majority of affected communities will act on their own, often beginning within days of a disaster to address their immediate shelter needs where they can. They will frequently construct makeshift temporary shelters or begin to repair or rebuild their homes using whatever materials are available to them, with no assistance or support from any external organisation (Flinn, 2015; Parrack, Flinn, and Passey, 2014).

Post‐disaster humanitarian shelter: supporting shelter self‐recovery

Historically, post‐disaster shelter support provided by humanitarian agencies was most likely to comprise the building of complete shelters for the most vulnerable beneficiaries. More recently, the shelter sector has moved away from the provision of complete dwellings towards the provision of assistance that supports beneficiaries’ own shelter self‐recovery (Schilderman, 2004; Davidson et al., 2007; Twigg et al., 2017). Supporting households to repair and rebuild their own homes enables shelter programmes to reach a significantly greater percentage of a disaster‐affected population as per‐dwelling costs are considerably lower (Flinn, Schofield, and Morel, 2017). This approach is also much more likely to lead to sustainable settlements, develop skills and create livelihoods, foster self‐empowerment, and contribute to the maintenance of local architectural traditions and cultural identity (Alexander, 1989; Lyons, 2009; Davis and Alexander, 2016; Hendriks et al., 2017).

Although an element of assisted self‐help has been a feature of humanitarian shelter response for some time, it is only recently that shelter programmes have begun regularly to state explicitly that supporting self‐recovery is an aim. The shelter response to Typhoon Haiyan in Southeast Asia in 2013 (Shelter Cluster Philippines, 2014) was the first major humanitarian shelter initiative to identify the provision of support for shelter self‐recovery as a strategic objective. It has been a strategic objective in other humanitarian post‐disaster shelter responses since then (Maynard, Parker, and Twigg, 2017), and the approach was given global institutional backing by the Global Shelter Cluster (2018) as part of their 2018–2022 Strategy (Global Shelter Cluster, 2018).

What is ‘self‐recovery’ in post‐disaster shelter?

The term ‘self‐recovery’ has only been part of the common vocabulary in post‐disaster shelter literature for a relatively short time. It has been used to encompass a broad spectrum of activities and processes, and its meaning often is ill‐defined (Twigg et al., 2017). At one end of the spectrum, ‘self‐recovery’ is used to refer to the unassisted self‐repair and self‐reconstruction which householders engage in without any external support (Parrack, Flinn, and Passey, 2014). In the context of humanitarian shelter programmes, it commonly refers to programmes that provide financial, material, or technical assistance (Maynard, Parker, and Twigg, 2017), where reconstruction and repair may be carried out by affected individuals themselves, or where households may engage the local informal building sector to carry out the work.

A key factor in understanding the concept of shelter self‐recovery is the degree to which disaster‐affected individuals are actors and decision‐makers in the many different shelter activities and processes that take place. Where there is no external support, householders are in charge of all shelter‐related decisions and actions, and the description of the shelter process as ‘self‐recovery’ is unambiguous. For the beneficiaries of a shelter programme, their degree of ownership and agency will lie at some point on a continuum, from very little involvement in decisions and choices to complete control over the construction process. However, even though humanitarian shelter programmes may be described as supporting ‘self‐recovery’ or use similar terms signifying beneficiary ownership of the shelter process, these labels may not be an accurate reflection of beneficiary involvement. An ‘owner‐driven’ programme, for example, implies that the householder has control over building design and construction, with the donor providing financial and technical support alone (Aubrey, 2008). In reality, beneficiaries of owner‐driven recovery programmes may have limited opportunities for choice and may not be in charge of many aspects of the initiative (Davidson et al., 2007; Powell, 2011; Maly, 2018). Management of the construction process does not guarantee full empowerment, as beneficiaries may still be obliged to accept options presented to them. A shelter programme that supports self‐building by providing cash, for example, may also specify designs to which beneficiaries must conform to receive support. Labels such as ‘self‐recovery’ or ‘owner‐driven’ cannot be relied upon on their own to determine whether or not a shelter programme has truly supported homeowners to recover themselves.

Defining shelter self‐recovery

As the term ‘self‐recovery’ has previously been poorly defined and used inconsistently in the humanitarian shelter literature, for the purposes of this study we developed a definition based on a key premise: that humanitarian shelter programmes support self‐recovery if beneficiaries have the capacity to make meaningful choices about the construction of their homes (Duyne Barenstein, 2006). In this review, shelter programmes were deemed to have supported self‐recovery when beneficiaries were active decision‐makers in how their homes were rebuilt or repaired, were able to make key decisions about the layout, the materials, and the construction details and techniques employed, and were in charge of the process, either building the shelter themselves or procuring local labour to do so.

Improving the building of safety in post‐disaster shelter

Aiming to ‘build back better’—to enhance the economic, physical, and social resilience of a community—has become an essential feature of post‐disaster shelter strategy (Blaikie et al., 2004; Clinton, 2006; Paul, 2011; Schilderman and Lyons, 2011; Mannakkara, Wilkinson, and Potangaroa, 2014). Making use of the window of opportunity opened during the recovery phase to achieve this aim is widely advocated by academics, policymakers, and practitioners alike; this period can be utilised to ensure that conditions of vulnerability are not recreated during the building process and that safer structures are built, ones that will be more able to withstand the shock of recurring hazardous events (Schilderman, 2010; Ong et al., 2016; Twigg et al., 2017).

Support from agencies delivering shelter programmes during the recovery period is more likely to lead to the repair and reconstruction of homes with increased hazard resistance, compared with the homes of households that rely on their own resources (da Silva and Batchelor, 2010; Duyne Barenstein and Iyengar, 2010). Contractor built houses are more likely to incorporate structural features in the building design, such as strengthened connections between the roof and the walls, strong foundations, or beams that tie the walls together, which are intended to make the building less vulnerable to future hazards. The degree of control that agencies exercise over construction design and implementation has been found to be correlated with the frequency of implementation of safer building measures—those built entirely by agencies most consistently include safety features (Shelter Cluster Philippines, 2016).

Although self‐construction may be the most viable option for many disaster‐affected households, it is also likely to be the most hazardous. When affected communities build back themselves, it is common for construction processes to include the same inadequate building practices as before, and for the repaired or rebuilt homes to leave householders at risk from future disasters (Coburn and Spence, 2002; Green, 2008; Parrack, Flinn, and Passey, 2014). The likelihood of householders incorporating safer construction methods in the rebuilding of their homes can be affected by many factors, including a lack of understanding of safer building methods (Yahya et al., 2001; Schilderman, 2004; Powell, 2011; Maynard and Barritt, 2015), the prohibitive expense of including additional hazard‐resistant construction techniques (Tran Tuan et al., 2014), the financial and technical support available, or the required level of compliance with local building codes (Twigg et al., 2017).

To enhance the likelihood that self‐built homes will be structurally safer than they were prior to a disaster, shelter programmes supporting self‐recovery must consider how to encourage the implementation of safer building methods (Flinn, Schofield, and Morel, 2017). Training in safer building techniques is known to have a significant impact on whether or not a reconstructed house is more resilient (Jha et al., 2010), as is providing access to financial and technical assistance (Twigg et al., 2017). Although shelter programmes supporting self‐recovery are likely to incorporate this understanding into programme design by including elements that attempt to address these specific issues, whether and how they are effective in their aim of improving building safety is less certain.

Improving the building of safety in supported shelter self‐recovery: evaluating the evidence

Humanitarian interventions that support shelter are still relatively under‐researched, and programmes that support shelter self‐recovery even less so (Peacock, Dash, and Zhang, 2007; Hendriks et al., 2017). Evidence about the impact of shelter self‐recovery support on householders’ knowledge of safer building techniques has been described as ‘unclear’ (Maynard, Parker, and Twigg, 2017). To the knowledge of the authors, there has been no analysis to date of the impact of shelter programmes supporting self‐recovery on building safety, even though it is now frequently a key programme outcome target. This review aims to evaluate the evidence about shelter programmes that promote safer self‐recovery and to identify important information gaps that currently exist.

Method

Much of the literature on humanitarian shelter programmes is published by humanitarian organisations, rather than in academic peer‐reviewed journals. Academic and humanitarian sources were searched separately using slightly different approaches.

Terms for ‘self‐recovery’

To ensure that the literature search was comprehensive, a set of alternative terms for ‘self‐recovery’ was developed systematically. An initial list of alternative terms, such as ‘owner‐driven’, ‘self‐built’, and ‘self‐repair’, was drawn up, based on the review team's current knowledge of the field. Additional terms were searched for by examining articles and reports in three key humanitarian shelter resources about humanitarian shelter programmes that typically support self‐recovery: the Humanitarian Library; the Global Shelter Cluster website; and the Shelter Case Studies repository on the Shelter Projects webpage (see Annexe 1 for a list of search terms).¹

Following Maynard, Parker, and Twigg (2017), this search included shelter programmes that provided:

• material assistance (such as construction resources, reuse of debris, salvaging, and tools);

• financial assistance (cash or vouchers); and

• technical assistance (such as construction monitoring, the provision of guidelines/mass communications, and training).

Academic literature search

EBSCO, Engineering Village, ProQuest, PubMed, and Web of Science were identified as containing articles in key subject areas.² The databases were searched for articles published between January 1970 and January 2018, and which contained terms for ‘disaster’, ‘self‐recovery’, and ‘shelter’ (see Annexe 1 for a full list of the search terms and the strategy).

Identification of articles for inclusion in the review

The titles and abstracts of all articles were screened by two members of the review team. Full texts were also scrutinised by two members of the review team. As terms for self‐recovery are often used to describe a range of beneficiary experiences in the shelter reconstruction process, the criteria that had to be met for an article to be included in the review were very specific, as follows:

• Written in English.

• Published from 1970 onwards.

• Describe programmes supporting post‐disaster shelter reconstruction, repair or rebuilding of homes that were damaged or destroyed by a specific disaster. Householders must be fulfilling their immediate shelter needs, and not be part of a longer‐term development programme.

• Disaster is defined as a ‘natural disaster’.

• Describe shelter repair or reconstruction activities that members of the disaster‐affected population took responsibility for themselves. This excludes processes where, for example, beneficiaries provided manual labour to rebuild homes under the instruction of other persons leading the project; it includes projects that provided assistance or guidance, but where beneficiaries were active decision‐makers in how their homes were rebuilt or repaired and were in charge of the process, either building the shelter themselves or procuring local labour to do so.

• Humanitarian shelter programmes are only included if beneficiaries have been able to make key decisions about layout, materials, construction details and construction techniques used. Programmes that required beneficiaries to choose a specified design, even where there were two or more designs to choose between, have been excluded. Programmes that recommend hazard‐resistant design, where beneficiaries were free to build according to this design or not, as they chose, are included. Programmes that required inclusion of certain safety features in the construction, usually as part of a cash‐based programme, have been included.

• Include discussion of aspects of safety (or lack of) in the construction process, where safety is defined as the degree to which the shelter structure is likely to withstand future hazards or shocks.

Humanitarian shelter literature search

To keep the humanitarian literature search manageable within the scope of the project, the review team identified 21 key humanitarian literature sources likely to contain information on post‐disaster shelter.

• Active Learning Network for Accountability and Performance in Humanitarian Action (ALNAP) (publications only);

• British Red Cross;

• Build Change;

• CARE International UK;

• Global Communities;

• Global Shelter Cluster;

• Habitat for Humanity;

• Institute of Migration;

• Integrated Regional Information Networks (IRIN);

• InterAction;

• International Federation of Red Cross and Red Crescent Societies (IFRC);

• National Society for Earthquake Technology – Nepal (NSET);

• Norwegian Refugee Council;

• Oxfam;

• Reach Initiative;

• ReliefWeb;

• Save the Children;

• Shelter Case Studies;

• United Nations Human Settlements Programme (UN‐Habitat);

• United Nations Office for Disaster Risk Reduction (UNDRR) (formerly the United Nations International Strategy for Disaster Reduction (UNISDR)) (PreventionWeb); and

• World Habitat (formerly the Building and Social Housing Foundation).

The search facilities on the websites of organisations vary hugely. Two complementary approaches were used to maximise the likelihood of retrieving all relevant documents:

• The website of each source was searched using only the set of terms for ‘self‐recovery’, via the website's general search facility.

• Dedicated ‘resources’ or ‘publications’ sections were searched, using subcategories such as ‘shelter’, where filters were available, or using ‘shelter’ as a search term. Alternative appropriate filters, such as ‘disaster risk reduction’ and ‘self‐help’, were used to search publications on shelter specialist websites.

The review team also investigated using advanced search facilities on Google, which allow one to search for specific terms on individual websites, and limit the search to particular document types, such as portable document format (PDF). While it is possible to use Boolean operands to some extent, the limitations of Google's search facility meant that this was not an efficient way to find relevant ‘grey’ literature, so this method was not incorporated in the literature search.

Identification of articles for inclusion in the review

Articles and reports for inclusion in the review were identified using the same criteria as for the academic literature search.

Quality

It is common practice to construct a measure of the quality of studies in an evidence review, with the aims of establishing the reliability of the information in each study, and how relevant the findings are to the research question. The quality rating is often used as a criterion for inclusion in the review, by limiting the review to studies that are of good enough quality, and therefore reliability. In the field of post‐disaster shelter safety, key areas for consideration are likely to include the overall level of reporting, the quality of the shelter programme, the appropriateness of the study design, the generalisability of findings, and how well outcomes are measured.

During the process of identifying articles for inclusion in this review, it became clear that the quality of reporting varied enormously and was often poor (as discussed below). Details of the reconstruction process, the shelter programme, and/or the safety of the construction were frequently sparse, and quantifiable measurement of safety was rare. The review team concluded that attempting to assign quality ratings to the documents would be of limited value. Instead, all of the documents that meet the other inclusion criteria are included in the review and contribute to an analysis of the quality of the evidence that is currently available in this field.

Studies included in the review

After deduplication and rejection of irrelevant items, the initial database search identified 1,233 academic articles. Of these, 62 passed the abstract screening stage, and full screening yielded four articles and book chapters that met the criteria for inclusion. Searching the selected humanitarian literature resources identified 1,776 reports and articles, of which 120 passed the first screening stage and 17 were assessed as meeting the inclusion criteria. Two of these were reports by one agency on the same shelter programme and were considered as one record for the purposes of analysing programme information, yielding a total of 16 shelter programme case studies or reviews.

Excluded articles

The majority of articles that were fully screened did not discuss safety in relation to any part of the building process or outcomes, and so were excluded. Articles were also excluded when beneficiaries did not have ownership of key decision‐making processes, such as where a programme described as ‘community‐led self‐help’ was in fact a donor‐led participatory programme in which beneficiaries provided labour. Other reports made general statements about community involvement, or described recovery as ‘owner‐driven’, but provided insufficient detail about the shelter reconstruction process for it to be possible to determine the precise role of beneficiaries. A lack of information about beneficiaries’ involvement in the recovery process was the most frequent reason for exclusion.

Characteristics of included reports

In total, 20 reports and articles met the criteria for inclusion (see Annexe 2 for full details of all documents that appear). The type of disasters most regularly causing shelter damage and destruction were typhoons/hurricanes/cyclones (n=9) and earthquakes (n=7), followed by floods (n=4), disasters most commonly occurring in the Philippines (n=7), Pakistan (n=4), and Indonesia (n=3). Of the four academic studies, three described case studies of shelter projects, and one was an evaluation of beneficiary experiences of a shelter response. Eleven documents were brief shelter programme case reports, and five were detailed agency shelter programme evaluation reports. All documents reported to some degree on the safety of reconstructed houses, but only four included an assessment of the hazard resistance of finished buildings.

Type of assistance offered to support safer self‐recovery

The amount of detail on the structure of the 20 shelter programmes and how they were delivered varied considerably. Some programmes were described in a few paragraphs as part of a short case report, others over many pages in a comprehensive programme evaluation. Technical assistance and cash or vouchers were the types of support most commonly offered (see Table 1). Key elements of each programme designed to encourage safer building practices have been identified as far as possible from the information available (see Table 2).

Table 1.

Support provided by shelter programmes

Type of support provided by shelter programme	Number of programmes offering each type of support (N=20)
Cash and vouchers	16
Technical assistance	16
Materials	7
Tools	3

Source: authors.

Table 2.

Key features of programmes aiming to improve safety

Feature	Number of programmes including feature (N=20)
Technical assistance	16
Training in safer construction techniques	16
Monitoring of construction process	13
Written guidelines/information on safer construction techniques	11
Householder/contractor participation in construction of demonstration building	5
Cash conditional on compliance with safer building guidance	10
Improved hazard resistance of local building techniques	6

Source: authors.

Technical assistance

Expert technical advice on safer construction was often provided through training sessions and during monitoring of the construction process. Programmes that provided cash conditional on compliance with guidance on safer building techniques offered monitoring and technical advice throughout the building process, to enable beneficiaries to meet the programme requirements. One programme report described householders who had begun reconstruction prior to the initiation of the shelter programme who had not incorporated hazard‐resistant techniques. The programme gave expert technical advice on remedial measures that could be taken to ensure that modified or non‐compliant houses could be made compliant [3].³

Training in safer construction techniques

Most of the programme descriptions (16 of 20) included an element of training in safer construction techniques. Construction training was regarded as a crucial route to delivering a programme's aim of improving safety, and participation in initial training sessions was often necessary for beneficiaries to receive further material or cash support. Amongst those programmes that provided training in safer construction, around one‐third (5 of 16) offered the opportunity for beneficiaries and local builders to participate in the construction of a prototype house. This approach was widely viewed as a successful means to build skills and transfer knowledge.

The demonstration of the model house was highly appreciated as the communities learnt a new skill and also became aware of DRR [disaster risk reduction] features that could strengthen their house. 86.9% respondents had participated in the demonstration of the model house by the project and 96% of them found it useful as they learnt about construction methods and [DRR] features and with this knowledge, they could help others to rebuild their house [9].

Six programmes explicitly described how the supporting organisation or its partners had developed the hazard resistance of construction materials and techniques that had traditionally been used by local communities. Of these six programmes, three conducted research into the vernacular construction of buildings that had survived a disaster.

The main aim was to record the different types of structures that survived, the techniques and practices that largely withstood the flood waters and the ones that led to house collapse. Once the best construction methods were identified and improved they were compiled into a construction manual used for practical and theoretical capacity building trainings for affected households [12].

A common feature of programmes that developed the hazard resistance of local construction techniques is that they had the resources to do this, especially the necessary time commitment.

One of the problems ITDG faced in later work was that development agencies were often in a hurry to reconstruct. In no other project was the ITDG team given the six months it had spent in the Alto Mayo to get the technology and methodology right. Haste does not allow for as much participation and training as would be wanted [1].

Communicating advice about safer building practices: other modes of delivery

Programmes that provided written advice and information on safer building practices most frequently used posters showing construction typologies and techniques that could make houses more hazard‐resistant, as well as instruction manuals. One programme produced a music video and jingles to promote build back safer messages [18]. Reporting on the use of these materials was limited, for example:

These behavior change communication methods reinforced knowledge/skills that resulted in the adoption of the ‘build back safer’ techniques [17].

Many field visits were made to ensure that the messages were being disseminated to communities and used in the construction [10].

Several programme reports stated that training or messages about safer construction were provided, but they gave no further information about their content or how they were delivered. Notably absent from reports about written advice and information was any assessment of how well beneficiaries had understood this information, or any effective evaluation of the impact that it had on subsequent construction.

Barriers to improving building safety

Monitoring construction was a key feature of the majority of programmes (13 of 20), but this was not always as effective as intended. The most significant issues reported were limitations in the capacity of programme staff to carry out programme activities, usually due to insufficient expertise, time, staff numbers, or a combination of all three. A reluctance among major donors to fund training and technical assistance was identified as a contributing factor in the failure of the management team to deliver training and technical assistance in line with the pace of reconstruction [3]. For instance:

Not all of the implementing partners had the shelter experience or the staff capacity to cope with the project requirements [10].

The 120 field monitors and community volunteers had only a few days technical training. It was not realistic to expect them to check the construction quality of 3,400 unique houses [7].

Enabling householders to own the process of shelter recovery had a negative impact on the efficacy of the support provided in some instances. The capacity of some organisations to monitor construction and provide technical advice was stretched by the freedom of choice exercised by householders. As two of the reviewed documents note:

Completed homes were likely to be ‘safer’ than the construction practices that have become prevalent over the past 30 years but cannot be described as earthquake or hazard resistant. The freedom which was a strength also lead to a wide variation in quality and divergence from design principles [7].

Because households were free to choose the construction materials they wanted, giving out disaster risk reduction advice to each household was difficult [10].

Knowledge about safer construction techniques did not guarantee their implementation. One evaluation [3] reported that local artisans who had learnt about seismic‐resistant technologies ultimately had little authority to make decisions about construction styles, as they largely rested in the hands of the homeowners who did not always prioritise building safety.

Another programme [20] that provided materials and training had anticipated that government cash assistance would complement the shelter initiative; when this assistance was delayed, households who were unable to build by themselves did not have the resources required to hire skilled labour or to purchase additional materials. Four months after the distribution of materials, only 50 per cent of the beneficiaries of this programme had used the materials received for repairs.

The cost of materials affected the influence of safer building advice given as part of another programme [19] that provided shelter materials, cash grants, and technical training. The beneficiaries of this programme incorporated more disaster risk reduction measures in their roofs than in their walls or foundations. This appeared to be the combined result of the differential cost of materials, the practical challenges of modifying existing foundations, and the training that had prioritised measures to strengthen roofs.

One programme review [18] reported on the impact that a lack of training and build back safer information can have. In the absence of these resources, beneficiaries used the distributed shelter kits to constructed unsafe shelters.

End of programme and follow‐up evaluations

End of programme and follow‐up evaluations that assessed building safety were described in just four documents (4 of 20). In addition, two case studies included some limited beneficiary‐reported information about the types of safer construction methods that they had used; all others (14 of 20) did not report any assessment of building safety at the end of the programme.

Three programmes carried out a follow‐up technical assessment of building safety [9, 18, 16]; two of these reported quantitative data [9, 18]. The most robust programme evaluation included a three‐month follow‐up that assessed the percentage of houses that had included specific safety features, such as posts anchored firmly into the floor, cross‐bracing, quality of roof fixings, and how well beneficiaries thought their houses would withstand hazardous conditions. Between 89 and 100 per cent of beneficiaries included three of the six safety features examined, all concerning secure fixings; 20, 46, and 62 per cent, respectively, incorporated wall cross‐bracing, appropriate roof projection, and adequate roof pitch [9].

The other quantitative assessment of safety at one‐year follow‐up identified aspects of the programme that had had a negative impact on building safety. This programme had provided beneficiaries with cash and materials in the absence of safer construction training or information dissemination. As the document notes [18]:

After logistic delays materials were distributed without IEC [information, education, and communication] materials or full training of builders in build back safer technology. … A September 2015, shelter repair assistance report on Post Distribution Monitoring on Building Quality highlighted serious shortcoming in improving resilience of shelters with 94% of roofs assessed as weak or very weak due to the lack of knowledge in build back safer by carpenters. In addition, 80% of walls still needed bracing and 80% of columns required treatment. Similar problems existed in other structural components

The third follow‐up technical assessment [16] was reported in less detailed terms, describing ‘generally a good uptake of the main build‐back‐safer measures … although in some cases this was not very well done’, as measured at 6 and 18 months. Self‐reported levels of understanding of safer construction measures, and how well beneficiaries thought they had applied them in the construction of their houses, were also measured. Quality of foundations was reportedly difficult to assess as almost all buildings had been completed. The quality of implementation of some build‐back‐safer measures was found to be poor in some houses, and suggested causes for specific construction weakness included a lack of resources and understanding of construction principles, and varying levels of technical support. The document points out that:

It was very noticeable that areas where the roving teams offered more support, or where community cooperation was more widely used, achieved much higher and more consistent construction standards. In the worst areas the application of build‐back‐safer measures was patchy and there were flaws in the basic structural arrangement of houses. In the best cases, the quality of construction was exceptionally good and the build‐back‐safer measures were fully applied [16].

The fourth document [1] described an indirect assessment of building safety due to a second earthquake in the region during the reconstruction process, demonstrating the hazard‐resistance of the new technology: all 70 houses that had already been built using the new technology withstood the event. Although no additional details are given, this report indicated that the spread of the safer building technology to other parts of the region was assessed sometime after the programme had finished:

An external evaluation the next year found that quincha ⁴ had become a mainstream technology in the Alto Mayo, accounting for 30% of the housing stock compared with 7% nationwide [1].

Two case studies described the frequency with which beneficiaries had included any of the key build back safer principles that the training had focused on [19, 20]. This information was gathered via beneficiary interviews; neither programme reported a direct assessment of the hazard‐resistance of the buildings after repair.

Unassisted safer self‐recovery

Four reports described anecdotal observations that safer construction techniques communicated to beneficiaries as part of a shelter programme may have been copied and implemented by non‐beneficiaries [11, 12, 14, 17]. Two documents reported that non‐beneficiaries replicated safer construction techniques used in the project [11, 12]; one project enabled free access to safer construction training [12]. One report said that carpenters and masons trained by the project had begun to advocate for the use of safer construction methods [11], and another stated that beneficiaries who learned new masonry techniques were employed by non‐beneficiary families to build their houses [14]. The fourth document reported that safer construction methods were implemented by households that had not received assistance thanks to improved understanding of build back safer measures within the wider community [16]. However, reporting on this issue was brief in all cases, and it was not possible to determine how the transfer of learning had taken place when training had not been provided, or to assess how well safer building techniques were implemented by non‐beneficiaries.

Summary of the evidence

The evidence available for this review was generally of poor quality and any findings need to be interpreted with caution. The main findings are summarised below in relation to technical support, adaptation of local construction techniques, and knowledge transfer.

Technical support

Training householders and artisans in safer construction techniques is a key component of shelter programmes that aim to support self‐recovery, but the impact of training is likely to be influenced substantially by how effectively construction is monitored. Providing technical support requires expertise, personnel, sufficient funds, and time. The capacity of the delivering organisation to provide technical assistance is crucial, particularly when householders are able to exercise freedom of choice in the range of materials and techniques used, as supporting this approach may be more resource‐intensive than other types of shelter programmes.

Adaptation of local construction techniques

Adapting local construction techniques to increase their hazard resistance was a common feature of a number of programmes. This approach can support affected householders with safer construction methods that are accessible and can be replicated easily. Communities are more likely to adopt new technologies if end‐users find them easy to modify, increasing the likelihood that such methods will continue to be used and developed (van Leersum and Arora, 2011). Effective adaptation of local methods requires a significant time commitment, which may be problematic due to reconstruction timetables or donor financing restrictions.

Knowledge transfer

Anecdotal evidence reinforces the theory that knowledge about safer construction techniques can transfer to non‐beneficiary households, either via free access to training, exchange of information between communities, or employment of trained artisans. Evidence about this process—how it takes place, what affects it, and its impact—is lacking.

Discussion

It is not uncommon for systematic reviews to conclude that insufficient evidence exists to answer the question in hand (Petticrew, 2003; Blanchet et al., 2017). Nevertheless, even when this is the case, a systematic review can help to identify knowledge gaps and consider why they may exist, and as a result it can provide guidance on how resources may be best used in order to develop the evidence base further.

The clearest finding of this review concerns the quality of the evidence itself; despite the fact that supporting safer self‐recovery is now a principal target of shelter programming, the evidence on the efficacy of these activities and their impact on safer building is limited and generally of poor quality. The most significant aspects of reporting that affected the quality of the available evidence are discussed below, as well as possible reasons for these evidence gaps and suggestions for how they may be addressed.

Quality of reporting

The principal methodological challenge of this review was to determine whether or not a shelter reconstruction process described in the literature could be defined as ‘self‐recovery’. In the absence of consistent use of terminology, being able to define the shelter reconstruction process as one of ‘self‐recovery’ depended on understanding to what degree the process was driven by beneficiaries. As Maynard, Parker, and Twigg (2017) found in their study, the most common reason for the exclusion of screened articles from the review was that they did not sufficiently detail the intervention or the outcomes. Numerous documents with potentially useful information about safety were excluded because of a lack of detail, where clear understanding of the role of beneficiaries, especially those parts that beneficiaries took responsibility for and were able to make decisions about, was impossible. For example, the literature search yielded many reports on and evaluations of owner‐driven recovery programmes that did not describe beneficiary roles and consequently could not be included, such as reports on the governmental response to the Gujarat earthquake on 26 January 2001, including the works of Duyne Barenstein (2006) and Powell (2011). To be able to contribute to the humanitarian evidence base, programme reports and evaluations need to include greater detail in their accounts of shelter self‐recovery processes.

Quality of evaluations

Measuring outcomes: are houses really safer?

To be most informative, an evaluation of a programme that aims to improve shelter safety needs to measure safety as an outcome, by assessing how well hazard‐resistant materials and techniques have been adopted by householders or how well houses withstand future disasters through long‐term follow‐up. Most of the reports in this review did not measure construction safety in any meaningful way. Very few evaluated building safety over the long term, reflecting previous findings that most of the information on the outcomes and impacts of shelter programmes comes from evaluations and observations that took place during or soon after the end of a project (Schilderman, 2014b). One of the most important elements that was lacking was robust measurement of the hazard‐resistance of the repaired and rebuilt homes. Without such information, it is not possible to assess whether the aim of improving building safety has been achieved.

Measuring impact

Similarly, an assessment of impact—causally linking intervention and outcomes—is a vital component of an informative programme evaluation (Cosgrave, Buchanan‐Smith, and Warner, 2016). Reviews of the evidence on humanitarian interventions in other fields, such as health, have shown that even where studies have been able to demonstrate changes in outcomes, many are unable to ascribe these to the intervention owing to deficiencies in the study design (Blanchet et al., 2017). These shortcomings are also apparent in the evidence about safer shelter self‐recovery. Where specific programme activities were intended to contribute to a particular outcome, assessment of whether or not they have had the desired effect is largely absent from programme evaluations and reports. The inclusion of programme features such as beneficiary training in safer building techniques suggests that agencies may have taken appropriate theories of change into account in the design of the programme itself (Funnell and Rogers, 2011; CARE International UK, 2012), but not in the design of the programme's monitoring and evaluation. Beneficiary interviews may suggest that build back safer messages have been communicated, but information on if or how they have been implemented during construction is vital—knowledge of safer construction practices alone does not guarantee that householders will include them when building their homes (Powell, 2011).

Why are there evidence gaps? And what can be done about it?

Limitations in the sources of evidence

The fact that much of the literature on humanitarian shelter programmes comprises programme evaluations, which agencies themselves frequently implement, means that there is greater potential for bias in the evidence available. Additionally, although the humanitarian literature search undertaken here followed the principles of a robust review (Stansfield, Dickson, and Bangpan, 2016), compared to an academic database search, there is a greater chance of not picking up relevant documentation, either by omitting suitable resources from the investigation, or because of how individual organisations choose to organise and label their publications on their websites.

Barriers to effective evaluation

End of programme assessment and reporting is routine in the humanitarian sector, at multiple levels of analysis—agency, disaster, programme, sector (Buttenheim, 2010). Traditionally, however, these have tended to be process evaluations, focusing on programme implementation and whether or not outputs have been achieved rather than on outcomes or impacts (White, 2009; Nath et al., 2017).

Gathering reliable information to support an assessment of outcomes and impacts can be problematic for humanitarian programmes for many reasons. The complex nature of the operating environment, ethical difficulties pertaining to the carrying out of research (Cosgrave, Buchanan‐Smith, and Warner, 2016), the short duration of interventions (giving only a brief time frame for the collection of data), constraints owing to a lack of capacity and staff turnover, and an organisational culture that may prioritise action over analysis can all be barriers to effective reporting and programme evaluation (Hofmann et al., 2004; Knox Clarke and Darcy, 2014). There can be many unpredictable variables that affect both outcomes and impacts in the humanitarian context, making attribution of cause and effect challenging. Impact evaluations also require the kind of research‐focused skills that may not be available within a humanitarian organisation (Proudlock, Ramalingam, and Sandison, 2009).

In addition, the need for specialist expertise to examine outcomes may be problematic. A technical assessment of safety features in particular may require observations by shelter experts, as illustrated by a pilot of the Shelter and Settlement Impact Evaluation Tools (SSIET), designed for specific use in the shelter sector. The trial found that the technical knowledge required to answer safety questions, such as how secure the roof was before and after the disaster and assistance, was frequently not available to the householder (UN‐Habitat, 2013).

Effective, informative data collection

The development of shared reporting guidelines and standards for data collection, as recommended by Clarke et al. (2014), would maximise the usefulness of the information gathered about individual shelter programmes, and would enable a comparative analysis of programmes and their outcomes. Recent work has begun to address this issue. In an attempt to develop a standardised instrument, Nath et al. (2017) have created a single evaluation tool based on more than 1,500 indicators selected from the numerous tools that exist to support shelter programme monitoring and evaluation.

At this time, the disaster risk reduction indicators in this and other tools are presently not sensitive enough to procure the kind of outcome information that is most relevant to evaluate effectively the impact of a shelter programme on building safety. Developing an evaluation tool that supports the collection of this information may be difficult; deciding which questions to ask in order to understand how safe a house is will be significantly context‐dependent (UN‐Habitat, 2013). Nevertheless, the principle of assessing this aspect of a programme's outcome can be built into a programme's monitoring and evaluation system, as can measurement of the percentage of houses with specific hazard‐resistant features as a proportion of all repaired or reconstructed houses.

Effectively evaluating impact in humanitarian shelter support

The importance of carrying out impact evaluations of humanitarian programmes, and the methodological issues to which this need gives rise, have started to be discussed more widely over the past decade (White, 2009; Puri et al., 2014; Cosgrave, Buchanan‐Smith, and Warner, 2016). The kind of experimental and quasi‐experimental impact evaluation methodologies frequently advocated in these discussions often are reliant on counterfactual logic to imply causality—‘what would have happened without the intervention?’. These approaches have been commonly assumed to be the best or only robust methods for evaluating the impact of an intervention. Yet, the complex nature of humanitarian shelter programmes and the circumstances within which they operate means that such methods are often unsuitable, leaving open the question of how these causal links can be established with some confidence.

The notion that these methods are necessarily superior has begun to be challenged, and it is becoming accepted that alternative evaluation methods can be equally robust and credible (Stern et al., 2012). Particularly relevant for the evaluation of shelter programmes is the possibility of utilising methods that rely on case‐based studies, especially those that make comparisons across a number of cases—in this context, a ‘case’ would be an individual shelter programme. Qualitative comparative analysis, for example, systematically compares cases in a way that permits an exploration of how and why some interventions were successful in achieving a particular outcome while others were not (Stern et al., 2012; Schatz and Welle, 2016). This technique could be used to compare the impact of shelter programmes supporting self‐recovery on construction safety.

Humanitarian organisations require significant resources to be able to carry out the type of work that is integral to an effective evaluation. The tools and methods that are necessary for robust assessment of the impacts of humanitarian programmes are already available; shelter providers need the resources and skills to be able to adapt and apply them appropriately according to context (Hofmann et al., 2004). Dedicated research and evaluation funds are needed to ensure that staff have sufficient time and the requisite skills to design and implement impact assessments (Proudlock, Ramalingam, and Sandison, 2009). Investment in skills and capacity development demands commitment from humanitarian organisations and from donors. By encouraging the programmes that they fund to evaluate outcomes and impacts and targeting funds for them to do so, donors can promote the building of an evidence base for programme effectiveness that can support their aims of maximising programme efficacy and value for money (Obrecht, 2017).

Conclusion

Building safety is an issue of indisputable significance in post‐disaster shelter reconstruction programmes, particularly among householders who self‐recover. In order to support disaster‐affected households in rebuilding their own homes to better withstand future hazards, shelter practitioners need reliable evidence that can be used to underpin shelter programme development. This review has demonstrated that existing evidence on building safety in shelter programmes that support self‐recovery is distinctly limited.

The poor quality of much of the available information means that this review can make only a limited contribution to advancing current understanding of what aspects of shelter programmes improve building safety, when they are supporting self‐recovery. However, understanding where there are deficits in good‐quality evidence can play an important role in the development of a sound evidence base. The gaps identified in this review indicate that future evaluations of shelter programmes that aim to support safer self‐recovery need to include more reliable reporting of key activities and effective assessment of outcomes. In particular, it reflects a need for programme evaluations to incorporate robust examination of how well knowledge acquired through training and other approaches is implemented in construction, and technical assessment of the safety of repaired and rebuilt houses. Ensuring that shelter programmes have sufficient dedicated resources for successful monitoring and evaluation will remain key to understanding what works and how, and to promoting an evidence‐based culture that can ensure that shelter programmes are effective and fully accountable.

Annexes

Annexe 1. Search terms

• (i) Search terms used to find articles about shelter programmes supporting self‐recovery, for the development of a set of alternative phrases for ‘self‐recovery’:

  • cash; communication; ‘community mobilisation’; ‘construction material’; ‘financial assistance’; flyer; guid*; information; leaflet; manual; market; ‘material assistance’; participation; poster; remittances; ‘tool’; training; voucher; ‘workshop’.

• (ii) Search terms used to explore academic databases and grey literature:

  • Shelter: accommodation; architecture; camp; ‘collective centre’; community; dwelling; home; hous*; housing; hut; neighborhood; neighbourhood; settlement; shelter; tarpaulin; tent; village.
  • Disaster: avalanche; catastrophe; ‘climate adaptation’; ‘complex emergency’; ‘complex political emergency’; conflict; crises; crisis; cyclone; disaster; displac*; drought; earthquake; famine; fire; flood*; hazard; hurricane; IDP; landslide; landslip; migration; refugee; starvation; storm; tidal wave; tsunami; typhoon; volcan*; war.
  • Self‐recovery: ‘community built’; ‘informal aid’; ‘local capacity’; ‘locally driven’; ‘self‐buil*’; ‘self buil*’; ‘self‐recovery’; ‘self recovery’; ‘own assets’; ‘own recovery’; ‘own resources’; ‘own solutions’; ‘owner‐driven’; ‘owner driven’; ‘recovery without assistance’; ‘self‐construct*’; ‘self construct*’; ‘self‐help’; ‘self help’; ‘self‐reconstruct*’; ‘self reconstruct*’; ‘self‐reliance’; ‘self reliance’; ‘self‐repair’; ‘self repair’; ‘self‐rescue’; ‘self rescue’; (build NEAR/5 themselves); (construct NEAR/5 themselves); (rebuild NEAR/5 themselves); (reconstruct NEAR/5 themselves); (repair NEAR/5 themselves).

• Terms for ‘shelter’ were searched for in the title and keyword database fields; terms for ‘disaster’ and ‘self‐recovery’ were searched for in the title, abstract, and keyword fields of all databases.

Annexe 2. Table of reviewed documents

No.	Author, year of publication, title	Country	Hazard type, year	Type of report/study	Summary	Assessment of safety
1	Schilderman, T. (2004) ‘Adapting traditional shelter for disaster mitigation and reconstruction: experiences with community‐based approaches’. Building Research and Information. 32(5). pp. 414–426.	Peru	Earthquake, 1990	Academic overview of community‐based shelter responses, with a case study.	Review of formal and informal community‐based approaches to disaster mitigation, including a detailed case report on a shelter reconstruction project in Peru using improved local construction techniques. Improved quincha houses better able to withstand hazards.	Evidence of hazard resistance owing to a second earthquake during reconstruction. Revisit 24 years after the project to observe how well building had withstood hazards.
2	Davidson, C.H., C. Johnson, G. Lizarralde, N. Dikmen, and A. Sliwinski (2006) ‘Truths and myths about community participation in post‐disaster housing projects’. Habitat International. 31(1). pp. 100–115.	Columbia	Earthquake, 1999	Academic overview of participatory shelter reconstruction projects, with case studies.	Case studies of four shelter reconstruction projects with varying degrees of beneficiary participation, in different regions following four disasters. One project is an assisted self‐recovery programme.	None.
3	van Leersum, A. and S. Arora (2011) ‘Implementing seismic‐resistant technologies in post‐earthquake Pakistan: a process analysis of owner driven reconstruction’. Habitat International. 35(2). pp. 254–264.	Pakistan	Earthquake, 2005	Academic evaluation of the experience of beneficiaries.	Interview study of beneficiaries’ experience of Pakistan's owner‐driven recovery (ODR) programme, involving beneficiaries from five villages. Interviews investigate opinions on new housing and the ODR process.	None.
4	Mu, J. and T. Zhou (2012) ‘Demonstrative study of the ecological post‐quake reconstruction of dwellings in poor rural regions of China’. Advanced Materials Research 374–377: Sustainable Development of Urban Environment and Building Material. Parts 1–4. pp. 339–345.	China	Earthquake, 2008	Academic case study.	Detailed description of village reconstruction project using improved local traditional earth‐based technology.	None.
5	Shelter Projects 2009. ‘Uganda – 2007 – slow onset floods’. Shelter case study. B.25. pp. 79–82. http://shelterprojects.org/shelterprojects2009.html.	Uganda	Flood, 2007	Shelter programme case report.	Brief description of pre‐disaster context, disaster, and shelter response, with a discussion of project strengths and weaknesses.	None.
6	Shelter Projects 2010. ‘Indonesia – Sumatra – 2009 – earthquake’. Shelter case study. A.14. pp. 44–46. http://shelterprojects.org/shelterprojects2010.html.	Indonesia	Earthquake, 2009	Shelter programme case report.	Brief description of pre‐disaster context, disaster, and shelter response, with a discussion of project strengths and weaknesses.	None.
7	Shelter Projects 2010. ‘Indonesia – Sumatra – 2009 – earthquake’. Shelter case study. A.15. pp. 47–49. http://shelterprojects.org/shelterprojects2010.html.	Indonesia	Earthquake, 2009	Shelter programme case report.	Brief description of pre‐disaster context, disaster, and shelter response, with a discussion of project strengths and weaknesses.	None.
8	Catholic Relief Services (2010) Catholic Relief Services Indonesia West Sumatra Transitional Shelter. Evaluation Report. 31 May. http://www.alnap.org/resource/5842.	Indonesia	Earthquake, 2009	Agency shelter programme evaluation.	Evaluation of shelter response, including household surveys, shelter completion checklist, and beneficiary interviews.	None.
9	CARE International UK (2011) Report of End‐line Assessment: Cyclone Giri Emergency Response Project, 6th December 2010 – 30th April 2011, Myebone Township. http://www.alnap.org/resource/10253.	Myanmar	Cyclone, 2010	Agency shelter programme evaluation.	Evaluation of shelter response, including household questionnaires and interviews to understand the satisfaction and experience of beneficiaries, and an assessment of the hazard‐resistant features of houses.	Follow‐up visits three months after the end of the programme; technical assessment of houses.
10	Shelter Projects 2010. ‘Pakistan – 2010 – floods’. Shelter case study. A.24. pp. 77–79. http://shelterprojects.org/shelterprojects2010.html.	Pakistan	Floods, 2010	Shelter programme case report.	Brief description of pre‐disaster context, disaster, and shelter response, with a discussion of project strengths and weaknesses.	None.
11	Shelter Projects 2010. ‘Philippines – 2010 – Typhoon Megi’. Shelter case study. A.26. pp. 83–85. http://shelterprojects.org/shelterprojects2010.html.	Philippines	Typhoon, 2010	Shelter programme case report.	Brief description of pre‐disaster context, disaster, and shelter response, with a discussion of project strengths and weaknesses.	None.
12	Shelter Projects 2011–12. ‘Pakistan – 2011 – floods’. Shelter case study. A.22. pp. 71–73. http://shelterprojects.org/shelterprojects2011-2012.html.	Pakistan	Floods, 2011	Shelter programme case report.	Brief description of pre‐disaster context, disaster, and shelter response, with a discussion of project strengths and weaknesses.	None.
13	Shelter Projects 2013–14. ‘Haiti – 2012 – Hurricane Sandy’. Shelter case study. A.6. pp. 21–23. http://shelterprojects.org/shelterprojects2013-2014.html.	Haiti	Hurricane, 2012	Shelter programme case report.	Brief description of pre‐disaster context, disaster, and shelter response, with a discussion of project strengths and weaknesses.	None.
14	Shelter Projects 2013–14. ‘Pakistan – 2012 – floods’. Shelter case study. A.20. pp. 69–72. http://shelterprojects.org/shelterprojects2013-2014.html.	Pakistan	Floods, 2012	Shelter programme case report.	Brief description of pre‐disaster context, disaster, and shelter response, with a discussion of project strengths and weaknesses.	None.
15	Shelter Projects 2013–14. ‘Philippines – 2013 – Typhoon Haiyan’. Shelter case study. A.25. pp. 86–89. http://shelterprojects.org/shelterprojects2013-2014.html.	Philippines	Typhoon, 2013	Shelter programme case report.	Brief description of pre‐disaster context, disaster, and shelter response, with a discussion of project strengths and weaknesses.	None.
16	CARE International UK (2015) CARE Philippines: Typhoon Haiyan Shelter Recovery Project Evaluation. http://reliefweb.int/sites/reliefweb.int/files/resources/CARE%20Philippines%20Typhoon%20Haiyan%20Shelter%20Recovery%20Programme%20Evaluation_1.pdf.	Philippines	Typhoon, 2013	Agency shelter programme evaluation.	Evaluation of shelter response, including household surveys and interviews to understand the experience of beneficiaries, such as understanding build back safer messages, and a field assessments of the hazard‐resistant features of houses.	Visits 6 and 18 months after the end of the programme; technical assessment of houses.
17	Ahmed, M. and A. Hrybyk (2016) Pintakasi: A Review of Shelter/WASH Delivery Methods in Post‐disaster or Recovery Interventions. Catholic Relief Services, Baltimore, MD. http://www.alnap.org/re-source/22558.	Philippines	Typhoon, 2013	Agency shelter response evaluation.	Evaluation of the efficiency, effectiveness, and appropriateness of different modalities for delivering shelter and WASH (water, sanitation, and hygiene) assistance, including beneficiary and staff focus groups.	None.
18	Levers, J. (2016) External Evaluation of the DEC‐funded Philippine Red Cross and British Red Cross Livelihoods and Shelter Recovery Programme, Iloilo Province, Philippines. 21 April. http://reliefweb.int/sites/reliefweb.int/files/resources/external-evaluation.pdf.	Philippines	Typhoon, 2013	Agency shelter programme evaluation.	Evaluation of shelter response, including beneficiary, non‐beneficiary, and staff interviews.	Mid‐term shelter repair assistance report on building quality.
19	Shelter Projects 2015–16. ‘Philippines 2013–2015 / typhoon’. Shelter case study. A.11. pp. 45–48. http://shelterprojects.org/shelterprojects2015-2016.html.	Philippines	Typhoon, 2013	Shelter programme case report.	Brief description of pre‐disaster context, disaster, and shelter response, with a discussion of project strengths and weaknesses.	Beneficiary‐reported inclusion of key safer building principles.
20	Shelter Projects 2015–16. ‘Philippines 2013–2015 / typhoon’. Shelter case study. A.12. pp. 49–52. http://shelterprojects.org/shelterprojects2015-2016.html.	Philippines	Typhoon, 2013	Shelter programme case report.	Brief description of pre‐disaster context, disaster, and shelter response, with a discussion of project strengths and weaknesses.	Beneficiary‐reported inclusion of key safer building principles.

Source: authors.

Acknowledgements

The authors would like to thank Cathrine Brun for her contribution to the discussions and document screening, as well as CARE International UK for providing funding.