Practitioner Approaches to Measuring Community Resilience: The Analysis of the Resilience of Communities to Disasters Toolkit

Abstract

To be useful for operational programs, measures of resilience must not just be valid, but be easy to use and useful. Unfortunately, while resilience measurement techniques have progressed tremendously over the past decade, most progress has been on improving validity rather than utility and ease of use. In this article we present a new tool for measuring community resilience that incorporates issues of utility and ease of use, the Analysis of Resilience of Communities to Disasters (ARC-D) toolkit. The toolkit was developed over the course of ten years by the international humanitarian and development organization GOAL to enable aid organizations to measure community resilience in a way that supports resilience building interventions. It offers an approach to measurement that is cognizant of the resilience policy landscape, including the Sendai Framework, approaches to data collection and measurement uses relevant to aid agencies. We first present the core tenants of community resilience measurement before describing the toolkit, which consists of 30 measures, a guidebook, and an online platform. To illustrate its use, we a case study of a resilience building program in Tegucigalpa, Honduras. By developing one of the first resilience toolkits focused beyond validity and providing a description of how such an assessment works, this article has implications for resilience researchers and practitioners.

Introduction

The resilience agenda has become a ubiquitous topic in both development and humanitarian communities. The recently enacted sustainable development goals make explicit reference to building resilience in a number of the targets [1]. Development bodies such as the World Bank and DFID have taken on resilience as a goal as well [2, 3]. Additionally, there have been growing calls to fundamentally alter the way that aid is practiced within the humanitarian community with calls to “re-shape aid” [4] and “do aid differently” [5]. These calls often point to building resilience as a means to improve humanitarian practice. It was a key theme of the recent World Humanitarian Summit and ensconced within the 2015 Sendai Framework for disaster risk reduction [6, 7].

While many different definitions of community resilience have been proffered through these initiatives, they generally reflect the ability to withstand shocks and stresses while maintaining core functions [8]. Variations may detail the subject, the type of shock or include adjectives such as resistance or robustness. Some may even move beyond core function to describe adaptation, thriving, and transformation—resilience as ‘bouncing forward’ rather than ‘bouncing back’ [9]. Fundamentally, however, the concept of resilience embodies the principle of making populations and systems better able to handle shocks before a crisis through an array of methods, rather than using narrower approaches to address risk such as hazard reduction alone or focusing solely on disaster response and management [10].

The reasoning and value presented by the resilience agenda is simple. Populations face a complex set of risks; improving their ability to withstand shocks requires a similarly complex and multidisciplinary approach [11]. Instead of focusing on emergency response, the resources invested in building resilience ahead of a shock is far more efficient, given the cost and consequences of disasters could be significantly reduced [12].

In light of this, the number and diversity of frameworks and instruments to measure resilience have grown over the years [13]. Their variations reflect their general purposes. They range from self-administered surveys to independent expert reviews. They also target various audiences from federal to municipal governments down to independent aid agencies. Some are composite metrics of various key categories while others are simpler models on a few core indicators. Even measures that address the resilience of the same subject to similar threats can be vastly different. This is due to the fact the science and evidence underlying many of these models is still developing [14]. As such, these various measures are not necessarily competitive with one another but represent a growing area of research and development.

A core need for practitioners is that any resilience measure be practical and able to be easily used by decision makers [15–18]. To be useful for decision makers, measurement tools must not just be valid, but must also be applicable, feasible, and useful. Decision makers only have limited resources for data collection and analysis and tools must be structured in ways that minimizes expenditures while maximizing benefit. The large literature on participatory appraisal techniques, is an example of this, as it offers relatively low cost way to collect data while having the added utility of galvanizing and empower target populations into action [19, 20].

While decision makers require resilience measures to be more than simply valid, only limited attention has been paid on how resilience measures are used in practice, and how the tools can fit decision maker needs [17, 18, 21]. Instead, efforts mainly focus on ensuring resilience measures are valid. By only focusing on validity, research outputs risk creating a lag between research and practice, and can in cases undermine the agendas that resilience practice sets out to implement [17, 21]. As a result, “tools are not useful, useable, or used, which defeats the purported purpose for measuring resilience in the first place” [18].

In this article, we present the Analysis of the Resilience of Communities to Disasters (ARC-D) toolkit, a practical toolkit developed by an international aid organization, GOAL (www.goalglobal.org), over the course of a decade of practice. GOAL is a humanitarian and development NGO with a mission to help the most vulnerable communities respond to and recover from humanitarian crisis and assist in mitigating poverty and vulnerability. Since its founding in 1977, GOAL has responded to almost all of the world’s major humanitarian crises, working with vulnerable communities in more than 60 countries. Its toolkit is designed for governmental agencies and local, national, and international aid organizations to measure community resilience to disasters in a way that is cognizant of the broader needs for applicability, feasibility, and usefulness of measures. In doing so, we bridge the gap between research on community resilience modeling and the practice of building community resilience as articulated in humanitarian and development agendas. To do so, we first review the resilience and resilience measurement literature to build an understanding of current approaches to measuring community resilience. We then overview the methodology used to develop the ARC-D toolkit before presenting the tool and demonstrating its use in a case, GOAL’s Barrio Resiliente project, a resilience building program in Tegucigalpa, Honduras. We conclude by discussing the strengths and limitations of the ARC-D and future areas of research.

Current approaches to measuring community resilience

Having a clear definition of community resilience is foundational to measuring community resilience; as Cutter [22] reminds us, when discussing resilience it is important to ask, ‘resilience to what and resilience for whom?’ There is not a monolithic understanding of resilience. The concept of resilience has origins in different sectors and disciplines, and as a result has been approached and conceptualized differently by various scholars, organizations, and agencies [23]. There are, thus, many different distinct meanings of resilience and of community resilience, as indicated by the numerous review articles of resilience (e.g. Stein [24], Winderl [25], and Manyena [9]) and of community resilience (e.g. Patel, Rogers [8], Sturgess and Sparrey [26]) that discuss competing interpretations. However, certain commonalities can be identified. In a comprehensive review of community resilience literature, Patel, Rogers [8] found that definitions fell broadly into three categories: (1) resilience as an ongoing process or capability of change and adaptation in a community affected by a disaster (exemplified Lemyre, Clément [27], Norris, Stevens [28], and Castleden, McKee [29]); (2) resilience as an absence of adverse effects, or the ability of a community to maintain stable functioning after a disaster (exemplified by Bonanno [30] and Gibson [31]); and (3) resilience as a range of attributes, a broad collection of recovery and response-related abilities within a community (exemplified by the UK Cabinet Office report on community resilience [32] and Coles and Buckle [33]). By emphasizing the human dimensions of resilience, these approaches are aligned with understandings that disasters are in no way ‘natural’, but instead reflect social processes that shape hazards and hazard exposure, vulnerability, and capacity and mean that addressing risk requires engaging in the social forces creating disaster [34].

Like resilience, ‘community’ is an equally contested and vague topic in the context of disasters. In the context of disaster, community often refers to a geographic collection of people, such as might be found in a neighborhood in a city or a census tract [17, 18]. While this geographically bounded version of community may exist, it does not to capture broader non-geographic forms of a community, from ethnic communities dispersed across geographic locations to epistemic communities that center on areas of practice or specific socioeconomic strata that may share similar risks and resilience capacities. It also obfuscates certain differences that may be found within communities, such as intra-community tensions, disparities or power relations that place one member above another, resulting in differential capacities of resilience within a community and challenging the idea of ‘community’ itself [35].

Balancing Needs

Measuring community resilience involves operationalizing definitions of community and community resilience into measures that balance many measurement demands, including related to validity, reliability, sensitivity, relevance, cost effectiveness and ease of use [16–18, 36]. Validity refers to whether a measure actually measures what it intends to measure. Reliability is whether the measure gives the same results under similar conditions if repeated. Sensitivity is the power to measure the changes it intends to measure. Cost effectiveness and ease of use reflect whether results justify the investment in time and other resources. Utility is whether the index can be used to achieve certain purposes or goals. Ideally, validity, reliability, sensitivity, and effectiveness and ease of use, and utility would be maximized, however there are tradeoffs between each attribute. For instance, a highly valid measure or set of measures often requires high levels of data collection and analysis, reducing cost effectiveness and ease of use.

There is currently no standard of practice for measuring community resilience that addresses issues related to validity, reliability, sensitivity, relevant, and cost effectiveness and ease of use. Measurement tools can be qualitative or quantitative, participatory or expert led, inductive or deductive, expert driven and top-down or bottom-up and participatory, and use primary or secondary data [37–39]. Each approach has its benefits and limitations. For instance, qualitative measures help capture people’s experiences, behaviors, opinions, attitudes, and feelings about resilience, while quantitative allow for broader, cross case comparison and can often be developed from pre-existing datasets. Participatory approaches might galvanize communities to action by providing ‘locally meaningful’ measures that are salient and in keeping with community values, but might sacrifice meaningfulness for other stakeholders [8, 20].

Modeling choices must account for the broad needs of end users. While some may argue that quantifying resilience for policy makers misses the point [40], decision makers often value quantification. Echoing calls to ensure that resilience targets both the who (people to be served) and the what (what hazards) [22], modeling resilience also encompasses issues of equity, and can support or undermine those with the greatest need [41]. How resilience is framed also has policy implications: focusing on resilience as a process of ‘bouncing back’ from disaster and returning to pre-disaster conditions rather than ‘bouncing forward’ to eliminate the conditions that allowed for disaster to occur in the first place can result in interventions that tacitly maintain the status quo [9, 42]. While these issues suggest the importance of justice, framing, and usability when making decisions related to modeling, only rarely are these issues explicitly addressed when making measurement decisions.

Although resilience measurement approaches all differ, many frame resilience as comprised of multiple dimensions – including those related to the natural and physical resources of the environment, social structures such as community networks and cultures, economic elements such as employment and financial savings, and institutional aspects such as community organizations and government structures [37]. This reflects the broader literature on resilience as the holistic outcome of a complex set of processes and the literature on disasters as a social construct and modes of resilience that capture these multiple dimensions [43]. These measures must also account for the nonlinear interacting networked properties produced by both localized and global interactions and their cascades [20, 44]. This framing is relatively consistent, however how this framing is operationalized is not, with measures characterizing the specific dimensions of resilience in different ways and ascribe different relations between measures. As a result, resilience measures ultimately represent resilience in a variety of ways and create outputs that vary within the same measurement area.

Most attempts to measure resilience also involve constructing some sort of index. An index is the composite of several qualitative or quantitative measures, which when taken together allow for a holistic picture of a context or issue. Indexes are useful for capturing complex processes that cannot be represented with single measures in a way that balances scientific accuracy and the information available at a reasonable cost and summarizes complex technical data into a more straightforward way all stakeholders can easily understand [45–47]. Their ability to represent complexity makes them useful for informing and guiding policy for complex problems. Key disaster indices includes the Cutter, Boruff [48] model of vulnerability, which uses factor analysis to distil key measures of vulnerability and applies to specific geographic areas, the Hyogo and Sendai frameworks, which both contain indicators of state progress toward disaster risk reduction, and the Mustafa, Ahmed [49] work on urban and rural specific vulnerability measures. Although useful for representing policy issues in a simple and comparable way, indexes also contain challenges, including estimation, selection of variables, measurement technique and aggregating procedures to collect and analyze the data [45]. Weaknesses can be reduced if indicators are easy to measure, tangible, adequate for aggregation and well defined, yet completely eliminating weaknesses is difficult and must instead balance competing goals. Policymakers must thus treat results of indexes as measures for guiding rather than determining policy.

The Analysis of the Resilience of Communities to Disaster (ARC-D) Toolkit

Development of the ARC-D toolkit

The ARC-D toolkit is the culmination of a 10-year project designed to develop a community resilience measurement approach that meets the needs of international aid agencies. The project entailed action-based research that drew on academic and applied literature on measuring community resilience, research and testing across 11 countries in a variety of contexts. Feedback were also received from GOAL staff, local, national, and international governmental and nongovernmental policymakers, and community members involved in community resilience. Action based research is useful for projects focused on facilitating change and developing practical outputs that can be used broadly [50–52]. ARC-D toolkit was designed explicitly as a tool to capture multiple dimensions of resilience and facilitate actionable change in an easy to use matter. Because of this, the action-based and engaged approach is therefore well suited for developing this type of toolkit.

GOAL began developing ARC-D in 2006 as part of its attempts to shift toward more preventative approaches to managing risk. To begin developing a community resilience measure, GOAL focused on developing an understanding how communities themselves understood and were managing risk. GOAL chose to focus on communities because of the organization’s community orientation and recognition of the need to work with and support rather than undermine community processes (a reflection of broader community-based disaster management approaches [53, 54]). To do so, GOAL undertook two studies of communities and disasters, both in La Moskitia, Honduras in 2007. The first, a knowledge, attitude, practice, and belief (KAPB) survey, focused on how communities understood the risks they were exposed to and their resilience to those risks. The second, a study on how they engaged in disaster risk reduction practices, was designed to understand the capacities that communities had to reduce risk and how they could potentially be supported through DRR interventions.

GOAL combined these studies with broader resilience measurement research to develop an initial community resilience measurement tool in 2010. The tool consisted of a community-level survey designed to assess community resilience to disasters comprised of 210 questions on key aspects of disaster resilience, collected as part of focus group discussions with community leaders. These initial questions drew from GOAL’s research on Honduras, as well as Twigg’s 2009 Characteristics of Disaster Resilient Communities, a widely-used document providing guidance on what constitutes community resilience [55]. The policy orientation, community orientation, and structure of the Characteristics document were useful for GOAL. From a policy perspective, the framework was designed for governmental and civil society organizations to assess and guide the implementation of community DRR supporting the Hyogo Framework for Action (HFA). To do so, it breaks resilience down into five thematic areas aligned to the five HFA priority areas. Each area has components, which include characteristics. The framework also has a specific community orientation, with its characteristics “about supporting communities to ensure that when any hazard impacts, they have the skills, resources and confidence to reduce the impact, manage the response and ensure a swift recovery” (p.4). Last, with links from characteristics to components to thematic areas, the framework was clearly organized and simple to use.

The initial survey that GOAL developed was useful because it was aligned with major policy frameworks, supported community efforts, and did so in an organized and systematic way. This made it an effective tool for measuring the progress of strengthening disaster resilience capacities, and one that was also consistent with other monitoring and evaluation tools that the organization was using. However, the survey was not easy to use, as its length was prohibitive due to the large number of items. To improve its ease of use while maintaining reliability, validity, and utility, GOAL undertook a series of activities from 2013 to 2014 to refine the measures and develop a draft toolkit. This involved field testing in Honduras, Haiti, Ethiopia, and Malawi, which consisted of training staff on toolkit use, using the toolkit, and collecting feedback to improve the toolkit. Measures were also designed to reflect resilience measurement conceptual frameworks and tools emerging in international aid policymaking and research communities (e.g. [56]). From these activities, GOAL developed a refined set of indicators that consisted of 30 key resilience measures, again grouped to align with the Hyogo Framework priority areas. To guide their use, in 2014 it also developed an initial version of a toolkit designed to offer guidance on how to use the measures.

After GOAL published the toolkit, it rolled it out to 11 of its country offices, Honduras, Haiti, Nicaragua, Niger, Sudan, South Sudan, Ethiopia, Kenya, Uganda, Malawi, and the Philippines. It also shared the toolkit with national disaster risk management agencies, UN agencies, other NGOs, and donor representatives. In Honduras the national government’s disaster management authority, the Permanent Contingency Commission of Honduras, began using ARC-D. Municipal authorities and GOAL used ARC-D as part of a recovery program. It was also featured in policy documents such as the European Union’s 2015 best practice compendium on resilience [57].

Feedback from users was incorporated into the final version of the toolkit that was published in 2016. To better capture the broader context, the toolkit was expanded to include tools that could be used for contextual analysis and identify the key environmental, governance, and planning resources enabling and obstructing resilience. The version was also updated to reflect new policy processes, namely the 2015 Sendai framework, the successor to the Hyogo Framework, and expanded to include an online dashboard to make it easier to use.

Conceptual parameters of ARC-D

ARC-D is based on GOAL’s definition of community resilience. GOAL defines community resilience as “the ability of communities and households within complex systems to anticipate and adapt to risks; and to absorb, respond and recover from shocks and stresses in a timely and effective manner without compromising their long-term prospects - ultimately improving their well-being” [58]. For GOAL, the aim of resilience is to ensure that communities can thrive in the face of shocks, stresses, and other serious adversities, preserve development gains, and reduce their need for humanitarian assistance. GOAL understands a ‘community’ to be a group of people who 1) live in the same area and are exposed to the same risks (i.e. a village or an urban neighborhood) and 2) have the same governance or decision-making structures and socio-economic fabric and 3) experience a set of shocks. Although different community members can have different levels of resilience [35], GOAL uses community as a unit of analysis. To facilitate inter-group reliability of resilience, it focuses on smaller groups within defined decision-making structures and similar hazard exposure as a way of reducing variation when describing community.

GOAL focuses on any shock, stress, or adversity that disrupts community functioning and creates widespread impacts. These include shocks derived from natural hazards, including floods, high winds, landslides, droughts, but also shocks from human derived hazards, such as conflict, technological disasters, and economic volatility. These also include stresses such as trends including natural resource degradation, loss of agricultural production, unplanned urbanization, demographic changes, climate change, political instability, and economic decline.

Figure 1 presents the conceptual framework underpinning ARC-D.

Figure 1.

GOAL’s community resilience framework (adapted from Frankenberger, Mueller [56]).

The framework draws from Frankenberger, Mueller [56]’s framework of resilience to provide a structured way to conceptualize resilience. The ARC-D framework conceptualizes communities as groups of people with a set of resources, vulnerabilities, and exposure to shocks and stresses. They operate within a community that is dependent on multiple interacting and interconnecting systems. It identifies 8 of these system sectors: (1) education, (2) economic, (3) environmental, (4) policy and governance, (5) health, (6) infrastructure, (7) social and cultural, and (8) disaster risk management. When shocks or stresses hit, their impact depends on those systems, and they can create feedback loops that affect those systems and increase or decrease vulnerability and resilience.

The ARC-D framework provides a structured way for evaluating communities and the community context to develop interventions that support community resilience. Necessary areas to examine include:

• Identifying the community interventions will benefit (resilience for who?)

• Evaluating the context within that community resides (resilience for who?)

• Assessing the shocks and stresses that the community faces (resilience to what?)

• Determining the level of exposure to shocks and stresses that the community faces (resilience to what?)

• Determining the systems and levels where interventions will occur (resilience how?)

The ARC-D measures

Table 1 presents the 30 key components for resilience, that serve as base measures for analysis.

Table 1.

the 30 resilient components, with their associated questions and Sendai Framework correlation

Component	Question	Sendai Framework priority area
1. Participatory risk assessment	Has the community carried out a participatory risk assessment (hazard analysis, VCA, impact analysis), shared the findings and have human resources capable for conducting and updating this assessment?	Priority 1: Understanding disaster risk
2. Scientific risk assessment	Does the community combine local knowledge and perceptions of risk with scientific knowledge, data, and assessment methods?
3. Dissemination of DRR information	Have community members been exposed to/have participated in DRR specific awareness events (campaigns, discussions and trainings) and have improved awareness and practices as a result?
4. Education of children on DRR	Are DRR and recovery knowledge and capacities being passed on to children formally through local schools and informally via oral tradition from one generation to the next?
5. DRR in development planning	Does the community see DRR as an integral part of plans and actions to achieve wider community goals (e.g., poverty alleviation, quality of life)?	Priority 2: Strengthening disaster risk governance to manage disaster risk
6. DRR in land use planning	Does the community decision-making regarding land use and management take disaster risk into account?
7. Community decision-making	Is the community leadership committed, effective, and accountable?
8. Inclusion of vulnerable groups	Are the vulnerable groups in the community included and represented in community decision making and management of DRR and recovery?
9. Participation of women	Do women participate in community decision making and management of DRR and recovery?
10. Rights awareness and advocacy	Is the community aware of its rights, relevant legal mechanisms and responsible actors for their fulfilment, and does it advocate for these?
11. Partnerships for DRR and recovery	Are there clear, agreed and stable partnerships between the community and other actors (local authorities, NGOs, businesses, etc.) that provide resources for DRR and recovery?
12. Sustainable Environmental Management	Does the community adopt sustainable environmental management practices that reduce disaster risk and new risks related to the effects of climate change?	Priority 3: Investing in disaster risk reduction for resilience
13. Water security and management	Does the community have access to sufficient quantity and quality of water for domestic needs during disasters?
14. Health access and awareness	Do community members maintain good health in normal times through appropriate awareness and practices (adequate nutrition, hygiene, and health care access)?
15. Secure and sufficient food supply	Does the community have a secure and sufficient food supply during disasters?
16. Hazard-resistant livelihoods practices	Does the community employ hazard-resistant livelihoods practices for food and income security?
17. Access to market	Are the local market links for products, labor and services protected against shocks?
18. Access to financial services	Are there affordable and flexible financial services (savings and credit schemes, microfinance), whether formal or informal?
19. Income and Asset protection	Are household asset bases (income, savings, and convertible property) sufficiently large and diverse and protected to ensure reduced vulnerability to disaster?
20. Social protection	Does the community have access to informal and formal social protection schemes that support disaster risk reduction and recovery?
21. Social cohesion and conflict prevention	Is there a sense of peace, security, and effective conflict prevention and mitigation mechanisms, both within the community and with other communities?
22. Critical infrastructure	Are the community’s critical infrastructure and basic services resilient to disaster (e.g. located in low-risk areas, using hazard-resistant construction methods, and structural mitigation measures)?
23. Housing	Is the community’s housing resilient to disaster (e.g. located in low-risk areas, using hazard-resistant construction methods, and structural mitigation measures)?
24. Contingency and recovery planning	Does the community use communally developed contingency and recovery plans that are widely understood and include measures to protect vulnerable groups?	Priority 4: Enhancing disaster preparedness for effective response, and to “Build Back Better” in recovery, rehabilitation and reconstruction
25. Early Warning System	Is there an operational early warning system in the community?
26. Capacity in preparedness, response and early recovery	Does the community have a trained and operating organization in disaster preparedness, response, and early recovery?
27. Health services in emergencies	Does the community have access to health care facilities and health workers equipped and trained to respond to physical and mental health consequences of disasters, and supported by access to emergency health services, medicines, etc.?
28. Education services in emergencies	Do education services have the capacity to continue operating in emergencies?
29. Emergency infrastructure	Are emergency shelters (purpose-built or modified) accessible to the community and have adequate facilities to meet basic needs for all of the affected population?
30. Leadership and volunteerism in response and recovery	Does the community play a leading role in coordinating preparedness, response and recovery, reaching all affected people – including the most vulnerable – through organized and trained volunteers?

Designed to assess how communities are resilient to specific shocks and stresses in order to develop community resilience interventions, and developed from the understanding of community resilience and resilience framework outlined above, the measures offer a comprehensive, multi-system perspective to community resilience. Reflecting the multi-systems nature of resilience, each measure corresponds to one of the 8 system sectors identified in the framework (Table 2):

Table 2.

the resilience components and their alignment to the 8 community resilience system sectors

System sectors	Associated questions
Education	4, 28
Economic	11, 15, 16, 17, 18, 19, 20
Environment	6, 12, 15, 16, 9
Political/governance	5, 6, 7, 10
Health	13, 14, 15, 22
Infrastructure	22, 23, 29
Social and cultural	3, 4, 7, 8, 9, 19, 20
Disaster risk management	1, 2, 3, 4, 6, 11, 16, 24, 25, 26, 29

To facilitate alignment to international policy processes, the measures have also been grouped under the four 2015 Sendai Framework Priorities for Action: (1) Understanding disaster risk; (2) Strengthening disaster risk governance to manage disaster risk; (3) Investing in disaster reduction for resilience and; (4) Enhancing disaster preparedness for effective response.

Each measure has rating guidelines, where assessors rate the community’s resilience from 1, minimal resilience, to 5, full resilience. Ratings account for awareness, action, comprehensiveness, and sustainability of community activities:

1. Minimum resilience: little awareness of issues and no action

2. Low resilience: some awareness and motivation, some action, but action is piecemeal and short term

3. Medium resilience: awareness and long-term actions, but these are not linked to long term strategy and/or not all aspects of the problem are addressed

4. Approaching resilience: actions are long term, linked to strategy and address main aspects of the issue, but there are still deficiencies in implementation

5. Resilience: actions are long-term, linked to a strategy, address all aspects of the issue, and are embedded in society and sustainable.

Scores from the 30 individual measures can be aggregated to provide an overall score of a community’s resilience (Table 3):

Table 3.

community resilience levels and ARC-D scores

Resilience level	score	Description
1. Very low resilience	30–45	Very limited awareness and knowledge of the problem(s). No action taken.
2. Low resilience	46–75	A certain awareness of the problem(s), willingness to act, some actions taken, but actions are fragmented, and solutions are only short term.
3. Medium resilience	76–105	Awareness of the problems and long-term actions taken, but not related to a long-term strategy and/or addressing all aspects of the problem(s).
4. Close to resilience	106–135	Long-term actions, in accordance with a predefined strategy, addressing the main aspects of the problem(s), but are inhibited by persistent shortcomings in their implementation.
5. Resilience	136–150	Long-term actions are undertaken in accordance with a pre-defined strategy assessing all aspects of the problem(s); they are sustainable and supported by the community.

Like the individual measurements, a community’s overall resilience can range from 1, very low resilience (30–45 points) to 5, resilience (136–150 points).

The ARC-D toolkit

The ARC-D toolkit, which is available online (http://resiliencenexus.org/arc_d_toolkit/what-it-is/), is designed to facilitate easy and consistent use of the ARC-D measures. The toolkit introduces ARC-D measures and its background, including the conceptual framework ARC-D is based on, describes the measures and their utility and rationale, and outlines the process for applying the measures, including preparation, implementation of measures, analysis, and use of results. The toolkit also has several annexes designed to make using the measures easier, including a glossary of terms, questionnaires for overall community assessments and for conducting the ARC-D assessment, steps for using the digital data gathering platform, descriptions of the 30 disaster resilience components and their rationale for inclusion, and an ARC-D assessment report template.

The toolkit outlines a two-stage data collection process for using the measures. The first stage is a preliminary assessment of the community and their risks, which is conducted before the ARC-D measures to understand the broad community risk context. This assessment involves collecting population data and data on vulnerable groups, local community organizations, community plans, the main shocks and stresses affecting the community, their impact and the community coping strategies. Data collected includes a review of secondary information, observations, and key informant interviews of community leaders from local institutions. These data are analyzed collectively by facilitators and participants to develop risk scenarios, which show how hazards and community institutions develop, interact, and evolve, including the probability and impact of different risks and community coping mechanisms for addressing risks. Together with community informants, a risk scenario emerges prioritized for the second stage of analysis of the ARC-D toolkit.

The second stage of data collection involves using the 30 measures to assess the community’s disaster resilience for the chosen risk scenario. Here, consensus-based focus group discussions (FGDs) are used to collect data. Reflecting GOAL’s experience implementing FGDs for other programs, FGDs were limited in range from 8–12 participants, who represent different vulnerable groups, community leaders, long standing community members, and health, education, and representatives of other governmental sectors if they live in the community. To facilitate the discussion the operator uses the 30 key questions and a series of guiding questions outlined in the toolkit to allow the assessment of each component. After each measure is discussed, the facilitator uses discussion results to rate each community’s resilience, and confirms ratings with FGD representatives. Ratings are then aggregated to develop overall scores, which are discussed with FGD participants.

To make collection and aggregation easier, GOAL developed an online data collection platform for the ARC-D measures. The platform (found at http://resiliencenexus.org/) is based on CommCare, which operates on Android devices and stores data on cloud-hosted servers. Data can be collected offline and sent to the project’s CommCare database once connected. Data can then be exported to a custom dashboard that can be used to monitor data collection in almost real-time. This dashboard can be used to generate reports that includes visualizations of overall resilience scores, the detailed risk scenarios, individual component scores and scores per sector categories (economic, social, health etc.) and per Sendai priorities. These reports are simple, informative, and easy to understand, making them useful for sharing with stakeholders and decision makers involved in community disaster resilience building.

The ARC-D toolkit in action: applying ARC-D to the Barrio Resiliente program in Tegucigalpa, Honduras

GOAL applied the ARC-D measures and toolkit during its 2013 to 2017 Barrio Resiliente resilience program. The application illustrates how ARC-D can be used for resilience programming. Barrio Resiliente, Spanish for ‘resilient barrio’, was designed to support the resilience of communities living in neighborhoods of José Arturo Duarte, José Angel Ulloa and Nueva Providencia in Tegucigalpa, Honduras. These neighborhoods are informal settlements in excluded geographic locations exposed to hazards such as landslides and flooding. Housing is poorly constructed, with little or no access to basic services nor technical services and high levels of poverty. To address these risks, the Barrio Resiliente program involved resilience building activities including carrying out technical studies on hazards, developing DRR and drainage master plans, among others. Although the focus was on community resilience, community resilience is a networked property dependent on the actions of many stakeholders outside of the community. Thus, these activities included activities at barrio, city, and national levels, including activities with community leaders, the Central District Municipality, academic and private enterprise sectors and the National Disaster Risk Management Agency (COPECO).

ARC-D was used to complement the project’s baseline and endline measures, and as a tool for facilitating change in communities. As a first step, GOAL staff, who have backgrounds in a wide variety of relevant areas such as community development, disaster management, livelihoods, and health, started by compiling basic information related to the hazard and risk environment. This analysis included review of academic and technical reports on neighborhood hazards, not only to better understand hazards, but also to define where to place physical infrastructure and develop the DRR master plan. The assessment identified flood, landslide, and rockslides as major hazards, with most houses built in areas prone to flooding, landslides, and rockslides, constructed with poor quality materials and without following building codes or other technical assistance and with deteriorating informal water point connections, resulting in soil saturation and increasing chances for flooding and landslides. Water and sanitation issues were also prevalent: 12% of the population relied on hand-dug wells for water, which is highly vulnerable to contamination, and only 22% of the houses had sewage connections, with the rest either using latrines (69%) or practicing open defecations (9%).

Following this contextual assessment, ARC-D toolkit was used for baseline, intermediate, and endline evaluations for each of the three neighborhoods. The baseline evaluation was conducted in 2013, and involved a home-level survey, interviews and FGDs with neighborhood representatives and other stakeholders, and the application of ARC-D. The intermediate evaluation included interviews and FGDs with neighborhood representatives and other stakeholders, and the application of ARC-D. The final evaluation, which occurred in 2013, was similar to the baseline in that it involved household surveys, interviews and FGDs with neighborhood representatives and other stakeholders, and the application of ARC-D.

The baseline and endline scores can be seen in Figure 2:

Figure 2.

ARC-D baseline and endline measures for the Barrio Resiliente program in Tegucigalpa, Honduras

The figure shows that some scores increased while others decreased. Decreasing scores, Community decision making, Capacities in preparedness and response and early recovery, and Education services in emergencies. The decrease in 7 is a result of the main community’s decision-making structures and the absence of local development plans which contributes to short-term actions and limited accountability of community leaders. The decrease in 26 is due to the limited number of people trained and still active in the local emergency’s committees. The decrease in component 28 is explained by the school’s deterioration resulting from a geological fault, no longer considered a safe shelter during emergencies. Increasing scores included Health access and awareness, Early warning systems, and Access to financial services. These were in part due to GOAL’s targeted efforts in community resilience building.

While there was variation, overall scores improved, increasing from 59 in 2013 to 76 in 2018. These scores indicate a shift from level 2, low resilience (“certain awareness of the problems, willingness to act, some actions taken, but actions are fragmented and only short-term solutions”) to 3, medium resilience (“Awareness of the problems and long-term actions taken, but not related to a long-term strategy or addressing all aspects of the problems”).

The toolkit also shows improvement in Sendai Priority areas (Table 4).

Table 4.

Sendai Framework baseline and endline scores

Sendai Framework Priority Area	2013 baseline score	2018 endline score
1: Understanding Disaster Risk	8	10
2: Strengthening Governance to Manager Disaster Risk	16	18
3: Reducing Vulnerability for Resilience	18	26
4: Enhancing Disaster Preparedness for Effective Response and to “Build Back Better” in recovery	17	21
Overall resilience score	59	75

The table shows that from 2013 to 2018 progress was made in achieving Sendai Framework for Action goals.

Along with using the ARC-D toolkit to develop baseline and endline measures, GOAL was able to use ARC-D to identify several ways that it could improve programming. Through a series of discussions and workshops with the project team and head office, GOAL identified several lessons that it could apply to its programs:

1. Establish a communication strategy that allows the mass dissemination of DRR information. Although the project disseminated DRR information, results of ARC-D made it clear that more was needed to reach the public. To do so, GOAL identified key messages and channels for transmitting DRR information to the public, resulting in training religious leaders and other community members as “Prevention Preachers” to reach target groups.

2. Address key issues of local DRR governance. Results of ARC-D related to Community decision-making, Capacities in preparedness and response, and Leadership and volunteerism in response and recovery, revealed tensions between community leaders that inhibited participation in project activities and in supporting emergency response committees. To address these social dynamics and increase participation and community support, GOAL incorporated community conflict resolution into the program.

3. Use a systems approach bring together communities, local government agencies, and other key stakeholders to reduce disaster risk. ARC-D helped identify interlinked and multi-level systems, such as social housing and drainage systems, that requires support from a multitude of actors including communities, state agencies, members of the private sector, and NGOs. GOAL can act as facilitator, bringing these stakeholders together to strengthen systems and improve resilience.

Discussion

Although there is inconsistency in community resilience research on how resilience is defined, understood, and therefore measured, resilience measurement tools are characterized by an explicit approach to systems thinking and recognition of the many economic, social, and political factors that shape the ability to handle shocks and stresses, and often employ some sort of index. These approaches should also be cognizant of the end goals of measurement users and may face tradeoffs between issues of validity, reliability, ease of use, and utility. This article presented the ARC-D toolkit, a community resilience measure intended to balance validity with ease of use and utility for both practitioners and policymakers.

Assessment of ARC-D

GOAL’s choices designing in the ARC-D toolkit – from key definitions to frameworks, to measures – reflect a version of community resilience that is cognizant of issues of validity, reliability, feasibility, and ease of use that aid organizations face. The focus on the multiple dimensions of resilience and the need to understand the deeper social reasons for vulnerability, articulated in GOAL’s definition of community resilience and resilience framework, align with important theories on risk and resilience. These include vulnerability theories of risk, that emphasize that inequality and marginalization is what creates risk for many people (see, e.g. [34]), and approaches to resilience that emphasize the importance of understanding and mitigating cross-system interactions (e.g. [11]). The operationalization of the framework, including through its definition of community and the 30 measures, reflect the need for a tool that aid organizations can use to enhance their programming. For instance, by incorporating Sendai framework measures, results of the toolkit can be used to contribute to important policy discussions; by relying on community FGDs for data, the toolkit aligns with NGO approaches to risk management that emphasize community knowledge and participation [53, 54].

The Honduras case offers a demonstration of how ARC-D can be applied to analyze different capacities that exist in the different contexts, and that it can be adapted to the specific circumstances of each community. The changes in scores capture the dynamic nature of resilience, showing how resilience changes due to both changes in community capacities and the systems that communities depend upon.

GOAL’s experiences using the ARC-D toolkit illustrate how resilience measurement tools can move beyond validity to consider issues of reliability, ease of use, and utility. Yet its experiences also illustrate the challenges in such measures, in that it was not able to construct a toolkit that was fully able to account for all measures (Table 5):

Table 5.

ARC-D strengths and weaknesses related to validity, reliability, ease of use, and utility

Validity

+ It provides a holistic snapshot of the components of resilience in a community in a locally meaningful way.

+ Based on literature and empirically tested across multiple cases.

− Subjective ratings are dependent on community and facilitator interpretation and vulnerable to bias and error.

Reliability

+ ARC-D is agnostic to hazards and context, so can work across a variety of communities and risk scenarios.

− For extremely sensitive issues, certain components may need to be removed from the community questionnaire.

− Issue of security may limit an open discussion on sensitive issues related to resilience, for example in understanding the influence of organized crime in insecure communities.

− Subjective ratings are dependent on community and facilitator interpretation and vulnerable to bias and error.

Ease of Use

+ It is user-friendly from the personnel training phase to the application in the field.

+ It aligns with key policy processes.

+ It offers a rapid qualitative assessment of the main risk scenarios that affect a particular community.

+ Resilience assessments can be processed quickly providing an easily understandable dashboard of the context of resilience in a community.

+ It is relatively quick and straightforward to implement and does not require significant human resources or funding and relatively little time investment from community representatives.

− There can be sensitivities around leadership, gender equality, accountability, and other issues in communities that can limit open discussion, requiring additional training from facilitators

− Part A can be time consuming if there is no information available.

− Attention needs to be given to getting the right profiles and balance of representatives for the focus group discussions.

− Attention needs to be given to get the right profile of facilitators. The guidebook provides recommendations on profile.

Utility

+ It increases communities’ knowledge and awareness of the risks they are exposed to, their level of resilience, and by focus group discussions that tie the measures to locally meaningful processes, incites them to discuss measures to increase their resilience.

+ Serves as an entry point into broader systems analysis.

+ It provides a space where community members and leaders meet and establish or strengthen relationships, and collaborative actions can be born.

+ It increases understanding of the root issues of poverty and vulnerability.

+ It can be used as a diagnostic tool, an evaluation tool and as a planning tool for the creation of strategic frameworks for resilience building.

+ It generates conversation, supports integration and fosters understanding between the survey team and the participants of the focal group.

+ It allows for comparisons between different context and risk scenarios.

+ It is fully aligned with the global policy on DRR as described in the Sendai Framework.

+ It provides the opportunity to learn from communities and understand existing coping strategies.

− Can only be applied to a singular hazard at a time.

The strengths and weakness of the ARC-D toolkit suggests that organizations seeking to measure resilience will need to balance tradeoffs in the measurement decisions that they make. Tradeoffs appear inevitable, so must be justified depending on the purpose of the measure. For instance, ARC-D relied on community views captured through a facilitator rather than expert views or external quantitative data, because community views represent an important ‘bottom up’ perspective in risk management, and community facilitation is a method that is quick, matches the data collection techniques typical to aid organizations, and if done right, facilitates community awareness, ownership, and action. While a community focus and use of facilitator enhances utility and ease of use, it also means that measures are dependent on the background profile of the community and facilitator, potentially impacting validity and reliability. The deliberate choices in developing the measure and toolkit thus represent an attempt at navigating a series of tradeoffs maximize the toolkit’s benefit and minimize its weakness.

Future research

Several areas of future research can be explored. First, there is a need to have a better understanding of measurement approaches that are cognizant of end user needs and can more easily model and capture community resilience from an explicit systems perspective. ARC-D highlights the numerous components necessary for community resilience, but does not conceptualize how those components interact to shape community resilience. As a result, it is not able to identify potentially critical nodes whose failure – or success – points might ripple through communities or lead to unexpected or unintended effects. Efforts should draw from broader systems measurement efforts and be tailored to the needs of NGOs and target communities. Fuzzy cognitive mapping might be beneficial as part of this research approach, as they are explicitly designed to develop models from multiple stakeholders from the needs of those stakeholders themselves [59–61].

Second, efforts should be directed at understanding how communities function as systems of systems. GOAL’s experience developing and using ARC-D highlighted the system of system nature of communities. For instance, Social cohesion and conflict prevention is an entire system within a community, and is dependent on other community system sectors, such as those related to education, the economy, and political and governance structures. Interventions designed to deepen resilience need to understand how these systems function, where they are weak, and how they should be strengthened. To guide resilience-strengthening activities, research should examine these systems from a community resilience perspective, drawing on models that show how risks are created, such as the Pressure and Release framework [34] and Forensic Investigations of Disasters [62], as a first step in identifying some of the factors explaining why these systems are at risk and how they might be strengthened.

Conclusion

In this article, we set out move beyond validity and reliability and develop measure of community resilience that meets the needs of practitioners. The resulting measurement toolkit, the ARC-D Toolkit, involves more than just validity and also incorporates issues of ease of use and utility into the selection of measures and development of the toolkit, making it well suited for aid organizations seeking to implement community resilience programs. In doing so, however, GOAL also had to make a series of tradeoffs over the measures, at times sacrificing ease of use for validity or vice versa. These experiences indicate that community resilience measurement is less about developing measures that can satisfy all design criteria and more about minimizing the negative elements of measurement choices and maximizing the positive. Efforts to improve community resilience measures should therefore be cognizant of these tradeoffs and explicitly account for how they balance validity, reliability, ease of use, and utility.