A rapid assessment framework for food system shocks: Lessons learned from COVID-19 in the Indo-Pacific region

Abstract

The frequency and severity of shocks to food systems is accelerating globally, exemplified by the current COVID-19 outbreak. In low- and middle-income countries, the impacts have exacerbated existing food system vulnerabilities and poverty. Governments and donors must respond quickly, but few tools are available that identify interventions to build food system resilience, or emerging opportunities for transformation. In this paper we reflect on the application of a systems-based rapid assessment which we applied across 11 Indo-Pacific countries in May-July 2020. Our approach was shaped by three design parameters: the integration of key informants’ perspectives engaged remotely within the countries, applicability to diverse food systems and COVID-19 experiences across the region, and the consideration of food systems as complex systems. For the rapid assessment we adopted an analytical framework proposed by Allen and Prosperi (2016). To include a development lens, we added the analysis of vulnerable groups and their exposure, impacts, recovery potential and resilience, and pro-poor interventions. We concluded that the framework and approach facilitated integration and triangulation of disparate knowledge types and data to identify priority interventions and was sufficiently flexible to be applied across food systems, at both national, sub-national and commodity scales. The step-wise method was simple and enabled structured inquiry and reporting. Although the systems concepts appeared more easily transferrable to key informants in some countries than others, potentially transformational interventions were identified, and also some risks of maladaptation. We present a refined framework that emphasises analysis of political, economic and institutional drivers of exposure and vulnerability, the constraints that they pose for building recovery potential and resilience, and trade-offs amongst winners and losers inherent in proposed interventions.

1. Introduction

Across the world the frequency and severity of shocks to food systems is increasing, driven by climatic, economic and socio-political events (Cottrell et al., 2019; Woetzel et al., 2020). Globalisation, characterised by inter-connectedness through trade liberalisation, transport, migration, finance and tele-communications (Goldin and Reinert, 2006) is accelerating the speed and scale of such shocks. This causes ‘synchronous failures’ where cascading impacts have far-reaching consequences (Homer-Dixon et al., 2015). The rate, scale and depth of shocks necessitates that society must respond with agility and flexibility, often with imperfect information or evidence (Quigley et al., 2020).

The current COVID-19 pandemic exemplifies this challenge (Barrett, 2020). Over a few months in 2019–2020 the SARS-CoV-2 virus is reported to have spread from China to infect most countries (Callaway et al., 2020), reversing many gains in poverty alleviation, food and nutrition security in low- and middle-income countries (Husain et al., 2020; Sumner et al., 2020; United Nations Development Program (UNDP, 2020). The pandemic has exacerbated existing deprivation and forced reconsideration of the Sustainable Development Goals (Naidoo and Fisher, 2020). In some regions COVID-19 has coincided with climatic disasters to create compounded ‘multi-hazards’ (Quigley et al., 2020), amplifying development impacts (McClean, 2020). Due to global population growth, encroachment and modification of natural habitats, zoonotic pandemics are likely to become more frequent (Di Marco et al., 2020).

It is against this backdrop of increasingly prevalent shocks that governments and donors are having to re-direct resources towards emerging vulnerabilities (Quigley et al., 2020), particularly for food systems in the developing world (Barrett, 2020; Husain et al., 2020). COVID-19 has triggered a raft of rapid responses by donors, for example the Food and Agriculture Organisation of the United Nations (Food and Agriculture Organisation of the United Nations (FAO, 2020; Husain et al., 2020; Torero Cullen, 2020), the United Nations Development Program (United Nations Development Program (UNDP, 2020), the United Nations Economic and Social Commission for Asia and the Pacific (United Nations Economic and Social Commission for Asia and the Pacific (UNESCAP, 2020) and the International Fund for Agricultural Development (International Fund for Agricultural Development (IFAD, 2020). These policy responses have largely been high-level, and not guided by a structured scientific diagnosis of the food systems of concern (Béné, 2020).

Currently tools and processes for carrying out rapid analyses of food systems’ vulnerabilities to shocks are underdeveloped, as are those for identifying interventions that kick-start rapid recovery and build capacity to buffer and respond to future crises (Ahmed et al., 2020). Since shocks may provide windows of opportunity for transformation (Birkmann et al., 2009; McSweeney and Coomes, 2011; Brundiers and Eakin, 2018), identifying key leverage points for transformational post-disaster investment is also critical. Prior to COVID-19 a step-change in smallholder productivity and sustainability was essential to meet global food demand and alleviate poverty (Herrero et al., 2017; Thornton et al., 2018), and this imperative is now even stronger. Equally, it is important that potentially maladaptive responses which exacerbate future exposure to shocks are recognised, addressed or avoided (Barnett and O’Neill, 2010; Wise et al., 2016).

In this Short Communication we present a reflection on a rapid assessment of the impacts of COVID-19 on food systems in the Indo-Pacific region. As part of the Australian Government’s response in this region, the Australian Centre for International Agricultural Research (ACIAR, an Australian Government statutory agency) commissioned this study to inform the re-prioritisation of regional food system research and development investments by Australia, its partners and other donors. Rather than detail the results of the assessment, in the mode of interdisciplinary, reflexive sustainability science (Salas-Zapata et al., 2013) we share our experiences of designing and implementing an analytical framework under acute time and logistical constraints. We then present a refined framework to support similar exercises seeking to guide responses to COVID-19 and other inevitable future shocks to food systems.

2. Design parameters for the assessment

In May-July 2020 we carried out a preliminary assessment of the impact of the COVID-19 crisis on 11 countries: the Philippines, Indonesia, Timor-Leste, Papua New Guinea (PNG) and seven Pacific Island Countries (PICs, Table 1 ). These are low- to middle-income nations with Human Development Indices ranging from 0.543 (PNG, ranked 131 in the world) to 0.724 (Fiji, ranked 98).

Table 1.

Summary of the rapid assessment countries, their Human Development Index (HDI), COVID-19 experiences and government policy responses.

Focal country	Population 2019a	HDI 2019 (rank out of 189 nations)b	COVID-19 cases (31 July 2020)	COVID-19 deaths (31 July 2020)	Policy responses
Philippines	110 million	0.712 (106)	89,374	1983	Presidential Declaration of State of Public Health Emergency in March-May; physical distancing suspending school, work and mass gatherings; limited pedestrian movement, but cargo transport allowed; only frontline work allowed (e.g. health, food services, security); local government access to Calamity Funds for response
Indonesia	274 million	0.707 (111)	106,336	5058	National government funding allocation to mitigate impacts, including social assistance and pre-employment training program; provincial governments implementing Large-Scale Social Restrictions; Agricultural Ministry ensuring access to agri-inputs for farmers; National Logistics Agency stabilising prices and prioritising distribution of key commodities
Timor-Leste	1.3 million	0.626 (131)	24	None	State of Emergency declared; closure of international borders and restricted domestic travel; closure of schools and many businesses; provision of temporary payments to some households and food relief; import of additional rice supplies
Papua New Guinea	8.9 millionc	0.543 (156)	63	2	State of Emergency declared; lockdown in March-June; fresh food markets closed; provincial borders closed; international borders restricted; variable responses at provincial level; some government support with international funding for medical response; further lockdown in July and ban on domestic travel
Fiji	896,000	0.724 (98)	27	None	State of Emergency declared; varying levels of lockdown measures; closure of international borders; varied economic packages for social protection and agricultural production
Solomon Islands	687,000	0.557 (153)	None	None	State of Emergency declared; closure of international borders; economic stimulus package with strong focus on agriculture; policies to control price increases; market in capital city rehabilitated to encourage access to fresh food
Vanuatu	307,000	0.597 (141)	None	None	State of Emergency declared; economic stimulus package to support unemployment; COVID- 19 Food Security Response Plan developed by government; food aid required to manage the impact of Cyclone Harold in April; closure of international borders
Tonga	106,000	0.717 (105)	None	None	State of Emergency declared; closure of international borders; economic stimulus package with agriculture and fisheries as a priority
Samoa	198,000	0.707 (111)	None	None	State of Emergency declared; closure of international borders; supplementary budget announced; reduction in pension contributions during 2020
Tuvalu	12,000	NA	None	None	State of Emergency declared; closure of international borders; economic stimulus package; home gardens and seedling supplies supported; agricultural projects fast tracked
Kiribati	119,000	0.623 (132)	None	None	National lock-down on people movement; citizens encouraged to return to their home villages

^aThe World Bank (2019).

^bUNDP (2019).

^cMcMurray and Lavu (2020).

The aims of the study commissioned by ACIAR were to rapidly identify investment priorities for future agricultural research, development and innovation, and other potential food system interventions. As a result, three design parameters delimited our approach and analytical framework.

2.1. Rapid integration of diverse data and knowledge

The 3-month activity window necessitated the design and implementation of a rapid assessment, defined as an intensive, team-based qualitative inquiry using triangulation, iterative data analysis and additional data collection to quickly develop a preliminary understanding of a situation from the insider's perspective (Given, 2008). We formed rapid assessment teams for each country, consisting of multiple disciplines (Table 2 ).

Table 2.

Characteristics of the rapid assessment teams, key informants, data and selected scale of interest for each focal country.

Focal country	Teams’ disciplines	Key informants	Number of interviews (women/men)	Secondary data	Scale of interest	Country report
Philippines	Agri-business	Agri-business practitioners, farmer associations, women’s groups, government officials	17 (7/10)	Government databases, donor reports	Rice, pork, cabbage and bananas in the National Capital Region (Manila)	Palo et al. (2020)
Indonesia	Rural sociology	Researchers (agricultural economists, sociologists), national government policy-makers	25 (7/18)	Grey literature, newspaper articles, donor reports, blogs, research papers	Rice and vegetables in Java; oil palm and rubber in Sumatra and Kalimantan Provinces; Papua and West Papua Provinces; dryland agriculture in Nusa Tenggara Timor Province; artisanal fisheries	McCarthy et al. (2020)
Timor-Leste	Anthropology	Researchers (agricultural experts, social scientists), agri-program advisors, government policy advisors, regional government staff	16 (8/8)	Donor reports, research papers on demographics, food and health, agricultural systems	National food system	McWilliam (2020)
Papua New Guinea	Agronomy, crop physiology, human geography	Agricultural researchers, donors, NGOs, national government (30), plus personal stories from individuals (27)	57 (33/24)	Donor reports, newspapers, on-line blogs, research papers, grey literature, email correspondence with others than those interviewed	National food system	Bourke (2020)
Large islands: Fiji, Solomon Islands, Vanuatu	Human ecology, sustainability science	Farmer organisations (NGOs, private sector), regional experts, locally based consultants,	10 (3/7)	Donor reports, regional organisation reports, research reports, national government databases, regional databases, media reports	National food systems	Davila and Wilkes (2020)
Medium islands: Samoa, Tonga			5 (2/3)
Small islands: Tuvalu, Kiribati			2 (0/2)
Regional agencies and donors			4 (3/1)

Due to travel restrictions the teams were limited to working remotely, and hence our research method relied on interviewing key informants with relevant knowledge and expertise in the food systems concerned, plus secondary data. Such data were varied in their qualitative and quantitative nature and reliability, making triangulation important. The in-country key informants were drawn from the teams’ networks of researchers, development practitioners and government officials. Such varied domains of expertise tend to generate ‘knowledge cultures’ with different interpretations of the same reality (Brown, 2008; Bohensky et al., 2016). While diverse perspectives and knowledge are necessary to understand complex problems, they must also be combined if transformational action is to be identified and catalysed (Brown and Lambert, 2015). Hence our framework and approach had to accommodate and integrate a plurality of views, knowledge, languages and data sources.

2.2. Diversity in food systems and COVID-19 effects

The degree of cultural, political and geographical diversity amongst the Indo-Pacific focal countries is marked, ranging from remote PICs with small populations (e.g. Tuvalu) to the large nations of the Philippines and Indonesia (Table 1). Even within these countries there is widespread diversity due to cultural, agro-ecological and climatic variability, and hence livelihoods and food systems (Butler et al., 2020; Farrell et al., 2020). Also, the relative exposure of remote countries such as the PICs to COVID-19 has been less than for more populous and well-connected nations such as Indonesia and the Philippines (Food and Agriculture Organisation of the United Nations (FAO, 2020). Consequently, the levels of COVID-19 infection and government policy responses have varied (Table 1). Our analytical framework therefore had to be sufficiently generic to capture this diversity in food systems and COVID-19 experiences.

2.3. Food systems as complex systems

The terms of reference given by ACIAR was to conduct a targeted analysis framed beyond food production and health which considered food systems as a whole. We defined a food system as “all the elements (environment, people, inputs, processes, infrastructures, institutions, etc.) and activities that relate to the production, processing, distribution, preparation and consumption of food, and the output of these activities, including socioeconomic and environmental outcomes” (HLPE, 2020, p. 11).

Central to this definition are complex linkages and dynamic feedbacks between components of a food system, in which socio-economic (e.g. consumer demand, market prices, government policies) and biophysical factors (e.g. climate, soil fertility, land use change) interact to influence food system outcomes, which in turn affect drivers external to the system (Ericksen et al., 2009; Doherty et al., 2019). Non-linear dynamics caused by feedbacks within a system can generate unexpected outcomes or ‘emergent properties’ (Gallopin, 1991). The ‘resilience’ of the system is determined by its ability to cope with disturbance or change and retain its fundamental function and structure, and its capacity to self-organise, learn and adapt (Walker et al., 2004; Doherty et al., 2019). For smallholder livelihoods in the developing world, these attributes can be intentionally supported and invested in (Marschke and Berkes, 2006) and should be the focus of food system interventions (Béné, 2020). However, purposeful non-incremental change (or ‘transformation’) may be required if a food system is trapped in an undesirable state (Walker et al., 2010).

This complex system framing presented a potential challenge for the rapid assessment because systems approaches have been criticised for oversimplifying or excluding social dimensions, especially those relating to power, structure, social diversity and equity (Béné et al., 2014; Fabinyi et al., 2014; Stone-Jovicich, 2015). In analysing food systems, this can lead to a lack of historical contextualisation, limited understanding of the plurality of interests and trade-offs between them, and the political and institutional processes that generate poverty (Foran et al., 2014; Leach et al., 2020). These elements are central to effective development, where there is a normative emphasis on identifying and enhancing the agency of marginalised groups, which is not necessarily the intent of social-ecological resilience thinking (Béné et al., 2014), but is critical when addressing pro-poor issues such as food and nutrition security and underlying drivers of vulnerability (Béné et al., 2019; Béné, 2020; Gillespie, 2020). In addition, experience of using systems approaches as participatory tools suggests that the concepts are not easily translated in cross-cultural settings and are also time and resource-intensive (Béné et al., 2011; Butler et al., 2017a).

3. Rapid assessment analytical framework and method

3.1. Analytical framework

We first reviewed analytical frameworks that might meet our design parameters, particularly the need for a ‘food systems as complex systems’ approach. Fraser et al. (2005) discussed the provision of food as subject to multiple systems characteristics, including cross-scale connectivity, and demonstrated their resultant vulnerability to global shocks, but did not offer an analytical framework to diagnose such vulnerability. Ericksen (2008) also proposed the analysis of food systems as complex social-ecological systems and went further by suggesting a step-based process for examining components of their vulnerability. However, this framing did not enable the consideration of responses to shocks. Darnhofer et al. (2010) examined the sustainability and resilience of a farm unit within a nested social-ecological system, but did not provide an analytical framework to assess the systems’ vulnerability to shocks. Finally, Tendall et al. (2015) examined the concept of food system resilience in terms of external and internal disturbances, and suggested a ‘resilience action cycle’ to identify preventative and reactive interventions to enhance resilience. While Tendall et al. (2015) advocated for the involvement of stakeholders and their knowledge in the assessment process, they did not offer a process with which to operationalise their conceptual framework.

The lack of examples that linked food systems responses with external shocks and a step-wise analytical process led us to adopt Allen and Prosperi’s (2016) approach which models a food system’s sustainability and resilience to global environmental and socio-economic drivers and shocks. The food system is geographically specified at the national or sub-national level, with a set of intrinsic endogenous features (exposure, sensitivity, recovery potential and resilience) which determine outcomes in terms of food and nutrition security. The system is impacted by exogenous variables or drivers of change, emanating from the broader regional or global scale. It is assumed to be a ‘driver-taker’, although there are feedbacks from food system outcomes to these higher-scale drivers. Most importantly, Allen and Prosperi outline a four-step process to operationalise the analysis: Step 1 defining the scale of analysis, Step 2 identifying drivers of change, Step 3 identifying food system outcomes and Step 4 examining exposure, sensitivity, impacts and recovery potential. The results inform detailed follow-up analyses of key issues or emergent properties.

This analytical framework potentially suited our design parameters for five reasons. First, our national or sub-national level of analysis established a geographically bounded food system. Second, COVID-19 represented a clear exogenous, global shock to the system. Third, the approach enabled the investigation of COVID-19′s coincidental interactions with other drivers and shocks (e.g. climate disasters, pest incursions) that created multi-hazards for the system. Fourth, the stepped process provided a clear and logical line of inquiry, and a structure for reporting. Finally, the assessment of recovery potential leant itself to identifying priority interventions that would enhance future resilience.

We modified Allen and Prosperi’s (2016) framework by expanding their four steps into 10 (Fig. 1 , Table 3 ). To incorporate a development lens, we included explicit identification of pro-poor food system outcomes in Step 4, and analysis of vulnerable groups in Steps 6, 7 and 8. In addition, we identified opportunities for transformation in Step 9, which we defined after Colloff et al. (2017) as generally irreversible and fundamentally changed structures and functions of a food system, including norms, goals, values, rules and practices. This step also allowed the assessment teams to screen suggested interventions for potentially maladaptive strategies, which we defined as actions which may increase vulnerability to future change over time, creating path-dependency and foreclosing future options (after Barnett and O’Neill, 2010; Wise et al., 2014).

Fig. 1.

The analytical framework and the 10 rapid assessment steps. Pro-poor food system outcomes are examples only.

Table 3.

The 10 steps and questions used to apply the analytical framework (see Fig. 1), and guiding notes.

Step	Question	Guiding notes
1	What is the system of interest? What are its boundaries?	Describe the food system, either at national or sub-national scales if the national scale is too coarse to capture important socio-economic, cultural and agro-ecological diversity
2	What are the characteristics of COVID-19 and the local response?	Describe the nature of the COVID-19 shock, including the date and mode of entry into the country, its spread and the policy response to the outbreak
3	What are the other drivers of change and their interactions with COVID-19?	Identify other global and/or regional drivers of change that are occurring simultaneously and their interactions with the COVID-19 shock to generate multi-hazard effects. These drivers could be immediate and proximate shocks (e.g. cyclones, pest incursions) or incremental (e.g. sea level rise)
4	What are the desired food system outcomes?	Identify the desired pro-poor food system outcomes. These are probably food and nutrition security, but there could be other specific national or sub-regional policy targets and indicators (e.g. Sustainable Development Goals 1 No Poverty, 2 Zero Hunger, 5 Gender Equality and 13 Climate Action), or associated stunting and non-communicable disease or climate action plans
5	How exposed is the food system to COVID-19 and other drivers?	Exposure is the first point of contact between the shock and the food system. Following the IPCC (2012), exposure is defined as the elements of the system that are susceptible to adverse effects from the exogenous environmental or socio-political stress or shock. This step should consider components of the food system that are exposed to COVID-19, both directly and indirectly, and compounding global or regional drivers or shocks identified in Step 3 which create multi-hazards
6	What are the sensitivities of and impacts on the food system?	Sensitivity refers to the potential magnitude of the consequences of exposure to shocks and drivers, and hence impact on the food system (Prosperi et al., 2014). This step examines the sensitivities of and impacts caused by its exposure to COVID-19, and any interactions with other shocks or drivers identified in Step 5. Impacts could be immediate, medium- and long-term. Sensitivities and impacts should be disaggregated to identify vulnerable social groups or household types, defined as people’s characteristics and their social, political, economic and environmental context which renders them susceptible to shocks (Kelly and Adger, 2000)
7	What is the current recovery potential of the system?	This step assesses the current potential of the system to respond to and absorb the disturbances in order to continue to function Allen and Prosperi (2016). Since recovery potential may differ amongst social groups or household types, this analysis should be disaggregated. We add a feedback from food system outcomes to recovery potential, because this relationship will be dynamic
8	How resilient is the food system? Can the desired system outcomes be achieved?	Resilience is the ability of the system to cope with disturbance or change and retain its fundamental function and structure, and its capacity to self-organise, learn and adapt (Walker et al., 2004). In this framework, resilience is driven by recovery potential. This analysis should be disaggregated to highlight key issues or groups with major challenges emanating from the shock. This step should also consider whether the desired food system outcomes can be achieved as a result of the previous steps’ results
9	What responses are needed to boost recovery potential?	This is the primary output of the analysis and identifies responses that will bolster recovery potential to COVID-19 and future shocks or drivers of change. It is informed by the impacts and recovery potential that different social groups exhibit (from Steps 6 and 7), and by food system outcomes (from Step 8) which influence options. Timeframes for responses can be categorised as short term (< 1 year), intermediate (up to 5 years) or long-term (up to 10 years). Transformational actions could be identified, and suggested interventions should be subsequently screened for potentially maladaptive responses
10	What are the impacts on regional/global drivers of change?	Allen and Prosperi (2016) consider that the potential economic, social and biophysical feedbacks from the food system to the global or regional drivers and shocks are secondary, since the food system is typically a ‘driver-taker’. However, this step should consider if there are food system outcomes that could influence regional drivers (e.g. refugee emigration to other countries, or political unrest influencing geo-politics)

3.2. Focal country assessments

All aspects of focal country assessments were overseen by a Reference Committee comprising ACIAR, the Australian Department of Foreign Affairs and Trade, and non-assessment team scientists from the research partner institutions (the Australian National University and CSIRO), and informed by ACIAR Country Managers (Robins et al., 2020). Step 1 (identifying the system of interest) was undertaken by the assessment teams following discussion with the Reference Committee about key food systems, geographies or commodities of interest. Step 2 (describing the COVID-19 experience) was derived by the assessment teams from secondary data and information. Having given their free, prior and informed consent, key informants were then presented with open questions framed around Steps 3–9, allowing them to report any information they considered relevant (Hennink et al., 2011). Assessment teams recorded their answers digitally and then transcribed the interviews.

Using triangulation, common themes and issues were distilled from the interviews and summarised into a standardised report format that followed the steps of the analytical framework. Step 10 was completed subsequently by a cross-country synthesis team (Alders et al., 2020). Each report concluded with recommended interventions to build recovery potential, categorised as short term (< 1 year), intermediate (up to 5 years) or long-term (up to 10 years). Also, follow-up analyses on emergent issues requiring further research were suggested. A peer review team was established to provide independent screening of draft country reports and their recommendations, which also acted to counter potential disciplinary bias amongst the assessment teams and key informants.

3.3. Reflections on applying the framework

The assessment teams reviewed their experiences of applying the framework through a weekly reflection discussion in June-July 2020, plus the peer-review of each draft country report and the cross-country synthesis in July 2020. The three design parameters which delimited our assessment (see Section 2) were examined in detail in terms of their challenges, strengths and weaknesses, namely:

• 1
  Integration of diverse data and knowledge:

  • - Ability to integrate different sources of data and knowledge

• 2
  Diversity in food systems and COVID-19 effects:

  • - Flexibility for application to different countries and food systems

• 3
  Food systems as complex systems:

  • - Transferability of exposure, sensitivity, recovery potential and resilience concepts to in-country key informants
  • - Consideration of power, structure, social diversity and equity dimensions
  • - Identification of transformational interventions
  • - Identification of maladaptive interventions

4. Results and discussion

4.1. Focal country assessments

In the Philippines, Indonesia, Timor-Leste and PNG the total numbers of key informants engaged ranged between 16 and 57 (Table 2). However, in the PICs the numbers of informants were limited due to the difficulty of contacting many in-country experts during the pandemic. Consequently, 10 interviews were conducted for the large countries of Fiji, Solomon Islands and Vanuatu and only two for the small countries of Kiribati and Tuvalu. Four experts from regional agencies or donors ameliorated this shortcoming, but still the numbers were relatively low. Overall, 136 key informants were interviewed; 63 (46 %) were women and 73 (54 %) were men.

The country assessments differed in their application of Step 1 (Table 2). In the Philippines the agreed focus was the production and value chains for four key commodities in the National Capital Region. In Indonesia a mix of priority commodities and/or provincial food systems were assessed. In Timor-Leste and PNG the national food system was analysed, while the PICs were assessed nationally but within regional clusters according to their size and geography. A total of 50 interventions were identified, of which only 7 (14 %) were potentially transformational; the remainder were tactical and incremental.

4.2. Reflections on applying the framework

The rapid assessment method effectively integrated different sources of data and knowledge. In all cases the preliminary scanning of secondary information on COVID-19 and food systems (e.g. government databases, newspaper articles and blogs) enabled assessment teams to remotely scope issues in order to guide selection and engagement of key informants. In the Philippines data were effectively triangulated with agri-business practitioners, farmer associations and government officials’ perspectives about value chain impacts and responses (Table 4 ). However, all teams struggled to distil large amounts of patchy information in such a short timeframe, and because of the crisis footing in all countries key informants were often difficult to contact. This was particularly evident in the PICs, and consequently some perspectives may have been under-represented for these countries. Although overall women represented almost 50 % of key informants, they were poorly represented in Indonesia (seven of 25) and the PICs (seven of 21), and hence issues relating to women and girls may not have been adequately considered in these countries.

Table 4.

Reflections on the analytical framework, relative to the three design parameters and challenges.

Theme	Challenge	Strengths	Weaknesses
1. Integration of diverse data and knowledge	Ability to integrate different sources of data and knowledge	In the Philippines the focus on agri-business practitioners, farmer associations and government officials enabled triangulation about value chain impacts and responses. In PNG key informants linked the assessment to their contacts who generated personal stories. In all cases secondary data were effectively used to first scope issues and then triangulate these with key informants’ subsequent perspectives	In the PICs it was challenging to integrate issues and responses across scales (e.g. farm and Pacific regional policy). In Indonesia and the Philippines it was difficult to integrate data and knowledge into a national-level assessment due to acute political, cultural and geographical diversity
2. Diversity in food systems and COVID-19 effects	Flexibility for application to different countries and food systems	In the PICs the framework was easily transferable across three groups of countries with similar geographies. In the Philippines the analysis successfully assessed impacts on four commodities in one region. In Indonesia, the complex national food system was broken down into commodities and/or provincial-scale systems
3. Food systems as complex systems	Transferability of exposure, sensitivity, recovery potential and resilience concepts to in-country key informants	In the PICs and the Philippines, the concepts of shocks, exposure, impacts and resilience are well known and understood following regular responses to climatic and natural disasters	In Indonesia, PNG and Timor-Leste the questions did not translate into local languages easily. Hence less technical terms were necessary, and assessment teams had to subsequently interpret responses relative to the concepts
	Consideration of power, structure, social diversity and equity dimensions	In Indonesia it was possible to identify the most vulnerable groups (e.g. fishing households, landless labourers). In the Philippines representatives of a women’s group were interviewed who were vulnerable due to their involvement in food retailing. In PNG women were also identified as highly vulnerable due to loss of cash income, exposure to police violence, and reduced access to health and the justice system. In the PICs and Timor-Leste the opportunity to include disenfranchised youth in future food production was identified	In the PICs there was some discussion of the impact on women’s labour burden resulting from increased local agricultural dependence, but only following prompts, because their roles are not commonly recognised
	Identification of transformational interventions	Potentially transformational interventions were suggested for PNG (e.g. increased crop diversity, improved teaching and research in tertiary agricultural education), the Philippines (e.g. increased smallholder flexibility through multi-cropping, introduction of risk-mitigation mechanisms such as insurance and guarantees) and Indonesia (e.g. social protection to address entitlement failures, e-platforms to shorten value chains)	In the PICs the primary recommendations were for short term and incremental responses, driven by pressure to respond to COVID-19
	Identification of maladaptive interventions	In Indonesia the suggested shortening of fragmented value chains using e-platforms could have unintended consequences by excluding some household types. In the PICs there remains a focus on exporting cash commodities into exposed global markets. In Timor-Leste social protection is being funded by the national Petroleum Fund, possibly creating a perverse incentive for smallholders to reduce production

The framework was also found to be sufficiently flexible and generic to be applied across the varied assessment countries’ food systems. For example, it was feasible to analyse seven PICs and their diverse geographies, COVID-19 experiences and national food systems, and also to consider at the sub-national scale both specific commodities (e.g. rice and oil palm) and/or provincial (e.g. Papua and Nusa Tenggara Timor) systems in Indonesia, and four commodities in the Philippines (Table 4). Despite the focus on individual commodity-centred sub-national food systems in these latter countries, the systems approach still provided a heuristic for exploring exposures and vulnerabilities inherent in each commodity’s supply and value chain. Food system resilience assessments can have various entry points, depending on stakeholders’ priorities, but commodity or value chain analyses may overlook some social aspects of the system (Tendall et al., 2015). Instead, a consumer-led approach may have advantages in examining important elements of culture and diet (Dixon, 1999), and this may be an interesting future avenue with which to test the framework.

The framework was also useful for analysing the conflation of shocks, for example in Vanuatu where COVID-19 lockdowns coincided with Tropical Cyclone Harold, exacerbating food system impacts. In addition, the assessment teams found the sequential step-wise implementation of the framework to be simple and logical, forming a structured process of inquiry and reporting, while also allowing iterative re-visiting of the steps as issues and themes emerged during the analyses. This simplicity may lend the framework to more applied contexts, for example in community-based participatory research where systems approaches can be overly complex and resource-intensive (Béné et al. 2011, Butler et al., 2017a).

The transferability of the concepts of exposure, sensitivity, recovery potential and resilience to in-country key informants was mixed (Table 4). In the PICs and the Philippines, these terms were well-understood due to their regular use in policy and donor dialogue following frequent natural disasters. However, in Indonesia, PNG and Timor-Leste the questions did not translate into local languages easily. Hence less technical explanations of the concepts were necessary, and the assessment teams had to later interpret informants’ discussions about COVID-19 issues and impacts relative to the terms. It is possible that the transferability of the concepts was dependent upon the characteristics of key informants engaged, and that perhaps scientifically-trained informants would have been more familiar with the terms. However, this did not appear to be the case, since in the PICs and the Philippines no researchers were interviewed (Table 2), and the terms were understood (Table 4).

Our adapted framework sought to include a normative focus on pro-poor poverty alleviation, which is often overlooked by systems- and resilience-based approaches, but there were mixed outcomes from this analysis with respect to identifying vulnerable groups and household types. These were readily identified in Indonesia (fishing households, landless labourers), PNG (women’s loss of cash income and exposure to police violence) and the Philippines (women food retailers), PICs and Timor-Leste (disenfranchised youth). However, it was less easy in the PICs to discuss women’s vulnerabilities because their roles in agriculture and food systems were not generally recognised, perhaps due to the lack of women’s representatives amongst the key informants (see Table 2). This suggests that key informant selection is important if comprehensive perspectives on gender and other social stratifications are to be captured. In addition, the institutional, political and economic causes of exposure and vulnerability, the role that they may also play in recovery potential, plus potential trade-offs amongst winners and losers inherent in suggested responses were not fully addressed by our analyses.

As well as short term incremental interventions, some potentially transformative interventions were suggested by key informants in PNG (e.g. increased crop diversity, improved teaching and research in tertiary agricultural education), the Philippines (e.g. increased smallholder flexibility through multi-cropping, risk-mitigation mechanisms such as insurance and guarantees), and Indonesia (e.g. social protection to address entitlement failures, e-platforms to shorten value chains). This was not the case in the PICs where only incremental responses were suggested (Table 4), perhaps due to the small numbers of key informants engaged. Notably, several of the potentially transformational interventions were institutional rather than agricultural production-orientated and may not have been identified without a complex systems approach. Because shocks such as pandemics may offer windows of opportunity for transformation, it is important that such interventions can be identified and promoted, particularly if they address systemic institutional and political issues that often underpin vulnerability (Pelling, 2011; Rodima-Taylor et al., 2012). Pin-pointing and acting upon these leverage points in food systems may yield significant returns on research and donors’ investments.

Similarly, the identification of potentially maladaptive interventions is important if perverse outcomes are to be avoided. In our analysis the suggested shortening of value chains in Indonesia may exclude some households, and social protection funded by the Timor-Leste Petroleum Fund may encourage smallholders to reduce food production, thereby exacerbating food insecurity (Table 4). However, the definition and screening of proposed interventions as transformational or maladaptive was dependent on the assessment teams’ perspectives and their own knowledge cultures. Instead, this analysis in Step 9 could more explicitly encourage key informants to consider transformational and maladaptive aspects of their recommended interventions, while remaining cognisant that their personal or political interests may also influence their judgments.

There were two more general challenges associated with implementing the framework, both conceptual and methodological. The first was the problem of aggregating recommended interventions across sub-national food systems to the national scale, which diluted necessary detail specific to certain food systems. This was particularly marked in PNG, Indonesia and the Philippines where there was a diversity of social, cultural and geographical contexts (Table 4). Related to this was the difficulty of conceptualising and then linking proposed responses within and across scales of food systems, largely due to the limited time available for the assessment. However, it is recognised that to achieve change in complex systems, and transformational change in particular, a suite of connected interventions is necessary at different scales, which requires an understanding of their possible linkages, feedbacks and political feasibility. This is true whether taking a purely social-ecological systems perspective (e.g. Walker et al., 2004; Westley et al., 2011), a social innovation (e.g. Seelos and Mair, 2018) or a development and pro-poor approach (e.g. Butler et al., 2014, 2017b).

Based on our reflections we have further refined the analytical framework (Fig. 2 ). Primarily we emphasise the examination of political, economic and institutional aspects of exposure and vulnerability, and the constraints that these may present to recovery potential. If clearly understood, it may be possible to place greater focus on transformational interventions that could address these systemic causes of vulnerability by shifting norms, goals, values, rules and practices. In addition, responses should consider potential trade-offs amongst winners and losers, as suggested by Ericksen (2008), but with a specific consideration of vulnerable groups, and also seek cross-scale linkages and coordination amongst the responses. Notably, some of these responses may simultaneously address causes of exposure and sensitivity, particularly in terms of institutional and political factors. While these normative additions alter the social-ecological systems focus of Allen and Prosperi’s (2016) original framework, we consider that they are conceptually compatible and can add value to the analyses of livelihoods, poverty and food systems in vulnerable regions (Foran et al., 2014; Butler et al., 2017b).

Fig. 2.

The refined analytical framework showing additional elements in italics. Pro-poor food system outcomes are examples only.

Ultimately, however, the utility of the framework and our approach is dependent upon the time and resources available for a rapid assessment, and the selection and knowledge of assessment teams, key informants and secondary data which shape the outputs. Nonetheless, despite these limitations in our assessment, the framework has proved sufficiently robust and flexible to yield an initial identification of food system-specific priorities for ACIAR and other donor support, and as a platform to conduct more detailed analyses on key topics and potentially transformational leverage points. We hope that our approach can be further tested in other COVID-19 contexts, and its flexibility may also lend itself to examining inevitable future food system shocks, varying in characteristics from further pandemics, to climatic, natural disaster, political and economic perturbations. It may also be a useful tool for the longitudinal and iterative monitoring of post-shock changes in food systems, particularly following interventions, and thus identify emergent properties and outcomes at different levels within food systems which require renewed attention by stakeholders.

CRediT authorship contribution statement

James R.A. Butler: Conceptualization, Methodology, Visualization, Writing - original draft, Writing - review & editing. Federico Davila: Conceptualization, Methodology, Visualization, Writing - review & editing. Robyn Alders: Methodology, Writing - review & editing, Visualization. R. Michael Bourke: Methodology, Writing - review & editing, Visualization. Steve Crimp: Methodology, Writing - review & editing, Visualization. John McCarthy: Methodology, Writing - review & editing, Visualization. Andrew McWilliam: Methodology, Writing - review & editing. Anton S.M. Palo: Methodology, Writing - review & editing. Lisa Robins: Funding acquisition, Supervision, Project administration, Methodology, Writing - review & editing. Michael J. Webb: Conceptualization, Methodology, Visualization, Writing - review & editing. Monica van Wensveen: Funding acquisition, Supervision, Project administration, Methodology. Todd Sanderson: Resources, Funding acquisition, Supervision. Daniel Walker: Resources, Funding acquisition, Supervision.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.