WICID framework version 1.0: criteria and considerations to guide evidence-informed decision-making on non-pharmacological interventions targeting COVID-19

Abstract

Introduction

Public health decision-making requires the balancing of numerous, often conflicting factors. However, participatory, evidence-informed decision-making processes to identify and weigh these factors are often not possible- especially, in the context of the SARS-CoV-2 pandemic. While evidence-to-decision frameworks are not able or intended to replace stakeholder participation, they can serve as a tool to approach relevancy and comprehensiveness of the criteria considered.

Objective

To develop a decision-making framework adapted to the challenges of decision-making on non-pharmacological interventions to contain the global SARS-CoV-2 pandemic.

Methods

We employed the ‘best fit’ framework synthesis technique and used the WHO-INTEGRATE framework as a starting point. First, we adapted the framework through brainstorming exercises and application to case studies. Next, we conducted a content analysis of comprehensive strategy documents intended to guide policymakers on the phasing out of applied lockdown measures in Germany. Based on factors and criteria identified in this process, we developed the WICID (WHO-INTEGRATE COVID-19) framework version 1.0.

Results

Twelve comprehensive strategy documents were analysed. The revised framework consists of 11+1 criteria, supported by 48 aspects, and embraces a complex systems perspective. The criteria cover implications for the health of individuals and populations due to and beyond COVID-19, infringement on liberties and fundamental human rights, acceptability and equity considerations, societal, environmental and economic implications, as well as implementation, resource and feasibility considerations.

Discussion

The proposed framework will be expanded through a comprehensive document analysis focusing on key stakeholder groups across the society. The WICID framework can be a tool to support comprehensive evidence-informed decision-making processes.

Key questions.

What is already known?

• Ad hoc decision-making on matters of public health and health policy, such as non-pharmaceutical interventions to contain the global SARS-CoV-2 pandemic, requires decision-makers to balance numerous and often conflicting factors.

• Insufficient consideration of relevant factors reduces acceptance and can limit the effectiveness of the intervention.

What are the new findings?

• Based on a content analysis of comprehensive strategy documents, we newly developed WICID (WHO-INTEGRATE COVID-19) framework that provides 11+1 criteria informed by 47 aspects, which are intended to support decision-makers in the balancing act of identifying and considering criteria of relevance.

What do the new findings imply?

• The usage of the WICID evidence-to-decision framework can support decision-makers and expert committees in making more balanced decision, even if not all voices of relevant stakeholders could be included in the process due to time constraints imposed by the rapid progress of the pandemic.

Introduction

The response to the SARS-CoV-2 pandemic highlights the challenges of inherent evidence informed of public health and health policy decision-making.1–4 These include among others: decision-making under time constraints, under uncertainty due to limited evidence, balancing numerous tradeoffs and the challenge of ensuring fair decision-making processes under such circumstances.

Due to exponential growth in the number of infections, the issue of timing is crucial in a pandemic. A delay on the implementation of public health interventions (eg, physical-distancing regulations) by days can have grave consequences.5 Therefore, to receive timely evidence-informed guidance, many governmental institutions set up expert committees to support public health and health policy decision-makers.6 One challenge these expert groups faced is the qualitatively poor scientific evidence, with often questionable transferability and applicability to the context of decision-making and the lack of reliable evidence.7 8 Not only due to the novelty of the pathogen, but also as assessing the effects of public health and health policy which often do not arise directly from the intervention, but from the system reacting to the introduction of the intervention within it9 (eg, school closures can lead to parents reducing working hours, which can lead to shortages in staff in the healthcare sector, limiting its ability to provide medical services and care).

However, even if strong scientific evidence was available, this in itself is insufficient to make sound recommendations, as evidence-informed decision-making is a deeply value-laden and often politised process.10–12 Decision-makers must balance numerous and often conflicting normative and technical factors to come to a promising and acceptable decision.13–15 This leads to the question: which criteria should be considered and how these should be weighed against each other?

The extent to which decisions are considered acceptable and legitimate depends on how the decisions were made (procedural considerations). Key considerations include, for example, transparency, inclusion of relevant stakeholders and an appropriate composition of the panel.10 16–21 Such approaches can increase the acceptability and perceived legitimacy of a decision17 22 23 even if—given varying and sometimes contradictory interests—no consensus regarding the right selection and weighing of criteria can be achieved.17 According to the Accountability for Reasonableness framework,20 a key condition is ‘relevance’: the decision or recommendation must rest on evidence, reasons and principles that all fair-minded parties can agree to be relevant, and must meet the diverse needs of affected stakeholders.20

Involvement of representatives of all relevant stakeholder groups to identifying reasons and principles for a given decision-making process is considered ideal.10 16 24 25 However, this ideal is often difficult if not impossible to meet in deliberative processes (eg, guideline development) under ‘normal’ circumstances due to logistical or resource constraints; which is further exacerbated by under the time constraints imposed by the rapid progression of the SARS-CoV-2 pandemic. While not intended nor able to replace stakeholder participation, Evidence-to-Decision (EtD) frameworks are a way to support this balancing act.26 27 EtD frameworks, which tend to comprise criteria and procedural guidance, are intended to ensure that all relevant factors are considered and the underlying rationale is made transparent.28 When developed and applied well, these frameworks can help identify and integrate the criteria of relevance, even if not all voices of all stakeholders could be heard. EtD frameworks may be of particular value in the current SARS-CoV-2 crisis, among others due to very restricted time frame, pronounced uncertainties and the rapidly evolving evidence base. Furthermore, the decision-making process on non-pharmacological interventions (NPIs) addressing SARS-CoV-2 is very fragmented—with measures implemented by a plurality of stakeholders in various settings affecting a broad range of outcomes and populations differently. Here, an EtD framework could be a valuable tool to bring together different perspectives so that they can be systematically researched, discussed and prioritised.

One of these frameworks is the WHO-INTEGRATE framework version 1.0.29 It was developed in a research project commissioned by the WHO to support guideline development on complex public health and health system interventions.30 Based on a conceptual and normative foundation29 - primarily based on WHO norms and values derived and public health ethics frameworks31–38 - it was developed based on a comprehensive literature review of real-world decision criteria,39 an assessment of complexity features,29 as well as qualitative research across four continents.40 As with most EtDs, the WHO-INTEGRATE framework is generic framework and requires adaption to the specific intervention and context.

The aim of this research project is to adapt the WHO-INTEGRATE framework to support the development of recommendations and decision-making for NPIs to address the global SARS-CoV-2 pandemic. For this, the framework addresses the national and subnational level and takes a plurality of viewpoints of affected stakeholders into account. The WICID framework (WHO-INTEGRATE COVID-19) adaption is intended to reflect decision-making challenges and opportunities on matters of public health in relation to COVID-19 by embracing a complex systems perspective. However, we aim to ensure that the tool is sufficiently generic to be applicable to a wide range of NPIs, contexts and decision topics.

Although procedural criteria, norms, principles and processual considerations are crucial for achieving fair processes, this research project focuses on the substantive decision-making criteria.

Methods

The development of the WICID framework is conducted in three phases, following an approach analogous to the ‘best fit’ framework synthesis and using an adapted version of the WHO-INTEGRATE framework as a starting point.41 An extended version of the methods used in this research project is provided as online supplemental file 1.

In phase I, we adapted the WHO-INTEGRATE framework through brainstorming exercises and applying it to case studies, in order to develop an analytical, generic tool (a priori framework). This was done through (1) discussion within the research team, (2) assessment of real-world decision-making criteria derived from a comprehensive overview of reviews39 and (3) conducting a brainstorming exercise guided by the application of the framework on two case studies (reopening high schools and reopening small businesses such as book shops). This preliminary a priori framework was then imported into the software MAXQDA V.20 (VERBI GmbH, Berlin, Germany), with the criteria and subcriteria of the a priori framework being translated into codes of the coding frame to be used in phase II.

Phase II consisted of a content analysis of a purposive sample of comprehensive strategy papers on lifting the lockdown measures in Germany (eg, reopening schools, increased testing measures), which were coded against the a priori framework developed in phase I following the technique of ‘best fit’ framework synthesis.41–43 ‘Best fit’ framework synthesis begins by creating or using a thematic or conceptual framework of a priori themes and coding data extracted from documents (in this case: the comprehensive strategy documents) against it. A new framework is created by performing a thematic analysis on any data that cannot be accommodated within the a priori framework.41

We assumed that selecting comprehensive strategy documents by expert commissions or expert groups would provide a broad, multi-perspective set of recommendations (in contrast to, for example, scientific publications or statements by individual groups, which do not reflect a broad range of relevant perspectives in concluding). We defined these as documents (a) intended to provide a comprehensive strategy or strategic suggestions for phasing out the lockdown measures (rather than providing information or pointing out individual aspects), (b) not exclusively or primarily focused on mitigating the health-related consequences of the SARS-CoV-2 pandemic but also including other societal, economic or health outcomes, (c) addressing various NPIs and their interplay, and (d) focusing on multiple considerations (>1) to be reflected in this process. Position papers of stakeholder groups reacting or positioning themselves to a document, measure or event without providing comprehensive strategy guidance were excluded (n=8) but will be considered in the upcoming phase III, where the current version of the WICID framework will be expanded (see "Advancing the WICID framework version 1.0 in phase III"). The eligibility criteria and the rationale for their selection are provided as a supplement in the expanded Methods section.

The search was conducted through multiple pathways: two researchers (JMS, LA) independently conducted (1) grey literature searches in the search engine Google and (2) on the websites of major newspaper outlets in Germany; (3) one researcher (JMS) searched the websites of the 16 German federal states, the national government and selected national government ministries, (4) we submitted freedom of information requests to the federal states’ governments, the national government, as well as selected national government ministries, and (5) contacted a sample of experts involved in public health decision-making or expert groups to provide us with strategy documents. As the documents are considered grey literature and mostly written in in German, we did not conduct a literature search in scientific databases for these types of documents at this point of time.

The coding process of the included strategy documents was conducted by two authors (JMS, LA). Following the coding of one sample strategy documents to assess the need for adaption of the coding frame, one author (JMS) used the software MAXQDA V.20 to code all identified strategy papers followed by a critical review by a second author (LA). The researcher applied the level 1 codes (referring to the criteria in the framework) and level 2 codes (referring to the aspects in the framework) of the coding frame to passages in the strategy document making references to criteria, considerations or values covered within the codes. When the content of such a passage was perceived as not adequately covered by the coding frame, new level 2 or level 1 codes were created. Unclear passages were assigned the code TBD code for later review. The coding frame is provided as online supplemental file 2.

After coding all selected strategy documents, two authors (JMS, LA) critically reflected on content saturation and dimensions of the framework insufficiently covered within the strategy documents and it was concluded that content saturation was reached.

Based on the coded passages, the researchers adapted the a priori framework by assessing the need to newly create, adapt, merge, separate or reword criteria (reflecting level 1 codes) and aspects (reflecting level 2 codes).

In a final step, two researchers (JMS, LA) went through each of the coded passages to critically reflect on whether the criteria, considerations or values contained within these passages were adequately covered within the newly adapted phase II framework.

Results

Development of preliminary adaption of the framework and a coding frame (phase I)

Following the process outlined in the Methods section, we developed in phase I a preliminary, adapted (a priori) framework, which consisted of 8 criteria comprising 36 aspects and the metacriterion quality of evidence (a criterion to be applied across the other substantive criteria). This framework is provided as online supplemental file 2. A preliminary version of this phase I framework, as well as the exemplary case studies, is part of a strategy document developed by two of the authors, among other experts, for the Federal Ministry of Interior and the Federal Chancellery.44

Documents identified and included for analysis for phase II

We identified 12 comprehensive strategy papers in total. Six strategy documents were publicly available: four strategy papers, developed by expert groups for federal states, the national government, or ministries of the national government, and two strategy documents developed by the corona expert commission of the federal state government of North Rhine-Westphalia.45 Two strategy documents were developed by an informal expert group for the German Federal Ministry of the Interior, one of which was publicly available through a press report; the other was provided to us through the German Network Public Health COVID-19 (Kompetenznetz Public Health on COVID-19).44 46

We identified eight additional comprehensive strategy documents, which were not directly commissioned by governmental institutions. Four of which were published by the National Academy of Sciences Leopoldina,47–49 one by a political party in the federal city-state of Hamburg,50 one by a research institute commissioned by the political foundation Hans-Böckler-Stiftung,51 one by the Boston Consulting Group,52 and one developed by researchers from diverse institutions under the coordination of two researchers with affiliation at the University of Wuerzburg and the IFO Institute.53

The adapted WICID framework

The adapted WICID framework consists of 11 substantive decision-making criteria, containing 48 decision-making aspects, and the meta-criterion quality of evidence, to be applicable across all criteria and aspects (outer circle, figure 1; tables 1 and 2). Depending on the intervention, the criteria and aspects are intended to be applied on and reflected for different population groups (center-most circle, figure 1). Depending on the measure and type of decision-making process, the decision-makers are intended to deliberate on the criteria and aspects taking one or multiple different perspectives (inner circle, figure 1). Analogous to the WHO-INTEGRATE framework, it aims to accommodate different features of complexity: depending on the impact the measure is assumed to have on the system it is implemented in, direct (those caused by the intervention) and indirect (those resulting from the system reactively changing due to the intervention) effects should be taken into account, as well as local, regional, national and even global implications. At the same time, both the immediate and the short, medium-term and long-term implications should be considered (figure 2). It is intended to guide the systematic reflection of the intervention in its context.

Figure 1.

WICID framework version 1.0. The colour of the 11+1 criteria of the WICID refers to their grouping and relation to the criteria of the WHO-INTEGRATE framework they are derived from. The center-most circle describes population groups onto which the criteria and aspects should be applied to. The innermost circle describes the perspective the decision-makers can take onto criteria and populations. WICID, WHO-INTEGRATE COVID-19.

Table 1.

Criteria of the WICID framework and description of what the criteria are intended to cover

Criterion	Description of the criteria
I. Implications for the course of the pandemic and its impact on health	Covers implications of the interventions or measures for the course of the SARS-CoV-2 pandemic, including the number of new infections, the resulting health-related consequences (eg, COVID-19-related mortality and morbidity), the implications for the capacity of the healthcare system to treat patients with COVID-19, as exceeding this capacity is associated with a pronounced increase in mortality. It furthermore covers the direct SARS-CoV-2-related health risk of individuals affected by the measures (probability of infection and probability of adverse consequences).
II. Implications for quality of life, social well-being and mental health	Covers the way the interventions or measures affect overall well-being and quality of life, which includes the degree to which the capability to shape everyday life according to one’s own wishes and needs is affected (eg, through restricting daily routines) or the experience of self-efficacy and of sense of coherence. It furthermore covers the social well-being of individuals (eg, the experience of loneliness) and the social cohesion of communities (eg, cohesion of families or non-family communities). The criterion furthermore captures the implications for the mental health of individuals and populations (eg, depression, anxiety disorders), including risk-factors—such as the experience of stress or fear, ability to practice coping mechanisms, receive support and other adverse mental health consequences (eg, suicides).
III. Implications for the physical health, health behaviour, health risks and healthcare beyond COVID-19	Focuses on implications of the intervention for behavioural, environmental, and interpersonal risk factors for health (including accidents and domestic violence) other than those directly related to COVID-19 (eg, physical activity) and their consequences for health. It furthermore covers the implications of the measures for availability, accessibility, acceptability, and quality of medical and care services and institutions for conditions other than COVID-19 (eg, willingness to seek emergency care in the case of myocardial infarctions).
IV. Proportionality and accordance with individual autonomy and fundamental rights	Covers whether the interventions or measures are in accordance with and how they—directly or indirectly—affect autonomy, self-determination, individual liberties and fundamental individuals’ rights (eg, privacy and data protection implications of a contact tracing app). It furthermore covers the intrusiveness of the intervention eg, providing information being a measure with a low intrusiveness; and restricting choice one with a high intrusiveness).36 The aspect of proportionality addresses whether intrusiveness and infringement of individual rights and liberties are proportionate to the expected benefit (or expected avoidance of harm).
V. Acceptability of and willingness to implement the measures	Focuses on the degree of acceptability and accepance of the measures and their consequences to the general population and different affected population groups. This includes the personal willingness to implement, adhere to, or enforce the measure (eg, whether reopening schools without any protective measures is acceptable to teachers with a high-risk profile such as pre-existing health conditions). While acceptability is an end in itself, this criterion is strongly linked to feasibility and assumed effectiveness.
VI. Equity, equality and the fair distribution of benefits and burdens	Covers the implications of the measure for vulnerable population groups, whether and how it affects stakeholder groups differently and thereby the risk for increasing or reducing inequalities (eg, men benefiting less from a measure in comparison with women, exacerbating health inequities), considerations of equity (treating people differently according to their need to allow them the same capability of achieving an outcome), considerations of equal treatment (eg, not treating people differently without sufficient justification), and implications for the risk of individuals or population groups to be stigmatised or be discriminated against.
VII. Societal and environmental implications & considerations	Covers the implications for civil society, social life, and culture from an individual-level and system-level perspective, and the implications for the functioning and cohesion of the society. It furthermore covers the implications of the measures for social determinants of health, including household income, social participation and education. A further aspect covered is the implications for the ecosystems resulting from the measures.
VIII. Economic implications & consideration	Addresses the implications for the economy as a whole (population perspective/systems perspective) and of stakeholders in their role as economic actors and their activities (individual perspective). Including the implications of the measures for the work force (eg, closing of schools forcing parents to stay at home to take care of their children).
IX. Resource implications & considerations	Covers the requirements of the measures for different resources in the context of the availability of these resources as well as how the measures affect the availability and quality of these resources (eg, how many face masks would be required to provide every teacher with a high-risk profile with one, are these masks available, and would this lead to a shortage of masks for, for example, health service providers).
X. Feasibility implications & considerations	Covers the practical, technical and political feasibility of implementing the measures, as well as their legal conformity. Other aspects address feasibility-related characteristics of the measures, such as the flexibility in extension, adjustment or withdrawal of the measures, and the possibility of evaluating the implications of the measure and reacting adequately to new information (eg, ability to test those affected by the measure and conduct contact tracing if needed).
XI. Interaction with and implications for the health system	Addresses how the intervention will interact (synergistically or adversely) with other measures to control and contain the SARS-CoV-2 pandemic and other not directly COVID-19-related components of the health system (within a broad understanding of health system in accordance with the WHO).73 For example, the combination of school reopening and lifting travel bans on other countries in close proximity timewise can have an interacting, synergistical (negative) effect that could be different from the effects of each of these measures by themselves.
Quality of evidence	This metacriterion is intended to be applied across all criteria and aspects. For example, taking the quality of the evidence on health impacts alongside with its strength and direction into account. This metacriterion reflects the confidence that the available evidence is appropriate, applicable, and adequate to support the decision or recommendation. Evidence is interpreted in a broad sense, beyond an understanding focusing on quantitative evidence of effectiveness derived from systematic reviews or randomised controlled trials. Different forms of evidence can be used and be the most appropriate type of evidence to inform on the criteria (eg, an appropriate form to assess the accordance with selected fundamental rights can be a legal assessment). Decision-making under uncertainty—as is often the case in a pandemic—often requires a decision based on stakeholder experience and judgement, when stronger evidence is unavailable.

WICID, WHO-INTEGRATE COVID-19.

Table 2.

Concise version of the criteria and aspects of the WICID framework

Criteria	Aspect
I. Implications for the course of the pandemic and its impact on health	Implications for the risk of infection and course of the pandemic
	Implications for COVID-19-related health consequences
	Implications for the capacity of the healthcare system to treat (severe) COVID-19 cases
II. Implications for quality of life, social well-being and mental health	Implications for the capability to shape everyday life according to one’s own wishes and needs
	Implications for individual well-being and quality of life
	Implications of the measures for social well-being
	Implications for social cohesion of communities
	Implications for the experience of fear and insecurity
	Implications for stress, stressors, the feeling of being overwhelmed and overloaded
	Implications for mental health and its consequences
III. Implications for the physical health, health behaviour, health risks and healthcare beyond COVID-19	Implications for health-related behaviour
	Implications for the exposure to environments that affect health
	Implications for the risk of suffering accidents or being affected by violence
	Implications for medical emergency and standard care beyond COVID-19
	Implications for care and other health and social support services
	Other health-related implications beyond COVID-19
IV. Proportionality and accordance with individual autonomy and fundamental rights	Implications for and accordance with individual liberties and fundamental rights including autonomy and individual self-determination
	Extent of intrusiveness and proportionality regarding the expected benefit
V. Acceptability of and willingness to implement the measures	Socio-cultural acceptability of the measures
	Implications for willingness to implement, adhere to, or enforce the measures
VI. Equity, equality and the fair distribution of benefits and burdens	Implications for health, social and economic inequalities
	Implications for affected social groups, in particular vulnerable and marginalised populations
	Degree of fair distribution of benefits and burdens between population groups
	Accordance of the measures with the principle of equal treatment
	Implications for the risk of stigmatisation and discrimination
VII. Societal and environmental implications & considerations	Implications for actors and institutions of civil society, social life, and culture
	Implications for the civil society, social life and culture
	Implications for social cohesion, solidarity, and the risk of social and political division
	Implications for the economic situation of individuals and capability of social participation
	Implications for education and psychosocial development
	Implications for other social determinants of health
	Implications for ecosystem, planetary boundaries, and environmental sustainability
VIII. Economic implications & consideration	Consequences of the measures for individual economic actors and their economic activities
	Implications for the economy; including resilience and ability to recover
	Implications for innovation and economic development opportunities
	Implications for the measures on the retention and release of labour forces
IX. Resource implications & considerations	Financial costs, available financial resources and budgetary implications for different actors
	Requirements of, availability of and implications for human resources
	Requirements of, availability of, and implications for protective equipment, hygiene products and medical devices
	Requirements of, availability of, and implications for infrastructure to implement, enforce, and/or monitor measures
	Requirements of, availability of, and implications for further resources and infrastructures
X. Feasibility implications & considerations	Political feasibility and legal conformity
	Practical and technical feasibility
	Flexibility in implementation, extension, adjustment and withdrawal of measures
	Possibility of evaluation and adequate reaction to new information
XI. Interaction with and implications for the health system	Interaction with other measures to control and contain the SARS-CoV-2 pandemic
	Interaction with components of the healthcare system
	Implications for the capability and willingness to develop alternative, local solutions
metacriterion: quality of evidence

WICID, WHO-INTEGRATE COVID-19.

Figure 2.

Complex system perspective on the implications caused by a measure being introduced into a system. The intervention (grey circle on the left) is introduced as an ‘event’ to a system. It directly affects other components of the system (non-grey circles on the right) which again interact with other components of the system, causing a chain reaction of the system reacting and adapting to the event; leading to the societal, economic and health-related consequences of the intervention.

Criteria and aspects of the WICID framework

The 11 criteria in the WICID framework consist of: three criteria focused on the balance of health benefits and harms: (I–III in figure 1: light red), two criteria focused on the accordance with human rights principles and sociocultural acceptability (IV–V, beige), one criterion focused on equity, equality and non-discrimination (VI, green), two criteria focud on the societal implications (VII–VIII, light yellow), and three criteria focus on feasibility and implementation (IX–X, blue) and health system considerations (XI, light blue). Table 1 describes the 11+1 criteria in detail; with table 2 containing the 48 associated aspects in their concise formulation. Online supplemental file 3 contains a more comprehensive version of the framework, providing more details and examples to guide users of the framework. The online supplemental file 4 contains exemplary passages from the coded strategy documents for criteria and aspects.

Considering criteria for different populations

Depending on the measure and decision-making context, criteria and aspects within the framework should be considered for the population as a whole, as well as for different population groups to assume relevant implications adequately for these groups. For example, an intervention such as a regulation forcing people not to leave their houses can have adverse effects which disproportionately affect people affected by insecure housing circumstances or school closures creating a disproportionately high burden on people that are directly involved in childcare. The stakeholder groups to be considered will depend on the type of measure (eg, closing schools vs closing nursing homes to the public). Building on the WHO-INTEGRATE framework and the strategy documents, we suggest to consider the implications for (figure 2, center-most circle): (a) the general population, (b) those intended to benefit from the intervention (eg, young school children in the case of school reopening), (c) those intended to implement the measures (eg, teachers), (d) populations with a high-risk profile (senior citizens with pre-existing conditions), and (e) other affected stakeholder groups (eg, employers). Within these population groups, further disaggregation based on relevant social characteristics with an emphasis on vulnerable and marginalised populations should be conducted (eg, school children with a family with a low socioeconomic status). These relevant characteristics are likely to vary depending on context, although socioeconomic status, age and gender are likely to be important across context. The PROGRESS Plus framework can provide guidance on the identification of relevant characteristics.54

Employing multiple perspectives: the population, the individual and the health system

Depending on the measure and decision-making process, decision-makers need to reflect on the different criteria from different perspectives to inform their deliberations (inner ring in figure 1). For example, the health implications of a measure on the SARS-CoV-2-related health risk (eg, reopening of schools) can be approached from a population perspective (looking at the implications of the intervention for the population as a whole, which can take place on a local or community level, federal state level and/or national level; for example, incidence rate of infections and associated mortality rates), a systems perspective (reflecting the intervention from the perspective of the health system, as well on a local, regional and national level; for example, taking the implications for the capacity of the health system and the availability of resources into account), or the individual perspective (taking the perspective of an individual affected by the population; for example, the risk for individual teachers working in reopened schools).

Taking a complex systems perspective

The implications of the NPIs can reach far. Therefore, the WICID framework embraces a complexity perspective2: the measures are regarded as ‘events in a system’,55 with the (intended and unintended) effects resulting from the interaction of the measures’ components with each other and components of the larger system. Within this perspective, disentangling the effects directly caused by the measure itself from the interplay of context and measure can be challenging if not impossible to do9; posing challenges for transferability and generalisability of evidence.

Analogous to the ripple effects caused by dropping an object in a pool of water, the effects initiated by the introduction of the measure to the system can lead to a chain reaction that can be followed outwards incrementally (figure 2). The more profound the impact of the measure to the system, the further the effects of the measure throughout the system can be observed. For example, the shutdown of a few companies in a region for a short time can lead to locally felt adverse economic consequences. However, a marginally more impactful event of closing the same companies for a marginally longer time and thereby exceeding an economic threshold can lead to the insolvency of these companies, causing—depending on the companies—a disruption of globalised production chains with economic consequences that can have regional, national and even global effects. Therefore, depending on how profound the impact of the measure is assumed, decision-makers need to consider whether the measure is likely to lead not only to immediate and local, but also regional, national or global consequences over the short, medium and long term. When reflecting on the measures, a focus should not only be on the direct effects along the intended causal pathway, but should also anticipate implications caused across several degrees of indirectness. These different dimensions are not necessarily related: for example, direct and indirect health-related, societal and economic implication occurring from immediate to long term could arise exclusively on a local level (figure 2).

Discussion

Summary of findings

We adapted the WHO-INTEGRATE framework to decision-making processes on NPIs intended to suppress or mitigate the effects of the SARS-CoV-2 pandemic. We used brainstorming exercises and content analysis of comprehensive strategy papers on the phasing out of the implemented lockdown measures in Germany. The resulting WICID framework version 1.0 consists of 11+1 substantive decision-making criteria, containing 48 decision-making aspects. Depending on the needs of the decision-making processes, these are intended to be applied by policymakers for and with different affected stakeholder groups using a multi-perspective approach. In line with the underlying complexity perspective, rather than only focusing on direct, immediate and local effects of the measure, the ripple effects caused by the introduction of the measure to a given system or policy field should be followed to adequately consider the implications a measure might have.

Advancing the WICID framework version 1.0 in phase III

The current version of the WICID framework version 1.0 will be expanded in a third phase. The limited diversity of expert groups established to inform policymakers on the handling of the SARS-CoV-2 pandemic has faced some criticism.6 Our approach of adapting the WICID framework based on strategy documents therefore comes with the risk that relevant criteria were overlooked due to the limited selection of expert groups and the stakeholder groups (not) represented within them. The third phase of the research project aims to address this issue by including the perspectives of various stakeholder groups across the society and expanding the WICID framework version 1.0 with considerations not adequately covered previously.

In phase III we will conduct a content analysis of key documents representing the opinions and perspectives of stakeholder representatives across the society (ie, of affected populations, non-governmental organisations, private sector) and using the results to validate and—where needed—expand the framework version 1.0. Using a sample of NPIs with broad societal implication as a starting point (closure/reopening of schools, closure/reopening of businesses, and ‘shelter-in-place’ regulations), we will include opinion pieces, position papers or press statements aimed at informing political decision-making on these measures. A first set of stakeholder group clusters (eg, social and welfare organisations) will be selected based on an initial brainstorming phase and stakeholder mapping56 and expanded in an iterative snowballing process. While it will not be feasible to cover all relevant organisations within a given cluster, we will analyse a heterogeneous sample which will be expanded in an iterative process based on the assessment of saturation. While this approach is not able to capture all voices of affected stakeholders, it allows for a broad and representation of societal values in decision-making across the society.

Short guidance on how to apply the WICID framework

While some EtD frameworks are tailored to specific decision-making processes (eg, on vaccination policies)57 and provide a fixed set of ‘ready-to-use’ decision-making criteria,25 others are more generic and require some form of adaption. The WICID framework is intended to be adequately generic to be applicable across a broad range of NPIs and decision-making contexts. While the 11+1 criteria can be used as a ‘ready-to-use’ EtD framework, we believe the framework to be most useful as a guide to systematically reflect on NPIs, as well as their interdependencies, and adapting them based on the specific needs of the decision-making process, using the WICID framework as guidance.

First, (1) a comprehensive logic model58 59 or systems map of the measure and the context is intended to be implemented and should be created, in order to describe possible implications. Next, (2) the WICID framework should be used to expand on dimensions not adequately covered (eg, by exploring the causal pathways from different perspectives, assessing the implications for different affected population groups, or using the criteria and aspects to assess its comprehensiveness regarding pathways and endpoints). Informed by the logic model, (3) an identification of relevant stakeholders should be conducted, ideally in the form of a comprehensive or focused stakeholder mapping.56 Next, (4a) those involved in the decision-making process need to define criteria which are assumed to be of relevance for deliberating on the measure. This can be done for example, by selecting individual aspects from within each criterion and adapting them to the context at hand. Using the example of school reopening, this could include the risk of outbreaks, health implications for teachers, for students, and for family members, the implications for the well-being of these groups, educational implications and so on. Given the complexity of the decisions at hand, it is likely not all factors of relevance can be covered in depth. On the one hand, this reflects the reality of the decision-making process; on the other hand, the rationale for the selection should always be provided to ensure openness and acceptability. (4b) The assumed importance of the criteria should be rated (eg, on a 1–5 scale from ‘less important’ to ‘critical’) and selected. (5) Efforts should be made to receive feedback on the expanded logic model and the selected criteria from key stakeholder groups identified in the mapping. Repeated rounds of steps 1–4 are likely to produce the best results. Next (6), efforts should be made to acquire appropriate sources of evidence to inform on the selected criteria (eg, by commissioning research or inviting experts’ judgements). (7) The retrieved evidence for each criterion should be summarised and presented alongside the assessment of the quality of the evidence and of the transferability to the context at hand. The group of decision-makers are now asked (8) to engage in the deliberation to balance the criteria against each other, taking their weight, direction, quality and transferability of the evidence into account. Finally, (9) the final judgement and the underlying rationale should be made transparent and public.

Relation to public health ethics framework

Various public health ethics frameworks providing guidance on principles and values to consider in public health and health policy decision-making have been published,38 60–62 some of which are more general,32 33 36 63 64 while others focused on public health emergencies and pandemics.65–69 Building on these foundations, institutions such as the German ethics council70 or the German Network Public Health COVID-1971 72 have outlined relevant values and principles for decision-making in the current public health crisis. These include the duty to provide care, health, non-discrimination, security, equity, individual liberty, privacy, proportionality, protection of the public from harm, reciprocity and solidarity, among others.71

The WICID framework was developed to be in line with these documents; primarily due to the underlying WHO-INTEGRATE framework being developed with a foundation in WHO norms and values and key public health ethics frameworks.29 The WICID framework aims to translate these principles and values into criteria applicable for real-world decision-making processes in the pandemic (eg, by translating the general moral considerations of producing benefits, and avoiding, preventing and removing harms32 into—among others—the criteria I, II, III, VII and VIII on the different positive and negative social, economic or health-related effects an NPI might have for individuals and populations). Furthermore, the framework aims to place criteria derived from these values and principles alongside other factors of relevance for real-world decision-making often not covered in depth in public health ethics frameworks, such as considerations of feasibility or the wider implications for the (health) system. While some values and principles have a direct representation in the framework criteria (eg, individual liberty, privacy or proportionality), others are introduced on the level of the perspectives or the populations the criteria should be applied to (eg, reciprocity being reflected in the consideration of those intended to implement the interventions).

Need for fair and transparent processes

The use of an EtD framework should not and cannot be an adequate substitute for the participation of all relevant stakeholders. This also applies to the WICID framework, which in itself is insufficient to achieve fair decision-making processes with results considered acceptable and legitimate.20 21 Since, especially at the beginning of a pandemic, decisions on measures to be taken often have to be made under limited scientific evidence, rapidly increasing knowledge and considerable time pressure, a comprehensive approach to stakeholder involvement is hardly possible. Efforts to involve the voices of affected stakeholders, for example, through rapid response statements by stakeholder representation organisations, are therefore of central importance. This can not only improve acceptance and legitimacy, but also lead to better outcomes. However, a rapid stakeholder engagement requires a degree of organisation, mobilisation and (scientific) literacy; which by itself can lead to a disbalance in representation. The needs of affected marginalised and vulnerable population groups without strong political capital (eg, people affected by homelessness or mental illness) are especially at risk of being overlooked in processes with limited participation efforts. In addition, special attention needs to be placed to the composition of the stakeholder groups.6 Other important values and principles underlying fair decision-making processes include accountability, inclusiveness, openness and transparency, reasonableness and responsiveness.71 72 Openness includes that not only the final decision, but also the underlying rationale, including the criteria and evidence, is made transparent and easily accessible. Documenting the process and the decisions made and providing—to some extent—access to these documents to the public can increase social acceptance for public health measures as well as lifting them. As the main focus within this research project lay on substantive criteria of the WICID framework and does not comprise procedural criteria, other procedural frameworks for example, from the field of public health ethics can serve as guidance.

Strengths and limitations

The WICID framework was developed by building on the WHO-INTEGRATE framework29 which was developed as a principles-based approach to ensure a solid, comprehensive normative foundation. It is also based on previous research such as the result of an overview of systematic reviews on public health and health system decision-making criteria,39 and expanding this foundational framework through a broad set of comprehensive strategy documents informing decision-making processes in Germany. Application in other country contexts therefore need to be tested and the framework, if necessary, updated and revised.

The WICID framework was developed using strategy documents intended to inform the German government. While we believe the resulting WICID framework can prove useful and applicable to other regions within and outside Europe, the need for adapting to the respective decision-making contexts is necessary. Likely, the basis of the WHO-INTEGRATE framework, which was developed not only for the global level at the WHO, but also to be applicable on national and subnational levels throughout the world, can cover factors not adequately captured in the German strategy documents.

The framework in its current version 1.0 was developed based primarily on comprehensive strategy documents developed by expert groups. The composition and the intention of these groups is likely to have influenced the criteria, consideration, values and principles covered within them. While we believe possible shortcomings and blind spots are in part compensated using the WHO-INTEGRATE framework as a basis, there is a risk of relevant factors being missed. We aim to address this issue in phase III of this research project.

Despite multiple approaches to identify the comprehensive strategy documents, we acknowledge the possibility of having missed on individual statements. Furthermore, likely other relevant strategy documents exist, but were not disclosed by the governments or leaked through to the public and therefore are not captured in our analysis. An update of the searches will be conducted as part of phase III.

In order to be used and improve decision-making, political decision-makers need to perceive the framework as acceptable, useful and relevant. The exploration of the perception of intended users was not part of this study, Future qualitative study can help to refine the framework and overcome barriers to applicability. Furthermore, while the framework aims to guide deliberation based on explicit, substantive criteria, we acknowledge that political decision-making should not be influenced by political context such as upcoming elections and personal preferences such as friendships and competition. We will explore how to integrate the influence of political context within phase III.

Another limitation is the distinction between comprehensive strategy documents and position papers by affected stakeholder groups, which was not always a clear cut. However, we believe this only to be a minor limitation, as all borderline documents were retained and will be included in the phase III of the project.

Conclusion

The WICID framework represents a comprehensive COVID-19-focused EtD framework intended to guide policy and public health decision-makers on making decisions on NPIs. It is rooted in WHO norms and values, criteria and considerations used to inform decision-making, and a complex systems perspective. While adapted to COVID-19-related challenges, it is intended to be generic in a way to be applicable across a broad range of decision-making processes, contexts and on a diverse set of measures. The WICID framework can be a useful tool for those involved in the difficult task of making decisions on NPIs at the local, regional and national level (eg, from decision-makers deciding on municipal regulations of how to (re)open a specific school to decision-makers deciding on state-wide regulation on protective measures in the educational system), as well as the scientific expert groups advising these political decision-makers. For those, it can be supportive to systematising the decision-making process, making the underlying rationale more transparent and contributing to the relevance of the decision criteria.