A WHO tool for risk‐based decision making on blood safety interventions

Abstract

Background

Risk‐based decision making is increasingly recognized as key to support national blood policy makers and blood operators concerning the implementation of safety interventions, especially to address emerging infectious threats and new technology opportunities. There is an urgent need for practical decision support tools, especially for low‐ and middle‐income countries that may not have the financial or technical capability to develop risk models. WHO supported the development of such a tool for blood safety. The tool enables users to perform both a quantitative Multi‐Criteria Decision Assessment and a novel step‐by‐step qualitative assessment.

Study Design and Methods

This paper summarizes the content, functionalities, and added value of the new WHO tool. A fictitious case study of a safety intervention to reduce the risk of HIV transmission by transfusion was used to demonstrate the use and usefulness of the tool.

Results

Application of the tool highlighted strengths and weaknesses of both the quantitative and qualitative approaches. The quantitative approach facilitates assessment of the robustness of the decision but lacks nuances and interpretability especially when multiple constraints are taken into consideration. Conversely, while unable to provide an assessment of robustness, the step‐by‐step qualitative approach helps structuring the thought process and argumentation for a preferred intervention in a systematic manner.

Conclusion

The relative strengths and weaknesses of the quantitative and step‐by‐step qualitative approach to risk‐based decision making are complementary and mutually enhancing. A combination of the two approaches is therefore advisable to support the selection of appropriate blood safety interventions for a particular setting.

1. INTRODUCTION

Risk‐based decision making concerns the selection of mitigating measures from a set of candidate interventions to prevent the occurrence of one or more undesired outcomes, or to reduce the probability or impact thereof. ¹ Such decision making requires collection of all relevant information concerning the foreseen occurrence of various undesired outcomes and organizing these data into a structure that supports the decision makers in making well informed choices. However, the decision‐making process itself is complex, involving contextual and political considerations, long‐term goals, diverging stakeholder interests, effective communication and ultimately, the acceptance of the consequences of the final decision. Within the blood transfusion community, the Alliance of Blood Operators (ABO) developed a Risk Based Decision Making (RBDM) Framework that addresses various elements of the complex decision‐making process, tailored specifically to the decision‐making process by blood operators. ² , ³ , ⁴ The framework consists of six stages (preparation, problem formulation, participation strategy, assessment, evaluation, decision), and for each of these stages guidelines and checklists are provided that help the decision makers in both managing the decision‐making process and building a case for the most appropriate decision to be made. ⁵ Whereas management of the decision‐making process through incorporation of a large number of considerations is a strong point of the ABO‐RBDM framework, a weakness of the framework is that it is highly conceptual and lacks hands‐on tools for its implementation. This can be especially challenging for small blood centers and more broadly in countries that do not have the financial or technical capability to develop risk models that are required to provide quantitative estimates for various outcomes (eg, how many deaths caused by transmission of an infection by blood transfusion will be prevented by implementing different candidate screening tests; how many units of blood will be lost through positive test results). Hence, WHO developed a simplified, practical and easy‐to‐use Risk‐Based Decision Support Tool for Blood Safety aimed specifically at decisions on the implementation of blood screening tests or pathogen reduction technology, which can be used by both blood operators and blood regulators and is accessible via the WHO website (www.who.int/health-topics/blood-products) or at the tool's own website (www.decisionsupportforbloodsafety.com). Additionally, the complementary qualitative and quantitative approaches provided in the tool may help decision makers to explain and obtain support for their decisional processes. Following a global consultation of stakeholders in 2017, a first version of this tool was developed in 2018 and pilot tested with 20 blood operators, policy makers and public health experts including representatives from 11 African countries. The tool was finalized in 2019 taking on board feedback obtained during this pilot exercise. In this paper the contents of the WHO risk‐based decision support tool, its functionality and added value for decision making are illustrated and discussed.

2. MATERIALS AND METHODS

2.1. Review of risk‐based decision‐making steps

There is a set of generic steps that can be found in any risk‐based decision‐making approach. ⁶ , ⁷ These steps, as well as a description of their content is shown in Table. For many practical applications of risk‐based decision‐making support for blood safety, like the decision to select the most appropriate screening test to reduce the risk of transmission of an infectious disease, the first few steps are easily defined. For instance, when considering the prevention of HIV transmission, the problem formulation (Step 1), the risk management strategies (Step 2) and the outcomes to consider (Step 3) are usually straightforward. If the problem is to reduce the transmission of HIV infections, the strategies relate to the candidate screening tests to consider and the outcomes will most likely include the number of transmissions prevented, the costs of testing, the prevented costs of life‐long treatment of infected patients, and possibly consequences of loss of blood units due to true positive and false‐positive test results. Slightly more challenging is the assessment of outcomes (Step 4), as this requires modelling of infectious disease transmission by blood transfusion and the effectiveness of screening tests. The real challenge in the decision‐making process however lies in evaluating and balancing the impact of various outcomes for each of the different risk management strategies (Steps 5‐7) and providing justification for a preference in case this would deviate from decisions for comparable cases (Step 8).

2.2. Specification of safety interventions and outcomes in the WHO tool

The outline presented in Table 1 is very generic and flexible. However, it was decided (based on feedback obtained during the pilot evaluation) that the WHO tool should consist of a limited set of restricted risk management strategies. Therefore, the WHO tool allows assessment of a preformatted set of interventions (eg, donor deferral, blood screening or pathogen reduction), and all interventions considered are compared to an alternative without any additional safety measures in place. The user is only required to enter a set of parameters related to the specific setting (number of donations, prevalence among donors, proportion recipients not affected, coverage rate for the safety intervention, total costs of treatment per patient, mortality rate of infected patients) and a set of parameters for each of the measures considered: e.g. donor deferral (effectiveness of donor selection), screening (test sensitivity, specificity and the costs of testing per donation), or the pathogen reduction technology applied (effectiveness of the technology, costs per donation, and the reduced yield of products that may take place in the production process). An overview of various general and intervention specific parameters is given in the first three columns of Table 4. In addition to these quantitative parameters, one qualitative parameter assessing the technological complexity per safety intervention was added. This was considered desirable to allow for weighting various safety interventions according to the practical viability of applying the technology considered in a specific setting. For each of the safety interventions, quantitative estimates for various outcomes were calculated. Table 2 shows an example of a safety interventions & outcomes table. The input parameters used for calculating these outcomes are provided in Table 4. The formulas used for these calculations are available in the user guide of the tool provided in the Supplementary Materials S1 or in the User Guide on the tool's website.

TABLE 1.

Generic steps in a risk‐based decision‐making approach

Step	Reference	Description
1.	Problem formulation	Defining the exact problem to be solved (eg, implementation of a new blood screening test for HIV to reduce the number of HIV transmissions), describe the context (eg, implementation for all new donors in the blood supply in a particular region or country).
2.	Describe risk management strategies considered	Describe various risk management strategies in detail. Each strategy might consist of a combination of interventions. In case some feasible alternatives or combinations of interventions are not included as a separate safety management strategy, provide the rationale for their exclusion. These concern for instance practical or financial considerations.
3.	Define outcomes	Define a set of outcomes. These consist of all relevant effects, both favorable and unfavorable, that each of the risk management strategies will have on the blood supply. Examples of outcomes might be the number of infections transmitted, number of deaths resulting from these infections, number of products wasted and costs of screening.
4.	Assess consequences using reference data	For each management strategy considered a quantitative estimate for each of the defined outcomes is derived. This assessment will require input of various kinds of reference data (eg, prevalence and incidence rates for infections, number of donors and donations, effectiveness of screening tests) and a model to derive an estimate of the outcome of interest from these reference data. The assessment results are presented in an “effects table”, a matrix where the assessed outcomes are presented for each management strategy.
5.	Explore trade‐offs	Determine how various outcomes are to be compared and their increases/decreases valued? For example, what is the value of a reduction in the number of deaths worth in monetary terms? Are there particular thresholds applicable?
6.	Address uncertainty	Most inputs used in the assessment (Step 4) are not fixed, but their exact values are surrounded by uncertainty. This also holds for any valuations found for various outcomes.
7.	Explore risk attitude(s)	In considering the trade‐off between various outcomes, individuals (or organizations) may have different perceptions on the acceptability of uncertainty of various outcomes. Increasing levels of uncertainty are generally associated with higher risk levels.
8.	Review comparable decisions	Consistency of decisions with respect to similar outcomes in a different setting or with respect to similar decisions in a different setting (e.g. country) may affect the derived preference.
9.	Summary of findings	The end result of the decision‐making process is generation of a concise summary of the findings from all of the above steps with determination of a preferred risk management strategy. If required, the preferred risk management strategy derived from the materials collected can be substantiated with argumentation.

TABLE 4.

Model parameters and viable range of values used, and outcomes of the sensitivity analysis

Description	Value	Units	Range of viable values		Changepoints a
			Min value	Max value
General model parameter
Number of donations	60 000	‐	60 000	60 000
Prevalence among donors	2%	‐	1%	20%	At 10% a change from 3 to 4
Proportion recipients not affected	10%	‐	1%	20%
Coverage rate for the safety intervention	50%	‐	50%	100%
Costs of treatment (per patient)	150.00	US$	100.00	200.00
Mortality rate of infected patients	17.5%	‐	10%	20%
Intervention specific parameters
Sensitivity of rapid serologic testing	96.0%	‐	65%	99.99%	At 99.4% a change from 3 to 2
Specificity of rapid serologic testing	92.0%	‐	80.0%	99.99%
Costs of rapid serologic testing (per donation)	1.12	US$	0.50	2.50
Sensitivity of laboratory serological testing	98.0%	‐	75%	99.99%	At 94.6% a change from 2 to 3
Specificity of laboratory serological testing	98.6%	‐	98.0%	99.99%
Costs of laboratory serological testing (per don.)	4.33	US$	3.25	5.40
Sensitivity of NAT testing	99.9%	‐	99.4%	99.99%
Specificity of NAT testing	99.7%	‐	99.6%	99.80%
Costs of NAT testing (per donation)	38.40	US$	24.90	59.50
Effectivity of pathogen reduction	90.0%	‐	75%	99.99%
Costs of pathogen reduction (per donation)	20.00	US$	15.00	30.00
Production loss of pathogen reduction	5.0%	‐	3%	7%
MCDA weight parameters
Total net costs	1.00	‐	1	1
Annual number of deaths	148 000	US$/death	0	1 500 000	At 1203 a change from 1 to 3; At 708 830 a change from 3 to 4
Annual cost of the intervention	0	‐	0	0
Annual number of products lost	148	US$/product	0	5000	At 3262 a change from 3 to 4
Medium technological complexity	100 000	US$	0	1 500 000
High technological complexity	300 000	US$	0	1 500 000

^a The numbers used reference the different risk management strategies: 1 ‐ No testing, 2 ‐ Rapid serologic testing, 3 ‐ Laboratory serological testing, 4 ‐ NAT testing, 5 ‐ Pathogen Reduction.

TABLE 2.

Safety interventions & outcomes table, showing estimates for various outcomes per safety intervention considered to prevent HIV transmission by blood transfusion in a fictitious setting

Optional safety interventions	Total net costs [US$]	Annual number of deaths [−]	Annual cost of the intervention [US$]	Annual number of products lost [−]	Technological complexity [−]	Incremental cost‐effectiveness ratio relative to no intervention[US$ per additional death prevented]
	Total cost of the safety intervention + cost of treatment of infected patients	Total number of deaths given that the safety intervention indicated is implemented	The total cost of the intervention (including costs of personnel, equipment training etcetera)	The number of blood products discarded due to false positive test outcomes or production loss of the safety intervention applied	The technological requirements considering education of personnel and availability of materials and support. These will affect the overall effectiveness of the intervention	Total net cost per additional number of deaths prevented as compared to the “no testing” intervention. This provides an indication of the “value for money” of the intervention relative to the baseline situation
No testing	162 000	189	0	0	Low	‐
Rapid serologic testing	117 840	98	33 600	2352	Low	−487 a
Laboratory serological testing	212 520	96	129 900	412	Low	546
NAT testing	1 233 081	95	1 152 000	88	High	11 346
Pathogen reduction	689 100	104	600 000	1500	Medium	6198

^a This outcome has a negative value because the net cost of the intervention (117 840 US$) is less than the net cost of the “No testing” intervention (162 000 US$), resulting in a negative incremental cost‐effectiveness ratio: (117840‐162 000)/(189‐98) = −44 160/91 = −487.

2.3. Multi‐criteria decision assessment (MCDA)

Multi‐Criteria Decision Assessment is increasingly applied to support the appraisal of new medicines. ⁸ , ⁹ , ¹⁰ This has evolved in the development of various methods for balancing the benefits and risks of medicines to support more transparent and consistent decision making . ¹¹ , ¹² , ¹³ , ¹⁴ The core of a multicriteria decision‐making process is a safety interventions and outcomes table as shown in Table 2. In different contexts the name of this table may differ (eg, “Performance Matrix”, “Consequence Table” or “Effects Table”), but all will show the alternatives considered vs their associated consequences.

2.4. Quantitative assessment

Deciding which risk management strategy to select requires making trade‐offs between various outcomes that involve individual judgements on the value of each of these outcomes. A method to derive a quantitative comparison of the alternative interventions is to assign relative weights to each of the outcomes and to provide a mechanism to aggregate the weighted outcomes into a single number that expresses an overall score (MCDA “utility”) based on an underlying common measure (eg, dollar cost). As judgements regarding weights are personal and there are no universal rules for making them, the weights are subjective in nature. There are various preference elicitation tools and techniques available for obtaining decision‐makers' preferences regarding the different outcomes (eg, analytic hierarchy process or discrete‐choice experiments). ¹⁴ , ¹⁵ However, these require expertise and experience in expert elicitation as well as commitment and input from various stakeholders. ¹⁶ In our tool we implemented a simple linear additive model as a practical approach. This means that per outcome only one single weight is used and that the overall score per optional safety intervention (the MCDA score) is just the sum of the weighted outcomes of the quantitative estimates. The best option is in our case the intervention with the lowest overall MCDA score as all outcomes are reflecting undesirable events.

2.5. Sensitivity assessment

The MCDA aims to provide an optimal decision by balancing outcomes for each of the optional interventions on the basis of the weights defined per outcome. The outcomes are calculated using estimates for each of the input parameters. However, as the point estimates for the input parameters may be subject to uncertainty, it is worthwhile evaluating the robustness of this optimal decision. As the optimal decision is directly calculated as a function of the input values and the outcome weights, it is straightforward to calculate the change in the optimal decision when the input value is varied over a range of credible values. This type of univariate sensitivity assessment is very intuitive and provides useful insights on the robustness of the overall preference. In the decision support tool, the user can provide a minimum and maximum value for each input parameter. In a sensitivity assessment the tool determines the value of the input parameter (“changepoint”) at which selection of the optimal intervention would be altered by the effect of that value on the MCDA scores for the interventions. As the optimal decision is similarly a function of the outcome weights, the user also can evaluate the sensitivity of the final decision to variation of the outcome weights.

2.6. Qualitative assessment

The quantitative character and the fact that the selection of the optimal intervention evolves from a numerical calculation rather than a narrative rationale might induce a lack of trust in the appropriateness of the outcome. In addition, a decision derived from a mathematical model based on a limited set of criteria (the pre‐defined outcomes) might also miss considerations that may change the ultimate preference for one intervention over another. This could be the case for example when it is known that one intervention would be strongly preferred by one of the decision makers while the quantitative disadvantage compared to the optimal intervention would be small. In such a situation it might be preferable not to implement the intervention suggested by the quantitative MCDA.

In a qualitative MCDA the decision makers judge the overall value of a technology by deliberating on its performance in relation to an explicitly defined set of criteria (ie, they provide a qualitative interpretation of the Safety interventions & Outcomes table). ¹⁴ Note that this process differs fundamentally from an intuitive prioritization (without any specific method) as the criteria considered are made explicit as are the considerations proposed by the decision makers. The cognitive burden of this process, however, may be extensive, especially when it involves the simultaneous evaluation of multiple technologies requiring complex trade‐offs between various criteria. ¹⁴ To support the deliberative process, we designed and implemented a novel step‐by‐step approach for the qualitative assessment using the WHO tool. In this approach the user first ranks the outcomes in descending order of importance. The decision process starts by comparing the candidate interventions based only on the two outcomes perceived as most important. The decision maker must provide argumentation for selecting the optimal intervention when considering only these two outcomes. Next, the third‐most important outcome is added for consideration and arguments for selecting the optimal intervention must again be provided. This process is then repeated until all outcomes have been included in the process. By sequentially adding outcomes to consider, the thought process underlying the decision is guided systematically and the complexity of interactions gradually increases. However, the argumentation is also likely to converge rapidly as each additional outcome will have a diminished impact on the choice of the optimal intervention selected in the previous step due to the lesser perceived significance of each successively considered outcome.

3. RESULTS

For illustration of the MCDA tool, we considered its implementation to determine the best screening or pathogen reduction technology to reduce the risk of HIV transmission in a model scenario. The fictitious setting, represented by the input values in Table 4, is that of an African country with a blood establishment supplying around 60 000 blood products per annum. The total budget that is available for blood screening or processing is 100 000 US$ per year. Due to small number of donors, substantial amounts of product losses are considered unacceptable. For purposes of illustration prevalence of HIV infections among donors is set arbitrarily at 2%, and about 10% of the transfusion recipients are presumed to be carriers of the HIV virus. Due to operational limitations, it is assumed possible to only implement additional screening or processing of blood products for 50% of the blood supply. The cost of treatment of an average HIV patient is 150 US$ and the excess mortality rate of HIV infected transfusion recipients is estimated at 17.5%. The outcomes presented in Table 2 represent the outcomes of candidate safety interventions for this setting. Table 2 indicates that the Total Net Costs without the implementation of any additional safety intervention are 162000 US$, as shown on the first row for the safety intervention “No screening.” This number is calculated as: the number of donations (=60 000) × prevalence of infection among donors (=2%) × (1 ‐ proportion of recipients not affected) (=90%) × the costs of treatment of an infected donor (=150 US$). The formulas used for the calculations in Table 2 are available in the user guide of the tool provided in the Supplementary Materials S1 or on the tool's website. Figure 1 shows these same results graphically. The quantitative estimates of various outcomes in this table are obtained from the set of input parameters (Table 4) by application of a risk calculation model that is embedded within the tool. Note that values for some of the model parameters might not represent real world characteristics of state‐of‐the‐art technologies but were modified for illustrative purposes.

FIGURE 1.

Estimates for various outcomes per safety intervention scaled to the maximum value per outcome [Color figure can be viewed at wileyonlinelibrary.com]

The MCDA assessment requires weights for each of the outcomes considered. In Table 3 the weighted outcomes for the candidate interventions and their overall MCDA scores are shown. This table is derived from the outcomes shown Table 2 in combination with (and multiplied by) the weights per outcome shown in the first row of Table 3. For this example, all outcomes are converted to an equivalent in US dollars. This means that the weight for the total net costs is set to 1. The MCDA weight for the annual number of deaths is set to 148 000 US$, implying that the decision maker is willing to spend 148 000 US$ to prevent one additional death by HIV transmission by blood transfusion. This number is the Value of a Statistical Life (VSL) estimate by Viscusi et al. for Zimbabwe. ¹⁷ The weight for the annual cost of each intervention has been set to zero as these costs are already incorporated in the total net costs from Table 2. For the production loss, a value 148 US$ per lost donation has been set. This number was derived from expert estimates that for every 1000 products lost one additional death is expected. Technological complexity is based on the expected negative impact (additional disturbances during operation) that the choice of a more complex technology will have on the blood supply. Note that the cost‐effectiveness ratio is not included in the MCDA as this outcome is derived from the total net costs and annual number of deaths which are already included in the MCDA.

TABLE 3.

MCDA score table

Outcome	Total net costs	Annual number of deaths	Annual cost of the intervention	Annual number of products lost	Technological complexity
MCDA weight [dimension]	1 [−]	148 000 [US$/death]	0 [−] a	148 [US$ per product lost]	100 000 [US$] (medium); 300 000 [US$] (high)
Optional safety interventions	Contribution per outcome to the overall MCDA score [US$] b					Overall MCDA score [US$]
No testing	162 000	27 972 000	0	0	0	28 134 000
Rapid serologic testing	117 840	14 545 440	0	348 096	0	15 011 376
Laboratory serological testing	212 520	14 265 720	0	60 917	0	14 539 157
NAT testing	1 233 081	13 999 986	0	13 054	300 000	15 546 121
Pathogen reduction	689 100	15 384 600	0	222 000	100 000	16 395 700

^a The MCDA weight for “Annual cost of the intervention” is set to zero because this cost is incorporated in the “Total net costs.”

^b For each cell the contribution to the “Overall MCDA score” is calculated by multiplication of the estimated outcome from Table 2 and the corresponding MCDA Weight.

Table 3 clearly shows that in this example the prevention of fatalities has the largest impact on the decision. This column provides by far the largest contribution to the overall MCDA score. The contribution from all other columns are more comparable for the different options, and the contribution from any of the other outcomes is one to three orders of magnitude less. From the overall MCDA score column it is also clear that “No testing” has roughly twice the (negative) utility as all other interventions, meaning that implementation of any of the interventions considered is strongly preferred.

Laboratory serological testing is preferred over rapid serological testing even though its costs are higher (96 300 US$). The lower number of products lost (1940 × 148 = 287 179 US$) and the lower number of fatalities (1.89 × 148 000 = 279 720) outweigh these additional costs of testing.

Laboratory serological testing is also preferred over NAT testing as the additional cost of NAT testing (1 022 100 US$) and the disadvantage of technological complexity (with a monetary equivalent of 300 000 US$) do not outweigh the benefits of the 1.8 fewer fatalities (with a monetary equivalent of 148 000 × 1.8 = 265 734 US$) and the 323 fewer products lost (with a monetary equivalent of 323 × 148 = 47 863 US$).

The results show that it is worthwhile to implement at least one safety intervention. With the weights used, a clear preference for laboratory serological testing was found. In a real‐world setting, this conclusion would be decisive if the input values were substantiated by expert opinion and supported by the decision maker.

Note that for the sake of illustration, we considered the impact of various safety interventions on the reduction of HIV transmission only. For pathogen reduction however, there would be additional benefits as its implementation would also reduce the risk of transmission of some other infections. Such scenarios would not affect the methodology presented but would merely increase the number of model parameters and potential scenarios to include.

3.1. Sensitivity analysis

As described in the methods section, for each of the input parameters and MCDA weights the change in optimal decision is determined by varying each of the inputs individually over a range of values. Table 4 shows each of the input values used in the model and the range of values considered. The last column in the table provides information on the changepoints within the range of viable values where the numbers identifying the interventions reflect the five optional safety interventions listed in the footnote below the table. From this column it can be found that within the range prevalence of HIV among donors between 1% and 20%, at 10% the optimal screening strategy will change from 3 to 4, that is, from “Laboratory serological testing” to “NAT testing.” In case the sensitivity of rapid serological testing would have been higher than 99.4% instead of the 96% used in the current assessment, rapid serological testing (option 2) would be preferred over laboratory serological testing (option 3). This would also be the case if the sensitivity of laboratory serological testing would have been less than 94.6%, instead of the 98% used in the current assessment. The changepoints for the weights used indicate that a change by a factor of 5 and 20 for the annual number of deaths and the annual number of products lost respectively, would be required to change the optimal interventions selected. The overall conclusion would therefore be that the preference for laboratory serological testing is rather solid.

3.2. Qualitative assessment

As described in the methods section, the first step in the qualitative assessment is determining the order of importance of the outcomes to consider in the assessment. For our fictitious case study we defined the order of importance of outcomes to be: (a) Annual number of deaths, (b) Annual cost of screening, (c) Annual number of products lost, (d) Technological complexity, (e) Total net costs, and (f) Cost‐effectiveness. There are a number of additional considerations relevant for this case study. These considerations are that (a) more than 100 deaths by transfusion transmitted HIV cases is considered unacceptable; that (b) the maximum budget available for screening is 100 000 US$; that (c) no more than 1500 products per annum may be lost due to screening or treatment by pathogen reduction; and finally, that (d) technological complexity above a medium level will be too difficult to maintain in practice.

In the last two columns of Table 5, the first two steps of the Step‐by‐Step qualitative assessment are shown. The conclusion of the first step is that “No testing” is not a viable option as the number of fatalities is considered unacceptable. This would be the most likely motivation for considering the implementation of an additional safety intervention in the first place. Any of the interventions considered are expected to roughly halve the number of fatalities. Also, pathogen reduction seems not to meet the proposed maximum number of fatalities considered acceptable. As the annual cost of screening is lowest for rapid testing, this is therefore the preferred option at this stage. Note that a purely quantitative assessment considering only these two outcomes, would result in a preference for laboratory testing as the value of preventing an additional two deaths with laboratory serological testing exceeds the increase in screening costs of 96 300 US$. However, it should also be noted that all other options exceed the maximum budget available for an additional intervention. In the second step of the assessment, the annual number of products lost are taken into additional consideration. Now a clear conflict appears as none of the options considered complies with all the restrictions previously defined. This means that the selection of the best safety intervention requires a review (or even a renegotiation) of these constraints. The budget for screening is most likely to be adjusted as this is exceeded by 30% whereas the maximum acceptable number of products lost is exceeded by almost 60%. Assuming the budget is expanded, consideration of the number of products lost shifts the preferred option from rapid testing to laboratory‐based testing. However, this may also be dependent on the setting. One might decide that accepting the additional number of products lost would be more easily overcome than extending the budget.

TABLE 5.

First two steps of the qualitative step‐by‐step MCDA assessment

Optional safety interventions	(1) Annual number of deaths [−]	(2) Annual cost of the intervention [US$]	(3) Annual number of products lost [−]	STEP 1 ‐ considering: (1) Annual number of deaths, and (2) Annual cost of the intervention	STEP 2 – Considering: (1) Annual number of deaths, (2) Annual cost of the intervention, and (3) Annual number of products lost
No testing	189 a	0	0	Unacceptable number of fatalities	Unacceptable number of fatalities
Rapid serologic testing	98	33 600	2352 a	Preferred option: Costs are low and the difference in the remaining number of fatalities between this and other tests is acceptably small	Costs are low and the difference in the remaining number of fatalities between this and other tests is acceptably small, but 4% loss of products (exceeding the threshold by 57%) is unacceptable
Laboratory serological testing	96	129 900 a	412	Exceeds the available budget	Preferred option: Costs of testing exceed the available budget, but as the number of products lost with rapid testing is not acceptable, a reconsideration of the budget constraint seems appropriate
NAT testing	95	1 152 000 a	88	Lowest number of fatalities but costs exceed the available budget by an order of magnitude	Lowest number of fatalities but costs exceed the available budget by an order of magnitude
Pathogen reduction	104 a	600 000 a	1500	Not a viable option, higher annual number of deaths at higher cost compared to both rapid and laboratory serological testing	Not a viable option, higher annual number of deaths at higher cost compared to both rapid and laboratory serological testing

^a Bold numbers exceed the acceptable maximum annual number of deaths (100), the available annual budget for the intervention (100 000 US$) or the acceptable maximum annual number of products lost (1500).

Steps 3 to 5 are not further elaborated in this paper in detail, but we refer the interested reader to the Supplementary Materials S1 to this paper or to the tool's website. However, it is clear that including technological complexity (Step 3) in the considerations will not affect the preference as it is identical for the two most favorable options determined in Step 2 (see Table 2). The same holds when additionally considering the total net costs (Step 4 of the assessment). Despite that the difference in total net costs between two options is slightly less than the difference in the direct costs of testing, this will not affect the preference for laboratory serological testing. Finally, when additionally considering cost‐effectiveness (Step 5), the fact that rapid testing seems to be cost‐saving (the cost‐effectiveness ratio for this option is negative, meaning the health care costs prevented by screening exceed the costs of screening) does not compensate for the excessive number of products lost. This means that the conclusion drawn after the first two steps of the assessment that laboratory serological testing is the preferred option is also the final outcome of the qualitative assessment.

In our scenario, both the qualitative and the quantitative approach lead to the same conclusion that laboratory serological testing is the preferred intervention. The biggest advantage of the step‐by‐step qualitative approach is that it allows incorporating much more subtle considerations such as inclusion of a threshold value and even reviewing the viability of an option despite its exceedance of a threshold. Such subtleties would be difficult—if not impossible—to incorporate in a quantitative model. Where a quantitative model may allow balancing preferences at a macro level, the final decision will always require the integration of considerations on a finer, micro level. As such, decision makers are recommended to always include a deliberative component in their process of formulating recommendations as this allows taking into account any considerations perceived relevant. ¹⁴

4. DISCUSSION

In this paper we reviewed the features of the WHO Risk‐Based Decision Support Tool for Blood Safety and demonstrated its functionality and usability based on a fictitious case study considering the selection of a safety intervention to reduce the number of transmissions of HIV by blood transfusion. Importantly, because of the illustrative purpose of the case study not all parameters used present a realistic characterization of the current health and technological data available.

One of the weaknesses of the WHO tool is that the assessment is restricted to the evaluation of a set of risk management options to reduce the risk of transmission of an infectious disease by blood transfusion. The tool was originally designed to be completely flexible in the risk management options and outcomes to include, but the pilot study showed that a pre‐structured design for a fixed decision problem would be preferable. Leaving risk management options open would require the user to define the appropriate risk models as well as implementing these in the tool. Hence, flexibility was sacrificed for ease of use and controllability. However, if the tool is successfully applied in practice, alternative risk management evaluations can be easily developed and implemented. For more general support on the complete risk‐based decision‐making process we strongly recommend reviewing the freely accessible website of the ABO Risk Based Decision Making Framework which describes a structured approach to support the decision making process in general and provides many useful checklists as well as a workbook to provide practical support. ⁵

A key feature of the WHO tool and starting point for the decision‐making process is a table with various outcomes assessed for each risk management strategy. This Safety interventions & Outcomes table can be used to support the decision‐making process in both a quantitative and a qualitative MCDA, which are both supported by the WHO tool.

The strength of the quantitative assessment is that in general it allows imposing more consistency over a broader range of decisions, ¹³ , ¹⁸ and that the robustness of the decision can be explored by evaluating the sensitivity of the intervention selected for changes in input parameters. The main disadvantage of this approach is that it does not consider any aspects that are not explicitly modelled. One might for example increase the MCDA weight whenever an outcome exceeds a threshold value to overcome these limitations specified in our case study. However, such modifications would introduce substantial additional complexity and hence reduce transparency. Also, the fact that the selection of the most favorable intervention is based on a numerical derivation from a quantitative model rather than a narrative rationale may not be easily accepted. ¹⁹ , ²⁰

The strength of the qualitative approach on the other hand is that it builds a case for the selected intervention by means of a documentation of considerations and argumentations used. This may reduce consistency over various decisions, but it facilitates communication of the reasoning behind the decision, which is likely to support acceptance of the decision. The step‐by‐step approach as presented on our tool helps structuring the thought process in a systematic manner.

For our case study, it is striking that despite the differing considerations applied in either method, the outcome of both assessments is the same. In the quantitative approach, the use of the weight for the number of deaths (the 148 000 US$ per death prevented) guides the decision toward laboratory serological testing. The sensitivity assessment shows that the preference for this intervention is rather insensitive to changes in model parameters. On the other hand, in the qualitative assessment, by simply inspecting the cost and the change in fatalities for each test, the preference points in the same direction. The fact that threshold values as well as any other considerations can be easily incorporated in the qualitative assessment results in a realistic argumentation in support of the intervention selected. All in all, the conclusion should be that the quantitative and qualitative approaches are complementary with each having its own strengths and weaknesses. Therefore, we conclude it is well‐advised to apply both approaches when selecting a new or re‐evaluating an existing intervention.

Hopefully, dissemination of the WHO tool for risk‐based decision support for blood safety will contribute to better blood safety decision making. The tool should motivate low‐ and middle‐income countries to find their own best suited safety strategies instead of trying automatically to implement the most advanced safety technologies employed in most high‐income countries. The tool in its current form only supports the evaluation of a set of risk management options to reduce the risk of transmission of an infectious disease by blood transfusion. However, if successful applications are reported, alternative scenarios can be easily developed and implemented in the future.

CONFLICT OF INTEREST

No conflicts of interest to disclose.

DISCLAIMERS

The authors alone are responsible for the views expressed in this article and their views should not be construed to represent the views, decisions, or policies of the institutions with which they are affiliated.

Supporting information

Appendix S1: Supporting information

Janssen MP, Nuebling CM, Lery F‐X, Maryuningsih YS, Epstein JS. A WHO tool for risk‐based decision making on blood safety interventions. Transfusion. 2021;61:503–515. 10.1111/trf.16231