Abstract
Background. Malaria is a major public health burden in sub-Saharan Africa. This study estimated the cost-effectiveness and budget impact of adding four-dose malaria vaccination in infants or children to existing interventions in 41 endemic countries in sub-Saharan Africa. Methods. A static Markov cohort model followed a simulated 2017 birth cohort (36.5 million children) for 15 years in 5-day cycles, comparing three strategies: child vaccination (doses at ages 6, 7.5, 9, and 27 months); infant vaccination (doses at ages 6, 10, and 14 weeks and 21 months); no malaria vaccination. The base-case analysis was conducted from the health system perspective with vaccine price assumed at USD5/dose and annual discounting of 3% for costs and disability-adjusted life-years (DALYs). Efficacy was based on the Phase III RTS,S clinical trial. Results. The model projected that 24.6 million children, or 26.2 million infants, would be vaccinated. Compared with no vaccination, child (infant) vaccination was projected to avert 16.8 million (16 million) cases of malaria and 113,000 (107,000) malaria deaths in the birth cohort over the 15-year period. The incremental cost-effectiveness ratio was USD200/DALY averted (USD225/DALY averted) for child (infant) vaccination, which represents 14% (17%) of the gross domestic product (GDP) per capita threshold. The estimated budget impact was overall larger for infant vaccination but mixed situations occurred across countries. Vaccine price, discount rate, and parasite prevalence had the largest effect on cost-effectiveness. Conclusions. Child vaccination with RTS,S would be more cost-effective than infant vaccination across countries. Adding RTS,S malaria vaccination to existing interventions would be cost-effective assuming one GDP per capita threshold for both child and infant vaccination in all examined countries except for 6 countries with lower transmission.
Keywords: Budget; cost effectiveness; Malaria; RTS,S vaccine; sub-Saharan Africa
Malaria is still a major public health burden in sub-Saharan Africa. In 2015, there were an estimated 191 million cases of malaria and 394,000 malaria deaths in the World Health Organization (WHO) Africa region. The malaria burden is concentrated in young children; in 2015, an estimated 292,000 children aged <5 years died of malaria in the WHO Africa region, accounting for 74% of all malaria deaths.1
Interventions to prevent malaria include vector control methods (mainly the use of insecticide-treated nets and indoor residual spraying), chemoprevention, and potentially vaccination.1 Prompt diagnosis and treatment can reduce the impact of malaria infections by reducing severe malaria cases and deaths.1
Vector control and improved access to treatment have contributed to a substantial decrease in malaria cases and deaths in the WHO Africa region; between 2010 and 2015, malaria cases decreased by 21% and malaria deaths by 31%.1 However, despite improvements in coverage, millions of people in Africa still do not receive malaria interventions. For example, the WHO estimated that in 2015 only 57% of the population at risk in sub-Saharan Africa were protected by insecticide-treated nets or indoor residual spraying; 20% of pregnant women did not attend antenatal clinics and of those who did 30% did not receive a single dose of preventive treatment; only a median value of 27% of children aged <5 years with fever received artemisinin-based combination therapy (ACT) based on Demographic and Health Surveys and Malaria Information Surveys conducted in 33 sub-Sahara African countries.
The addition of a malaria vaccine could complement existing malaria interventions, thereby offering the potential for further reductions in malaria burden. The RTS,S vaccine candidate has shown modest efficacy in a Phase III trial conducted in seven countries in sub-Saharan Africa in a context of high coverage of insecticide-treated nets and optimal access to ACT. Addition of four doses of vaccine to these existing malaria interventions resulted in a 36.3% reduction in clinical malaria cases over 48 months of follow-up on average in children who received the first dose at age 5 to 17 months and 25.9% reduction over 38 months of follow-up on average in infants who received the first dose at age 6 to 12 weeks.2
Based on the Phase III trial results from the group receiving the first dose at age 5 to 17 months, four different models developed by four independent research groups to estimate the potential public health impact of the vaccine indicated that the addition of a four-dose vaccination program could reduce the malaria burden by a median of 116,480 cases and 484 deaths per 100,000 fully vaccinated children.3 At a vaccine price of USD5 per dose, the median cost-effectiveness of a four-dose vaccine schedule was estimated at USD25 per case of clinical malaria averted.3
The present analysis uses one of these models, the model developed by GSK, 1) to extend the exploration of the potential economic impact of malaria vaccination to younger infants, and 2) to estimate the potential budget impact as well as cost-effectiveness of introducing a four-dose malaria vaccination program in either children (aged ≥6 months) or infants (aged ≥6 weeks) in 41 countries in sub-Saharan Africa using results from a recent study conducted in five sub-Saharan Africa countries, estimating the cost per fully vaccinated person in both age groups.
Methods
Model Structure
The model has been fully published elsewhere.4 Briefly, the model was constructed as a static Markov cohort model following a birth cohort over 15 years. The model structure is shown in Figure 1.
Figure 1.
Model structure. Modified from Sauboin et al.4
Severe episodes expressed as a proportion of clinical cases. Mortality expressed as a proportion of severe cases. The risk of severe disease decreases with the number of previous infections. Vaccine protection modelled as a reduction of the risk of infection. Vaccine protection assumed to wane over time (reduction of 50% after 5 months, slower reduction thereafter).
The model has the following states:
M: protected by maternal antibodies. This protection wanes at a fixed rate (transition wm).
S: susceptible to malaria infection. The probability of infection is determined by a fixed probability of infection (q), which varies according to malaria transmission intensity, multiplied by an age-dependent susceptibility factor (s).
I: infected with malaria. Infected individuals may be asymptomatic, after which individuals revert back to state S with increased immunity (probability a).
C: clinical episode of malaria. This may be an uncomplicated episode after which individuals recover (probability r) and revert back to state S with increased immunity.
F: severe malaria. A fixed proportion (f) of clinical malaria episodes, depending on the level of immunity, becomes severe. A fixed proportion of severe episodes leads to death. The other individuals recover (probability r, the same probability as recovery from state C) and revert back to state S with increased immunity.
R: resistant to clinical infection. This resistance can wane, returning individuals to the susceptible state (not shown in Figure 1).
The model includes six different immunity levels (S1–S6). Each successive immunity level has a lower probability of clinical malaria and severe malaria than the previous level. This allows the model to represent gradually increasing levels of naturally acquired immunity with repeated infections. Although there are only six immunity levels, children in the model can experience a larger number of infections. Children in the model with more than six infections have immunity level S6.
Static models are unable to reproduce transmission dynamics and therefore cannot simulate any potential indirect effect of vaccination by reducing transmission. This choice is consistent with the expected effect of the vaccination strategy as explained in the discussion section.
Input Data
Parasite prevalence data were obtained from 2015 Malaria Atlas Project (MAP) data at the level of subnational administrative regions (Admin-1 data).5 A relationship between parasite prevalence and the modelled probability of infection q was estimated, so that each parasite prevalence value corresponds to a certain force of infection in the model. This relationship was calculated for 12 levels of parasite prevalence (3%, 5%, 7.5%, 10%, 15%, 20%, 25%, 30%, 35%, 40%, 45%, and 50%). Estimates of malaria cases and severe malaria cases matching a MAP parasite prevalence value were simulated by making a linear interpolation between both parasite prevalence levels around that value.
The process for calibrating the model in absence of vaccination has been described in detail in the previous publication.4 Briefly, the natural history arm of the model was calibrated to reproduce simultaneously the incidence of clinical malaria observed in the control arm of the Phase III trial for both age groups (infants and children) in low, moderate, or high transmission categories (defined according to MAP data) across the entire follow-up period of the Phase III trial, and also the age distribution of malaria up to the age of 5 years in moderate and high transmission areas with no marked seasonality reported by Carneiro et al.6
The percentage of severe malaria cases was also estimated by calibration, fitting to data on the median incidence of severe malaria cases in both age groups (infants and children) in the Phase III trial over 18 months, together with the age distribution of the moderate transmission areas reported by Carneiro et al.6 The percentage of severe cases hospitalized was country-specific and assumed to be the same as the percentage with access to ACT (See Appendix File 1), derived from all available Demographic and Health Surveys (DHS) data with the indicator “Children with fever who took antimalarial drugs”.7 Other calibration parameters in the absence of vaccination were the same as those previously published.4
The proportion of severe malaria cases expected to result in long-term neurological sequelae was 1.7% (range = 0.85% to 2.54%)4 estimated from published data on the proportion of severe cases categorized as cerebral malaria (14.4%)8 and the percentage of cerebral malaria cases resulting in neurological sequelae (11.5%).9 Malaria mortality was estimated by applying a fixed case-fatality rate (13.6%; 95% confidence interval [CI] = 8.4% to 18.8%) for treated cases, three times this value for untreated cases4 to the number of severe malaria cases, based on published literature.10
Based on the mode of action of pre-erythrocytic malaria vaccines, the vaccine protective effect was modelled as a reduction in the force of infection, that is, a reduction in the probability of moving from state S (susceptible) to state I (infected). It should therefore be noted that the model input refers to the initial infection risk reduction. This differs from the clinical efficacy, which is based on the reduction in malaria symptomatic episodes over a follow-up period. Typically, larger values are obtained for percentage reduction in infection risk than for clinical efficacy. Vaccine efficacy against clinical malaria, as estimated by the model, was fitted to the vaccine efficacy observed in the Phase III trial simultaneously across the full follow-up period and in each 3-month follow-up period for three levels of transmission intensity. Compared to the previous publication,4 a new procedure based on a Monte Carlo Markov Chain generated with a Metropolis-Hastings algorithm was used for calibrating the vaccine effect, as described in Appendix File 2. For a three-dose schedule, the parameters were the following: 1) the initial reduction of the force of infection after three doses, 2) a parameter on decay of protection for a first phase and 3) for a second phase, and 4) the time between onset of these decay phases (transition time). For the fourth dose, calibrated parameters were the following: 5) the additional risk reduction of infection after the fourth dose and 6) a parameter on the decay of protection (only one phase). Parameter values and confidence intervals are shown in Table 1.
Table 1.
Input Parameters Used in the Model
| Parameter | Value | Source | |
|---|---|---|---|
| Probability of Asymptomatic Infection | Point Estimate | 95% CI | Fitted to Phase III Trial Data and Age-Distribution From Carneiro et al.6 |
| a1 | 8.57% | 0.29%, 17.79% | |
| a2 | 38.49% | 7.74%, 43.62% | |
| a3 | 38.58% | 7.94%, 43.82% | |
| a4 | 38.68% | 8.37%, 43.93% | |
| a5 | 38.90% | 15.65%, 44.12% | |
| a6 | 54.19% | 25.87%, 61.82% | |
| Percentage of Clinical Cases That Become Severe at Each Level of Immunitya | Point Estimate | 95% CI | Fitted to Phase III Trial Data and Age-Distribution From Carneiro et al.6 |
| f1 | 2.29% | 1.16%, 5.23% | |
| f2 | 2.14% | 1.16%, 3.01% | |
| f3 | 2.13% | 1.16%, 2.98% | |
| f4 | 2.08% | 1.16%, 2.92% | |
| f5 | 1.91% | 1.16%, 2.92% | |
| f6 | 1.31% | 0.96%, 1.92% | |
| Force of Infection | Point Estimates | 95% CI | |
| q, low transmission | 1.88 e-3 | 1.224 e-3, 2.217 e-3 | |
| q, moderate transmission | 30.50 e-3 | 19.222 e-3, 37.682 e-3 | |
| q, high transmission | 184.8 e-3 | 87.972 e-3, 272.147 e-3 | |
| Probability of full immunity, rimm | 1.97% | 1.93%, 2.17% | |
| Age-related susceptibility factor, k included in factor s = 1 − exp(−k*n) | 1.58 e-2 | 1.02 e-2, 3.63 e-2 | |
| Percentage of severe cases hospitalized | Country-specific | Assumed to be the same as access to ACT in public health facilities, obtained from DHS data or Malaria surveys. Average of values if not available | |
| Vaccine Efficacy Parameters Fitted | Point Estimates in Infants (95% CI) | Point Estimates in Children (95% CI) | |
| Three-dose efficacy half-life, phase 1 | 0.31 years (0.16-1.54) | 0.23 years (0.14-0.38) | |
| Three-dose efficacy half-life, phase 2 | 3.13 years (0.56-693) | 0.72 years (0.41-7.69) | |
| Fourth-dose efficacy half-life | 0.44 years (0.22-8.29) | 0.56 years (0.28-4.23) | |
| Phase transition time | 0.363 years (0.01-0.907) | 0.881 years (0.24-2.09) | |
| Initial infection risk reduction after three doses | 38.1% (30.2% to 49.9%) | 77.8% (68.0% to 90.3%) | |
| Additional infection risk reduction after the fourth dose | 31.2% (15.5% to 48.8%) | 19.9% (6.8% to 34.8%) | |
| Fixed Parameters | Value | Source | |
| Half-life for the waning of maternal protection, Wm | 3 months | Assumption | |
| Rate of recovery from clinical disease, r | 1/3 | Assumption, for a context of good access to care | |
| Relative risk for an untreated uncomplicated episode becoming severe (compared with a treated uncomplicated episode) | 1.84 (95% CI 1.68-2.01) | Calculated from the modelled number of severe cases, the % of severe cases hospitalized, and access to treatment | |
| % of severe malaria cases resulting in neurological sequelae | 1.7% (range = 0.85% to 2.54%) | Calculated | |
| Case-fatality rate as % of severe cases | |||
| Treated | 13.6% (95% CI 8.4% to 18.8%); 3× treated cases | Thwing10 | |
| Untreated | 40.8% (95% CI 25.2% to 56.4%) | Thwing10 | |
| DALY weights | |||
| Uncomplicated malaria | 0.211 for <5 years of age | Murray and Lopez (1996)18 | |
| Severe malaria | 0.195 for ≥5 years of age 0.6 | Assumption | |
| Neurological sequelae | 0.436 | Murray and Lopez (1996)18 | |
| Duration of DALY impact | |||
| Treated uncomplicated malaria | 4.8 days | Assumption | |
| Untreated uncomplicated malaria | 5 days | ||
| Treated severe malaria | 8.75 days | ||
| Untreated severe malaria | 17.5 days | ||
| Neurological sequelae | Life expectancy | WHO life tables | |
ACT, artemisinin-based combination therapy; DHS, Demographic and Health Surveys; CI, confidence interval; DALY, disability-adjusted life-year.
The number of severe cases was first generated assuming that all clinical episodes were treated. Then the country-specific relative risk factor for untreated severe cases (derived from the percentage of severe cases hospitalized) was applied to the untreated proportion of clinical episodes and added to the severe cases.
Appendix File 2 displays the results for the observed and modelled vaccine efficacy over time at each site in the Phase III trial, for infant and for child vaccination. Vaccine efficacy in the model was assumed to wane over time, as described in Appendix File 2. The calibration parameters for vaccine efficacy and its decay used in the model are shown in Table 1. Adverse events were not included in the model, because the only serious adverse event related to the vaccine is an additional risk of febrile convulsion (0.84 per 1,000 vaccinated children and 0.23 per 1,000 vaccinated infants), which would represent an extra cost of less than 0.1% of vaccination costs assuming each case would cause one day of hospitalization.
Model Simulation Procedure
The model was run at the individual level assuming a cohort of 10,000 individuals. First the baseline incidence without vaccination was derived for 12 different levels of parasite prevalence (from 3% up to 50%). For a given parasite prevalence level, individual-level heterogeneity of exposure to infected mosquitoes was captured by variation in the force of infection (based on a uniform distribution of parameter q). Also, values were randomly sampled from non-parametric distributions for parameters related to natural malaria history (ai, fi, rimm, k) to account for uncertainty around calibrated parameters values. For scenarios with vaccination, vaccine parameters were varied for each individual in the cohort as well (Initial risk reduction and waning) by randomly selecting a set of values from the non-parametric distribution shown in Appendix File 2, Figures A-2a (children) and A-2b (infants). The results were aggregated for the cohort and a given parasite prevalence level. This process is repeated 50 times for each of the 12 levels of parasite prevalence and each vaccination scenario in order to estimate the joint variability related to uncertainty on calibrated parameters for natural history of malaria and vaccine effect parameters.
Estimation of Country-Level Impact
The simulation of malaria incidence in each country was based on the country-specific parasite prevalence and demography at provincial level. The parasite prevalence level of the province was either matched with one of the 12 levels of prevalence simulated by the model, or an interpolation was made between two levels if required. Country-specific parasite prevalence data at the first administrative level (province) were obtained from MAP 2015,5 and demographic data were obtained from the United Nations.11,12 The proportion of clinical malaria cases in real-life settings with access to ACT was obtained from DHS data where available.7 For remaining countries the average was applied.4
Coverage for the first three doses of RTS,S vaccine candidate in infants was assumed to be the same as the country-specific coverage rate for the third dose of the diphtheria-tetanus-pertussis (DTP3) vaccine, obtained from WHO/United Nations Children’s Fund (UNICEF) 2017 estimates released in July 2018.13 In children, RTS,S vaccine candidate coverage rates were assumed to be the same as the country-specific coverage rate for the first dose measles vaccine given at 9 months of age. Coverage for the fourth RTS,S vaccine candidate dose was set at 80% of first three dose coverage to account for non-compliance.
The RTS,S vaccine candidate price was assumed to be USD5 per dose in the base case, ranging from USD2 to USD10 in sensitivity analysis similarly to previous published analysis.3 Vaccination implementation costs were derived from a study conducted in five African countries.14 When the fourth dose was administered in an outreach setting rather than at the health facility, the average implementation cost increased by a maximum of USD1.14 In the present analysis, the implementation cost in infants was based on three doses given in the health facility and one dose given as outreach. The published study reported only an infant schedule, and the implementation cost in children was extrapolated based on one dose given in the health facility and three doses given as outreach. In the base case, the cost per fully immunized child ranged from an average of USD26.08 in Burkina Faso to USD37.39 in Kenya (including the cost of the vaccine at USD5 per dose), and the cost per fully immunized infant was slightly lower (Table 2).
Table 2.
Cost per Fully Immunized Infant or Child in Five Countries
| Country | Cost per Fully Immunized Infant or Child, 4 Doses (USD) | ||
|---|---|---|---|
| Vaccine Price | |||
| USD5 per Dose | USD2 per Dose | USD10 per Dose | |
| Children aged 5–17 months | |||
| Ghana | 28.28 | 12.48 | 56.64 |
| Kenya | 37.39 | 22.93 | 61.07 |
| Mozambique | 28.33 | 15.19 | 55.80 |
| Tanzania | 30.08 | 15.47 | 54.34 |
| Burkina Faso | 26.08 | 11.89 | 53.73 |
| Infants aged 6–12 weeks using Expanded Program for Immunization visits | |||
| Ghana | 26.77 | 11.56 | 53.34 |
| Kenya | 37.14 | 22.80 | 60.75 |
| Mozambique | 27.56 | 14.40 | 55.90 |
| Tanzania | 29.34 | 14.96 | 52.79 |
| Burkina Faso | 25.64 | 11.69 | 51.05 |
Costs and disability-adjusted life-years (DALYs) were discounted at 3% per year. Age weighting was not applied for DALYs. All costs were reported in 2015 US dollars and were inflated with the country-specific consumer price index when necessary.
The model followed the simulated 2017 birth cohort from birth to 15 years of age with a cycle time of 5 days, comparing the following strategies:
Without malaria vaccination;
Four doses of RTS,S vaccination in infants, administered at ages 6, 10, and 14 weeks with a fourth dose at age 21 months;
Four doses of RTS,S vaccination in children, administered at ages 6, 7.5, and 9 months with a fourth dose at age 27 months.
Model outcomes (e.g., number of malaria cases, severe malaria cases, malaria hospitalizations, DALYs, costs and incremental cost-effectiveness ratio [ICER]) were accumulated over the 15-year simulation period, thereby allowing to cover the full effect of vaccination also after protection has waned. The impact of neurological sequelae and deaths were accounted for over a lifetime.
The analysis was conducted from both the health system and the societal perspectives. Costs include vaccination costs (vaccine price plus administration costs) and treatment costs. Treatment costs to the health system were defined according to a previously published study in three sub-Saharan African countries,15 and included visits to healthcare facilities, hospitalizations, and resources used for treatment (tests and medications). The analysis from the societal perspective also included costs to the household based on the same study,15 which included costs of traditional treatment and productivity losses.
Sensitivity Analysis
One- and two-way sensitivity analyses were conducted by varying the values of key parameters and examining the impact on the ICER in the health system perspective. For one-way sensitivity analyses, parameters are varied one at a time, while holding the values of all other parameters in the model constant. The two-way analyses combined the impact of changing two parameters simultaneously: price and prevalence, price and effectiveness (combining the initial risk reduction effect of the vaccine and its waning), and prevalence and effectiveness, at the base case discounting rates. An undiscounted two-way price/effectiveness sensitivity analysis was also performed.
Prevalence was varied using the confidence interval from malaria MAP project. Access to ACT was varied according the interquartile range of access data across countries from 18% to 39% and access to hospital for severe cases was varied over the same range. The multiplication factor used to estimate the case-fatality rate in untreated severe malaria cases, which was 3 in the base case (i.e., in the base case the case-fatality rate in untreated severe malaria cases was set at 3× the rate in treated severe malaria cases), was varied from 2 to 4. The minimum and maximum values for hospital-based mortality were 0.084 and 0.188, respectively, based on the 95% CI reported in the original source.10 Vaccine price was varied from USD2 to USD10 per dose (base-case value = USD5), and the discount rate for both costs and health benefits was set at 0% and 6% in the sensitivity analysis (base case 3%).
A probabilistic sensitivity analysis (PSA) was conducted on the ICER for the child and infant vaccination strategies by simultaneously varying all probability parameters included in the one-way sensitivity analysis assuming a beta distribution with 95% confidence interval corresponding to the ranges described in the one-way sensitivity analysis. The discount rate was not varied in the PSA.
A further sensitivity analysis was conducted, for child vaccination only, by including in the model only regions with parasite prevalence of >5%, or >10%, according to the MAP Admin-1 data. We investigated this because an earlier study using the same model reported that the cost-effectiveness of vaccination varied with the parasite prevalence level.3
Budget Impact Analysis
The budget impact analysis estimated the annual total cost of each of the malaria vaccination strategies from the perspective of the health system for each country. The total cost for a given year included vaccine price plus administration cost, minus savings in malaria treatment costs for that year. The total annual cost was calculated for the first 15 years of vaccine introduction, thereby accounting for savings in malaria treatment costs from the consecutively growing number of vaccinated ageing cohorts. No discounting was applied.
Results
Public Health Impact
Table 3 shows the results of the base case public health impact estimates across all 41 countries combined.
Table 3.
Events and Disability-Adjusted Life-Years (DALY) Averted of Infant or Child RTS,S Vaccination Across All 41 countries, Base Case
| Events Averted Over 15-Year Follow-up Period | Events Averted Over 15-Year Follow-up Period per 1,000 Vaccinees | |||||
|---|---|---|---|---|---|---|
| Median | 95% Confidence Interval | Median | 95% Confidence Interval | |||
| Lower Bound | Upper Bound | Lower Bound | Upper Bound | |||
| Child vaccination (doses at 6, 7.5, and 9 months with a fourth dose at 27 months) | ||||||
| Number vaccinated | 24,569,548 | |||||
| Clinical malaria cases | 16,764,732 | 14,236,975 | 19,382,566 | 682 | 579 | 789 |
| Severe malaria cases | 359,962 | 176,314 | 542,284 | 14.7 | 7.2 | 22.1 |
| Malaria hospitalizations | 192,213 | 95,727 | 288,158 | 7.8 | 3.9 | 11.7 |
| Malaria deaths | 112,881 | 55,011 | 170,306 | 4.6 | 2.2 | 6.9 |
| DALYs averted (discounted) | 3,385,585 | 2,170,699 | 4,792,303 | 138 | 88.3 | 195 |
| Infant vaccination (doses at 6, 10, and 14 weeks with a fourth dose at 21 months) | ||||||
| Number vaccinated | 26,212,458 | |||||
| Clinical malaria cases | 15,980,852 | 13,399,059 | 18,656,822 | 610 | 511 | 712 |
| Severe malaria cases | 340,683 | 156,343 | 532,447 | 13.0 | 6.0 | 20.3 |
| Malaria hospitalizations | 181,187 | 83,983 | 282,447 | 6.9 | 3.2 | 10.8 |
| Malaria deaths | 106,965 | 48,940 | 167,302 | 4.1 | 1.9 | 6.4 |
| DALYs averted (discounted) | 3,158,769 | 1,917,650 | 4,610,007 | 121 | 73.2 | 176 |
Vaccination of one cohort of children with four doses of the RTS,S vaccine candidate in addition to existing malaria interventions was projected to avert 16.8 million cases of malaria and almost 113,000 malaria deaths over the 15-year follow-up period, compared with no vaccination. With a strategy of vaccinating infants, the projected impact of adding vaccination to existing strategies would be 16 million cases of clinical malaria and 107,000 malaria deaths averted, compared with no vaccination. Child vaccination would also avert more DALYs than for infant vaccination, 3.4 million versus 3.2 million DALYs averted. The larger number of malaria events averted and DALYs averted with the strategy of vaccinating children, compared to a strategy of vaccinating infants, reflects the higher efficacy of the RTS,S vaccine candidate in children observed in the RTS,S Phase III clinical trial. For a few countries (e.g., Angola, Ethiopia, Gabon, Madagascar, Sierra Leone, and South Sudan), a higher number of DALYs averted was estimated for infant vaccination. This resulted from the higher vaccination coverage in infants (based on DTP3 coverage) compared with children (based on measles vaccination coverage). A more detailed table with estimates for each country is available in Appendix File 3.
Countries with the largest number of malaria cases and deaths averted over 15 years, for both the child and infant vaccination strategies, were Nigeria (3.1 million vaccinated children, 3.6 million cases, and 21,300 deaths averted with child vaccination; 3.3 million vaccinated infants, 3.3 million cases, and 19,800 deaths averted with infant vaccination) and the Democratic Republic of Congo (2.4 million children vaccinated, 2.5 million cases, and 17,600 deaths averted with child vaccination; 2.5 million infants vaccinated, 2.3 million cases, and 16,400 deaths averted with infant vaccination). Details of country results are given in Appendix File 3.
Cost-Effectiveness Analysis: Base Case
Table 4 shows the results of the cost offset and cost-effectiveness analysis from the health care system and societal perspectives across all 41 countries combined. Overall, discounted vaccination costs were about USD697 million and USD729 million, discounted health system cost offsets were USD19.8 million and USD18.4 million, and societal cost offsets were USD65.6 million and USD61.0 million for child and infant vaccination strategy, respectively. The lower vaccination costs in child vaccination are caused by a smaller target population (due to infant mortality) and an overall lower vaccination coverage (coverage of measles vaccination versus DTP3 coverage in infants). However, in 9 out of 41 countries, where coverage of measles vaccination exceeds DTP3 coverage, there are more vaccinees with a child vaccination schedule than the infants schedule.
Table 4.
Vaccination Costs, Costs Offset, and Cost-Effectiveness With Child and Infant Vaccination (in 2015 US Dollars)
| Child Vaccination | Infant Vaccination | |||||
|---|---|---|---|---|---|---|
| Median | 95% Confidence Interval | Median | 95% Confidence Interval | |||
| Lower Bound | Upper Bound | Lower Bound | Upper Bound | |||
| Vaccination costs (discounted) | 697,345,540 | — | — | 729,228,602 | — | — |
| Health system costs offset (discounted) | 19,780,949 | 16,495,033 | 23,594,247 | 18,370,025 | 15,110,234 | 22,339,527 |
| Societal costs offset (discounted) | 65,647,274 | 55,969,131 | 76,910,204 | 60,950,608 | 51,380,486 | 72,680,804 |
| Incremental cost-effectiveness ratio in USD per DALY averted (health system, discounted) | 200 | 141 | 314 | 225 | 153 | 372 |
| Incremental cost-effectiveness ratio in USD per DALY averted (societal, discounted) | 187 | 129 | 295 | 212 | 142 | 353 |
DALY, disability-adjusted life-year.
The projected ICER from the health system perspective at a vaccine price of USD5 per dose with both costs and effects discounted was USD200 per DALY averted for child vaccination and USD225 for infant vaccination. As for the DALYs averted, the more favorable ICER for the child vaccination strategy compared with the infant vaccination strategy is driven by the higher effectiveness of the child vaccination strategy. The overall ICER across countries would represent 14% of the onetime gross domestic product (GDP) per capita threshold (weighted by the country population size) for children and about 17% for infants.
Within countries, the ICER was consistently lower (more favorable) for vaccination of children than infants. There are however considerable differences between countries. In the health care system perspective, the ICER ranged from USD89 (Côte d’Ivoire) to USD8,414 (Djibouti) per DALY averted for child vaccination and from USD99 (Côte d’Ivoire) to USD9,068 (Djibouti) for infant vaccination. Except for 6 countries (Ethiopia, Djibouti, Eritrea, Somalia, Gambia, and Rwanda ordered by decreasing ratio ICER on GDP per capita), the country-specific ICER for both child and infant vaccination was under the threshold of one time GDP per capita in both health system and societal perspectives, and under the threshold of 3× GDP per capita for 39 out of 41 countries.
When comparing both vaccination schedules, child vaccination dominated (less costly and more DALYs averted) infant vaccination in 23 of 41 countries. In 12 countries, infant vaccination resulted in lower incremental costs versus child vaccination but also in less DALYs averted. In those countries, the ICER of child versus infant vaccination was however always below 1× GDP per capita except for Djibouti where none of the vaccination strategies are cost-effective. In six countries, more DALYs were averted with infant vaccination versus child vaccination following the higher number of vaccinated infants based on DTP3 coverage assumption (15% to 35% more vaccinees). In those countries, the ICER of infant versus child vaccination was under 1× GDP per capita for three countries (Angola, Gabon, and South Sudan) but was above that threshold for the other three countries (Ethiopia, Madagascar, and Sierra Leone) with vaccination in Ethiopia not being cost-effective at a 3× GDP per capita threshold.
Details for each country are given in the Appendix File 3, including the ratio of the ICER on the GDP per capita for each vaccination strategy and the comparison of both schedules for each country.
Cost-Effectiveness Analysis: Sensitivity Analysis
Figure 2 shows the effect of varying the values for key parameters in the model on the ICER for child (A) and infant (B) vaccination, either with two parameters simultaneously or one at a time, as described in the Methods section.
Figure 2.
One and two-way sensitivity analysis, cost-effectiveness: (A) In children; (B) In Infants
ACT, artemisinin-based combination therapy; CFR, case-fatality rate; DALY, disability-adjusted life-year; HB, higher bound; LB, lower bound; MAP, Malaria Atlas Project; PE, point estimate; RR, relative risk; y, year.
For child vaccination, the parameters with the largest impact on the estimated ICER in the health system perspective were the vaccine price, discount rate, parasite prevalence, hospital-based mortality, and vaccine effect parameters. Varying the costs, DALYs, risk of sequelae, life expectancy, and percentage with access to ACT had smaller effects on the results. Similar results were obtained with infant vaccination, except for a higher variation due to the uncertainty around vaccine effectiveness and waning. This parameter had a slightly larger impact than prevalence variability.
In the two-way sensitivity analysis, when combining the variability of two parameters simultaneously at their “worst case scenario,” that is, the highest price (USD10 per dose) combined with the lowest bounds (LB) of parasite prevalence (MAP LB), the highest ICER of USD660 per DALY averted was obtained for child vaccination, which is still below a cost-effectiveness threshold of 1 GDP per capita across all 41 countries. The ICER did not increase to the same extent, however, when no discounting was applied or when vaccination was restricted to areas with more than 10% parasite prevalence. Combined varying price/effectiveness and prevalence/effectiveness resulted in a comparable ICER around USD590 per DALY averted in the worst case. Similarly, for infant vaccination, the largest ICER estimate was up to USD750 per DALY averted when assuming a higher price combined with lowest bounds of parasite prevalence. This ICER would remain below a cost-effectiveness threshold of 1 GDP per capita across all 41 countries.
The PSA results also confirmed that for all simulations the ICER does not exceed the 1 GDP per capita threshold across all 41 countries for both child and infant vaccination (Figure 3). The cost-effectiveness thresholds corresponding to 80% probability to be cost-effective were USD296 and USD332 per DALY averted for child and infant vaccination, respectively.
Figure 3.
Probabilistic sensitivity analysis, cost-effectiveness acceptability curve.
When vaccination in children was confined to regions with parasite prevalence above a threshold of 5% according to MAP Admin-1 data, the estimated cost-effectiveness of child vaccination from the health system perspective was USD148 per DALY averted, with over 16.2 million malaria cases and 106,000 malaria deaths averted over a cohort time horizon of 15 years with 17.6 million children vaccinated instead of 24.6 million in the base case. With vaccination confined to regions with parasite prevalence above a threshold of 10%, the estimated cost-effectiveness of child vaccination was USD125 per DALY averted, with 15.2 million malaria cases and over 97,000 malaria deaths averted over a cohort time horizon of 15 years with 13.8 million children vaccinated.
Budget Impact Analysis
The estimated budget impact for the first year of child and infant vaccination over 41 countries were USD554 million and USD575 million, respectively. For the second year, the budget of the infant vaccination strategy increased to USD725 million due to the fourth dose given at 21 months. In the child vaccination strategy with the fourth dose given at 27 months, this increase only appeared in the third year, up to USD688 million. Thereafter the budget decreased for both vaccination strategies because of health care cost offsets. The countries with the largest budget impact of vaccination were Nigeria (USD69.4 million at year 1 and USD84.6 million at year 3 for child vaccination, USD70.4 million at year 1 and USD88.2 million at year 2 for infant vaccination), the Democratic Republic of Congo (USD52.8 million at year 1 and USD65.3 million at year 3 for child vaccination, USD54.9 million at year 1 and USD69.0 million at year 2 for infant vaccination), and Ethiopia (USD46.2 million at year 1 and USD58.6 million at year 3 for child vaccination, USD51.9 million at year 1 and USD65.7 million at year 2 for infant vaccination), reflecting population size and vaccination coverage. Across countries, the budget impact was generally larger for infant vaccination than for child vaccination, reflecting the higher number of vaccinated infants and lower vaccine efficacy resulting in less cost offsets. However, for 15 countries, infant vaccination had a lower budget impact, which can be explained by a lower number of vaccinees in 9 out of 15 countries compared with child vaccination and by a lower cost of vaccination resulting in a marginally lower budget in the remaining 6 countries.
Detailed country-specific budget estimates are provided in Appendix File 3.
Discussion
This modelling analysis estimated the cost-effectiveness and budget impact of adding either child or infant vaccination with four doses of the RTS,S vaccine candidate to existing malaria interventions in 41 countries in sub-Saharan Africa. Despite the lower coverage, the child vaccination strategy was projected to avert more cases and deaths than infant vaccination. Across the countries modelled, child vaccination was consistently more cost-effective than infant vaccination; however, child vaccination dominated infant vaccination in 23 countries and provided less DALYs averted in 6 countries. For most countries, the budget impact was higher for infant vaccination except in countries with higher coverage in children. Confining vaccination only to regions with high parasite prevalence was projected to improve the estimated cost-effectiveness of child vaccination further, while only slightly reducing its overall public health impact.
The vaccine price is considered without external funding from a third party such as donors or the GAVI Alliance. External funding supporting the malaria vaccine would improve the cost-effectiveness results from the country perspective.
Vaccination of children is likely to be more expensive per subject than vaccination of infants, because in infants the vaccine can be administered at routine Expanded Program on Immunization (EPI) visits, whereas in children additional clinic visits are required specifically for RTS,S vaccination. Based on a study in five sub-Saharan African countries, the cost of vaccination in the model was higher in children than infants, although the difference was modest. The results of the present analysis indicate that the improved effectiveness of the vaccine when administered to children rather than infants would outweigh the additional cost, resulting in better cost-effectiveness for vaccination of children compared with infants. It may be possible for the fourth dose to be given at age 24 months, instead of 27 months, allowing it to be combined with administration of the second dose of the measles vaccine, which could reduce the cost of administration of child vaccination. Furthermore, it is uncertain whether the expectation of lower costs for infant vaccination due to administration at EPI visits remains realistic, as immunization visits are already crowded. Administration of three or four injections during the same visit may not be practical, suggesting that even infant vaccination may need additional visits or might result in the delay of some recommended vaccines. Child malaria vaccination administered at older ages beyond the existing EPI visits may offer new opportunities for improving healthcare, such as improving coverage through catch-up of missed doses of other vaccines. Any such additional benefits have not been accounted for in this analysis.
If the administration cost of child vaccination would be USD3.60 higher per vaccinated child compared with the base case, the overall ICER of the child vaccination strategy would be similar to the ICER of infant vaccination. Although there is uncertainty over coverage and administration costs, it is likely that child vaccination would still have better projected cost-effectiveness than infant vaccination and would have at least a similar projected impact.
These results are consistent with previously published results on the projected public health impact of RTS,S vaccination from the same model using clinical trial data with 18 months of follow-up.4 The earlier analysis predated the availability of results from the fourth vaccine dose, and therefore included only three vaccine doses. The present analysis reported larger estimates due to higher vaccination coverage in the largest countries and the effect of including the fourth dose in the model. A comparison of four different models, one of which was the model used in the present analysis, estimated the median cost-effectiveness at USD87 per DALY averted in the health system perspective for a four-dose vaccination schedule in children (doses at 6, 7.5, 9, and 27 months, the same schedule as the present analysis) at a vaccine price of USD5 per dose.3 This is comparable to the base-case estimate of USD98 per DALY averted for child vaccination when we use the same discounting assumptions as in the comparison publication.
The model has a number of limitations. It is not a dynamic model and thus cannot take account of any herd immunity effects. However, herd immunity is not expected for a partially efficacious pre-erythrocytic vaccine for which implementation is limited to young children only. Only a fraction of malaria episodes would be averted in children and a large part of transmission is attributable to adults with parasitemia who will continue to infect mosquitoes.16,17 It does not distinguish between different types of severe malaria, other than to include the long-term neurological sequelae that can result from cerebral malaria. Different types of severe malaria may differ in terms of outcomes and resource use, and any such variation would not be captured in the current model. The model also assumes that malaria transmission remains stable over the full 15-year follow-up period, and thus does not take account of any potential changes in access to or use of other interventions, any changes in vaccine coverage or any changes in insecticide or treatment resistance. The model did not take account of any vaccine adverse events, because the extra cost considered for managing vaccine-related adverse events would be marginal in comparison with vaccine costs. Estimating disease management costs from the health system perspective is difficult due to the various co-payments incurred when patients access public health care. This analysis relies on a single multi-country study.15 However, variation in the health system unit costs has a very limited impact on the cost-effectiveness results as shown in the sensitivity analysis.
The sensitivity analysis indicated that the results were sensitive to the values used for vaccine price, the discounting rates, parasite prevalence and hospital-based mortality, and vaccine efficacy. With the two-way sensitivity analysis, the combined effect of a higher vaccine price and low parasite prevalence had the most important impact by increasing the ICER to USD660 per DALY averted. The combine effect of higher price/lower effectiveness as well as low prevalence/lower effectiveness resulted in a comparable ICER increase to about USD590 per DALY averted. In the worst-case scenarios and the PSA, the estimated ICER remained below the cost-effectiveness threshold across all 41 countries. Long-term vaccine effectiveness was modelled using a mathematical function to extrapolate the clinical trial results generated over 3 to 4 years of follow-up in total to the 15-year period used in the model. The analysis should be updated with longer-term real-world data as more information becomes available in the future.
Conclusion
Addition of child or infant vaccination with four doses of the RTS,S vaccine candidate to existing malaria interventions in sub-Saharan Africa is expected to be cost-effective in 35 out of 41 countries using a cost-effectiveness threshold of 1× GDP per capita. However, a strategy of vaccinating infants was estimated to be less cost-effective than a strategy of vaccinating children, owing to the lower effectiveness of the RTS,S vaccine candidate in infants than in children.
Supplemental Material
Supplemental material, Appendix_file_1_online_supp_Revised_MXB for Economic Impact of Introducing the RTS,S Malaria Vaccine: Cost-Effectiveness and Budget Impact Analysis in 41 Countries by Christophe Sauboin, Laure-Anne Van Bellinghen, Nicolas Van De Velde and Ilse Van Vlaenderen in MDM Policy & Practice
Supplemental Material
Supplemental material, Appendix_file_2_online_supp_Revised_MXB for Economic Impact of Introducing the RTS,S Malaria Vaccine: Cost-Effectiveness and Budget Impact Analysis in 41 Countries by Christophe Sauboin, Laure-Anne Van Bellinghen, Nicolas Van De Velde and Ilse Van Vlaenderen in MDM Policy & Practice
Supplemental Material
Supplemental material, Appendix_file_3_online_supp_Revised_MXB for Economic Impact of Introducing the RTS,S Malaria Vaccine: Cost-Effectiveness and Budget Impact Analysis in 41 Countries by Christophe Sauboin, Laure-Anne Van Bellinghen, Nicolas Van De Velde and Ilse Van Vlaenderen in MDM Policy & Practice
Supplemental Material
Supplemental material, Focus_On_Patient_online_supp for Economic Impact of Introducing the RTS,S Malaria Vaccine: Cost-Effectiveness and Budget Impact Analysis in 41 Countries by Christophe Sauboin, Laure-Anne Van Bellinghen, Nicolas Van De Velde and Ilse Van Vlaenderen in MDM Policy & Practice
Acknowledgments
The authors would like to thank Carole Nadin (Fleetwith Ltd, on behalf of GSK) for medical writing assistance and Business & Decision Life Sciences platform for editorial assistance and manuscript coordination, on behalf of GSK. Fabien Debailleul coordinated publication development and editorial support.
Footnotes
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: GlaxoSmithKline Biologicals S.A. funded this study (GSK Study Identifier: HO1414956) and all costs related to the development of the related publications. Christophe Sauboin and Nicolas Van de Velde are employees of the GSK group of companies and hold shares in the GSK group of companies. Ilse Van Vlaenderen and Laure-Anne Van Bellinghen report that the GSK group of companies paid consulting fees to CHESS for its contribution to model development as well as consulting fees for other projects commissioned to CHESS by the GSK group of companies.
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Authors’ Note: Previous presentations of data: ISPOR, 18th Annual European Congress, Milan, Italy, November 7–11, 2015; ASTMH, 64th Annual Meeting, Philadelphia, PA, USA, October 25–29, 2015; The African Health Economics & Policy Association—The 4th Biennial Scientific Conference, Rabat, Morocco, September 26–29, 2016.
Author Contributions: All authors comply with the ICMJE criteria for authorship. CJS provided substantial scientific input to the study, participating in the method selection and development, model elaboration and population, determination of inputs and the acquisition of data, statistical data analysis and the sensitivity analysis, elaboration of the study report and its critical review. LAVB and IVV provided substantial scientific input to the previously published study and model, critically reviewing those model inputs and study report. NVDV provided substantial scientific input to the study, participating in the determination of model settings and inputs. All authors read and approved the final manuscript.
ORCID iD: Christophe Sauboin
https://orcid.org/0000-0003-0913-039X
Supplemental Material: Supplementary material for this article is available on the Medical Decision Making Policy & Practice website at https://journals.sagepub.com/home/mpp.
Contributor Information
Christophe Sauboin, GSK, Wavre, Belgium.
Laure-Anne Van Bellinghen, CHESS In Health, Bonheiden, Belgium.
Nicolas Van De Velde, GSK, Wavre, Belgium; ViiV Healthcare, London, UK.
Ilse Van Vlaenderen, CHESS In Health, Bonheiden, Belgium.
References
- 1. World Health Organization. World Malaria Report 2016 [cited August 22, 2017]. Available from: http://www.who.int/malaria/publications/world-malaria-report-2016/report/en/
- 2. RTS, S Clinical Trials Partnership. Efficacy and safety of RTS,S/AS01 malaria vaccine with or without a booster dose in infants and children in Africa: final results of a phase 3, individually randomised, controlled trial. Lancet. 2015;386(9988):31–45. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Penny MA, Verity R, Bever CA, et al. Public health impact and cost-effectiveness of the RTS,S/AS01 malaria vaccine: a systematic comparison of predictions from four mathematical models. Lancet. 2016;387(10016):367–75. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Sauboin CJ, Van Bellinghen LA, Van De Velde N, Van Vlaenderen I. Potential public health impact of RTS,S malaria candidate vaccine in sub-Saharan Africa: a modelling study. Malar J. 2015;14:524. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Malaria Atlas Project. Plasmodium falciparum parasite rate in 2-10 year olds 2015 [cited April 14, 2017]. Available from: http://www.map.ox.ac.uk/explorer/
- 6. Carneiro I, Roca-Feltrer A, Griffin JT, et al. Age-patterns of malaria vary with severity, transmission intensity and seasonality in sub-Saharan Africa: a systematic review and pooled analysis. PLoS One. 2010;5(2):e8988. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. STATcompiler. Available from: http://statcompiler.com/en/
- 8. Reyburn H, Mbatia R, Drakeley C, et al. Association of transmission intensity and age with clinical manifestations and case fatality of severe Plasmodium falciparum malaria. JAMA. 2005;293(12):1461–70. [DOI] [PubMed] [Google Scholar]
- 9. World Health Organization. Severe malaria. Trop Med Int Health. 2014;19(Suppl. 1):7–131. [DOI] [PubMed] [Google Scholar]
- 10. Thwing J, Eisele TP, Steketee RW. Protective efficacy of malaria case management and intermittent preventive treatment for preventing malaria mortality in children: a systematic review for the Lives Saved Tool. BMC Public Health. 2011;11(Suppl. 3):S14. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. United Nations Department of Economic and Social Affairs. World population prospects 2019. [cited May 29, 2015]. Available from: http://esa.un.org/unpd/wpp/index.htm
- 12. World Bank; World Health Organization; UN Children’s Fund. Levels and trends in child mortality: 2012 report [cited August 22, 2017]. Available from: https://reliefweb.int/report/world/levels-and-trends-child-mortality-2012-report
- 13. World Health Organization. WHO/UNICEF estimates of national immunization coverage 2015. [cited April 10, 2017]. Available from: http://www.who.int/entity/immunization/monitoring_surveillance/data/coverage_estimates_series.xls?ua=1
- 14. Sicuri E, Bocoum FY, Nonvignon J, et al. The costs of implementing vaccination with the RTS,S malaria vaccine in five sub-Saharan African countries. Medical Decision Making Policy & Practice. 2019. DOI: 10.1177/2381468319896280. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Sicuri E, Vieta A, Lindner L, Constenla D, Sauboin C. The economic costs of malaria in children in three sub-Saharan countries: Ghana, Tanzania and Kenya. Malar J. 2013;12:307. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Smith T, Ross A, Maire N, et al. Ensemble modeling of the likely public health impact of a pre-erythrocytic malaria vaccine. PLoS Med. 2012;9(1):e1001157. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Churcher TS, Bousema T, Walker M, et al. Predicting mosquito infection from Plasmodium falciparum gametocyte density and estimating the reservoir of infection. Elife. 2013;2:e00626. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. Murray C.J.L., Lopez A.D. (1996) The Global Burden of Disease: A Comprehensive Assessment of Mortality and Disability from Diseases, Injuries and Risk Factors in 1990 and Projected to 2020. Harvard School of Public Health, Boston. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Supplemental material, Appendix_file_1_online_supp_Revised_MXB for Economic Impact of Introducing the RTS,S Malaria Vaccine: Cost-Effectiveness and Budget Impact Analysis in 41 Countries by Christophe Sauboin, Laure-Anne Van Bellinghen, Nicolas Van De Velde and Ilse Van Vlaenderen in MDM Policy & Practice
Supplemental material, Appendix_file_2_online_supp_Revised_MXB for Economic Impact of Introducing the RTS,S Malaria Vaccine: Cost-Effectiveness and Budget Impact Analysis in 41 Countries by Christophe Sauboin, Laure-Anne Van Bellinghen, Nicolas Van De Velde and Ilse Van Vlaenderen in MDM Policy & Practice
Supplemental material, Appendix_file_3_online_supp_Revised_MXB for Economic Impact of Introducing the RTS,S Malaria Vaccine: Cost-Effectiveness and Budget Impact Analysis in 41 Countries by Christophe Sauboin, Laure-Anne Van Bellinghen, Nicolas Van De Velde and Ilse Van Vlaenderen in MDM Policy & Practice
Supplemental material, Focus_On_Patient_online_supp for Economic Impact of Introducing the RTS,S Malaria Vaccine: Cost-Effectiveness and Budget Impact Analysis in 41 Countries by Christophe Sauboin, Laure-Anne Van Bellinghen, Nicolas Van De Velde and Ilse Van Vlaenderen in MDM Policy & Practice



