‘It takes two to tango’: Bridging the gap between country need and vaccine product innovation

Rachel A Archer; Ritika Kapoor; Wanrudee Isaranuwatchai; Yot Teerawattananon; Birgitte Giersing; Siobhan Botwright; Jos Luttjeboer; Raymond C W Hutubessy

doi:10.1371/journal.pone.0233950

. 2020 Jun 10;15(6):e0233950. doi: 10.1371/journal.pone.0233950

‘It takes two to tango’: Bridging the gap between country need and vaccine product innovation

Rachel A Archer ^1,^*, Ritika Kapoor ², Wanrudee Isaranuwatchai ^1,³, Yot Teerawattananon ^1,², Birgitte Giersing ⁴, Siobhan Botwright ⁴, Jos Luttjeboer ^5,⁶, Raymond C W Hutubessy ⁴

Editor: Ray Borrow⁷

PMCID: PMC7286512 PMID: 32520934

Abstract

Background

Despite a growing global commitment to universal health coverage, considerable vaccine coverage and uptake gaps persist in resource-constrained settings. One way of addressing the gaps is by ensuring product innovation is relevant and responsive to the needs of these contexts. Total Systems Effectiveness (TSE) framework has been developed to characterize preferred vaccine attributes from the perspective of country decision-makers to inform research and development (R&D) of products. A proof of concept pilot study took place in Thailand in 2018 to examine the feasibility and usefulness of the TSE approach using a rotavirus hypothetical test-case.

Methods

The excel-based model used multiple-criteria decision analysis (MCDA) to compare and evaluate five hypothetical rotavirus vaccine products. The model was populated with local data and products were ranked against decision criteria identified by Thai stakeholders. A one-way sensitivity analysis was performed to identify criteria that influenced vaccine ranking. Self-assessment forms were distributed to R&D stakeholders on the usability of the approach and were subsequently analysed.

Results

The model identified significant parameters that impacted on MCDA rankings. Self-assessment forms revealed that TSE was perceived as being able to encourage closer collaboration between country decision makers and vaccine developers.

Conclusions

The pilot study demonstrates that it is feasible to use an MCDA approach to elicit stakeholder preferences and determine influential parameters to help identify the preferred product characteristics for R&D from the perspective of country decision-makers. It found that TSE can help steer manufacturers to develop products that are better aligned with country need. Findings will guide further development of the TSE concept.

Introduction

All United Nations Member States have agreed to accelerate progress towards universal health coverage (UHC) by 2030 [1]. Today, at least half of the world’s population do not have full access to essential health services. Each year, 20 million infants do not receive their full course of recommended vaccines, resulting in approximately 1.5 million deaths from vaccine preventable diseases. Over 50% of these infants live in six countries with the weakest health systems infrastructure [2]. In terms of inequitable health coverage, it is estimated that 100 million people live in extreme poverty due to out of pocket expenditure on health [3]. Effective global immunisation is essential to attaining the UHC goal, however to achieve this, vaccines must be designed suitable for use in resource constrained settings. To have the greatest public health impact, the product attributes of vaccines that are intended for deployment in low-and middle-income countries (LMICs) must be informed by the needs of LMIC vaccine delivery systems.

LMICs are confronted with unique challenges when implementing successful immunisation programmes [4–6]. Persisting difficulties include but are not limited to: logistical complexity, barriers and gaps in delivery systems, constraints in human resource and challenges in prioritisation of vaccines over other competing interventions. Issues of ensuring timely delivery and appropriate storage conditions to preserve vaccine potency and safety prevail. In addition, the emergence and introduction of new vaccines, improved vaccine products and novel delivery technologies requires that policy and decision makers have the information and tools to assess and select the products that will be of greater impact for their programmes, after product licensure as well as during product development, to ensure that product characteristics are aligned with country need.

The above challenges faced by each country may vary significantly based on their geographical, healthcare, and political constraints [7]. This impacts the priorities and goals for vaccine programs of every country and differentiated vaccine product characteristics may be desirable to overcome these constraints [8]. Despite wide acknowledgment of these practicalities, there is no clear methodological approach of identifying the vaccine product attributes that would address these context specific barriers, and communicating these country priorities to inform vaccine R&D by manufacturers.

R&D describes the discovery and technical advancement of a new or existing technology [9]. The development of novel vaccine products and innovative delivery mechanisms is vital to address to the current programmatic challenges and combat the threats of emerging and existing infectious diseases. Vaccine R&D is not an easy endeavor; vaccines are complex and costly biological preparations. The development and introduction of one new vaccine product requires immense amounts of money, time, and resources and there is no guarantee of a return on this investment [10, 11]. Aligning the vaccine design and development initiatives with the government vaccine product preferences will likely help to incentivise and accelerate the development of vaccines that are optimal for local contexts, but these vaccines may also have broader potential for uptake and impact at the regional or global level. Designing new vaccine products that better anticipate market demand and challenges and are more responsive to government needs, will decrease the risk in investment, and increase the potential for return.

Early stage Health Technology Assessment (HTA) is the evaluation of new technologies in development to determine their potential value and provides an opportunity to facilitate dialogue between government and market actors. Performing such an evaluation with inputs from key stakeholders including decision makers, immunisation programme managers, clinical experts, and manufacturers can help to identify preferred vaccine attributes, thereby informing R&D about the key needs of innovation and accelerating product uptake [12].

The World Health Organization (WHO) and partners are developing the concept of Total Systems Effectiveness (TSE), which aims to strengthen structured and transparent vaccine product selection processes at the country level and to build a platform for stakeholders in LMIC immunisation programmes to shape vaccine supply and R&D. TSE has since been renamed Capacity-led Assessment for Prioritisation on Immunisation (CAPACITI) [13]. The aim of the TSE approach is to foster a stronger collaboration between industry and country decision makers to develop products that are more closely aligned with the needs and context of LMICs.

The WHO TSE framework is designed to support the evaluation of different vaccine products for product selection at the country level and is based on multi-criteria decision fanalysis (MCDA), in order to consider trade-offs between different product characteristics. Whilst WHO envisages utilising the tools within its TSE framework to assess vaccine products, the framework approach itself is based on a generic concept that could be adapted to other applications, e.g. drugs and medical devices. Whilst the literature reports an increase in application of MCDA for healthcare evaluations related to a range of decisions including reimbursement, prescription and resource allocation decisions, there is limited documented use of MCDA to identify priorities for R&D based on LMIC need [14–16].

To test the assertion that the TSE approach could inform R&D of vaccine products for LMICs, WHO, the Health Intervention and Technology Assessment Program (HITAP) and National University of Singapore (NUS), piloted TSE in Thailand between April and August 2018. The objective of this article is to explore the application of TSE approach to identify the preferred product characteristics of a vaccine product that is preferable from the perspective of decision makers in Thailand, and to assess how this can be used to inform the country’s manufacturers for the R&D process. Firstly, this study examines whether the TSE approach can be used to identify important characteristics for a vaccine product to have market advantage over existing products. Secondly, this study also assesses the perceived usefulness of the TSE approach among stakeholders in three Southeast Asian countries, namely Indonesia, Thailand, and Vietnam. The pilot study used five example Rotavirus vaccine products as a hypothetical test case, because of the variable product characteristics of both existing, licensed and pipeline rotavirus vaccines.

Methods

Three workshops were held during the pilot to better understand criteria of importance in national decision-making and the perspective of different stakeholders towards TSE. An initial workshop in May 2018 was convened to obtain preliminary feedback from stakeholders involved in vaccine policy decision making and research in Thailand. In addition to WHO, HITAP and NUS, 15 participants were in attendance and included representatives from: Subcommittee of the National List of Essential Medicines, The National Immunization Technical Advisory Group (NITAG), academia, pharmaceutical industry and other ministerial departments. During the workshop, stakeholders were asked to complete an open-ended survey questionnaire to nominate top criteria important for vaccine product selection. The survey template was used previously by the WHO in a stakeholder meeting convened in Indonesia. The survey instrument was subsequently fine-tuned and modified for this study context. The survey instrument is available in S1 File. Responses were collected from 15 respondents and the top five criteria were selected as the decision-making criteria for the pilot study in Thailand.

For the purpose of the study, an excel-based model for rotavirus product selection, based on the TSE approach, was modified for the Thai context. The TSE rotavirus product selection model included a generic set of decision criteria, with defined indicators, and enabled comparison between five hypothetical rotavirus vaccine products (RVV1 to RRV5, product characteristics detailed in S1 Table). The final list of decision criteria for the Thai model was identified by stakeholders in Thailand. The final five criteria and associated outcomes for the Thai model are as follows: 1) health outcomes (i.e. cases averted, hospitalisation cases averted, and deaths averted due to the vaccination); 2) cost estimates (total programme costs, healthcare costs, and a five-year budget impact); 3) safety data (intussusception cases); 4) budget impact; and 5) cost effectiveness. Since budget impact and cost-effectiveness had not been included in the generic TSE rotavirus model, they were added for the purpose of the exercise.

The excel-based model utilized methodology that has been used in established models such as UNIVAC and the Vaccine-Technology Impact Assessment (V-TIA) tool to calculate the outcomes defined for each criterion, using country and product specific data inputs [17]. Locally relevant parameter inputs were generated from government reports, published literature and expert opinions. They also included local inputs on socio-economic status, coverage of vaccine programs, vaccine efficacy, and schedule, costs for storage, training and administration, and other epidemiological data on birth cohort, disease burden and resource use (S2 Table).

Following MCDA methodology, the outcomes for each criterion were transformed into a score based on a common absolute scale ranging from 0–100, to calculate an aggregate score for each option, resulting in a final ranking of each options according to performance across all criteria [18]. A vaccine with a higher total score was considered to be the better choice. Equal weights were used for all criteria, as the research team were unable to collect information on stakeholder preferences for weighting. A full description of the model and how the model outcomes were generated is available elsewhere [17].

A one-way sensitivity analysis was conducted on the vaccine characteristics, to identify the thresholds of each characteristic that could modify the overall ranking of products. It is hypothesised that this could form the basis for identifying minimum and aspirational bounds for target product profiles that meet country needs. We chose the vaccine product which is currently being ranked the second in the MCDA analysis, and via the sensitivity analysis, investigated if by varying the vaccine characteristics, it could become the top ranked vaccine. The sensitivity analysis was restricted to base and best case values for vaccine characteristic for identifying the variable changes which could make the second best product become the best ranked product. The base case was determined by the input values that reflected the most likely scenario and the input values for the best case represented the most optimistic scenario. The details on the inputs for the vaccine related variables for the base case and the best case can be found in the S3 Table.

To identify the usefulness of such an approach from the R&D perspective, a separate workshop was held with stakeholders working in R&D in LMICs from both public and private sector organisations in Indonesia, Thailand, and Vietnam. The results from the rotavirus product MCDA evaluation were disseminated to illustrate the principle that–with information on country priorities for vaccine products–manufacturers could model a pipeline product against those of competitors to identify important attributes and thresholds given uncertainty. Key challenges and priorities in the countries were presented, and discussions were encouraged to deliberate the relevance and usability of TSE. Qualitative data was also collected from 17 self-assessment forms on the applicability and usefulness of this approach to inform R&D. The self-assessment form included a mixture of closed and open-ended questions to gather participant opinions of TSE and is available in S2 File. The form was developed for the purpose of this meeting and participation was completely voluntary. Participants were made aware that the forms were anonymized and that responses were to be kept strictly confidential and reflect personal opinions rather than the positions of their employers. Thematic analysis was deployed to analyse main patterns in the responses within the questionnaires. The forms were read repeatedly to enhance the overall understanding, then coded and classified. Then codes were grouped into emerging themes. A third workshop was convened to present the pilot study results back to the stakeholders who attended the initial meeting.

Results

Criteria selected by decision makers in Thailand

The top 5 decision-making criteria identified by stakeholders for choosing between rotavirus vaccine products were as follows: health impact, safety, budget impact, cost-effectiveness and delivery costs. The different outcome measures which represented the criteria are illustrated in Table 1. The scoring of the vaccine products reported RVV-3 to be the most preferred vaccine product, followed by RVV-2 as the second rank vaccine product followed by RVV-4, RVV-5 and RVV-1 as the rank 3rd, 4th, and 5th ranked vaccine product respectively (S4 Table).

Table 1. Top 5 decision making criteria for Thailand stakeholders for vaccine policy making.

Decision Criteria	Description of related outcome measure
Safety	Intussusception hospitalisations due to vaccine
Health Impact	Rotavirus cases averted due to vaccination
Budget Impact	Overall 5-year budget impact including the cost of program and healthcare cost^*
Delivery cost	Transport and storage costs for vaccines
Cost-effectiveness	Incremental costs per DALY

Open in a new tab

* The overall budget impact of the vaccine includes the cost of the immunization program and the also the healthcare resources spent on the rotavirus cases and the intussusception cases the population experiences.

Vaccine characteristics influencing the vaccine performance on decision criteria

Multiple vaccine characteristics were then identified which could impact the scores of the vaccine products on the decision criteria nominated by stakeholders. One vaccine characteristic could influence the vaccine performance in one or more decision criteria since decision criteria are not mutually exclusive/interdependent. The different vaccine characteristics included the volume, number of doses and doses per vial, cooling method, commodity cost, efficacy, duration of protection, relative risk of intussusception, and vaccine schedule, see Table 2. The non-vaccine parameters included disease epidemiological data (e.g. disease burden, severity of the disease, associated mortality and numbers of outpatient and hospitalisation), system cost data (training costs, salaries of health care workers, cost of hospitalisations), and other parameters such as coverage and socio-economic status.

Table 2. A breakdown of the influential vaccine characteristics.

Decision criteria	Vaccine characteristics influencing criteria	Other parameters influencing criteria
Safety	Relative risk of intussusception, Number of doses, Vaccine schedule
Health impact	Vaccine efficacy, Number of doses, Duration of protection, Vaccine schedule	Socio-economic status, Coverage
Budget impact	Commodity cost, Volume of vaccine, Method of cooling, Number of doses per vial, Vaccine efficacy, Number of doses, Duration of protection, Vaccine schedule, Relative risk of intussusception	Training costs, Trends in size of birth cohorts, Trends in coverage, Salary additional health care workers
Delivery cost	Vaccine volume	Electricity price, Petrol price, Number of deliveries, Distance between the level in the supply chain
Cost-effectiveness	Commodity cost, Volume of vaccine, Method of cooling Number of doses per vial, Vaccine efficacy, Number of doses, Duration of protection, Vaccine schedule, Relative risk of intussusception	Rotavirus incidence, Proportion severe, Rotavirus mortality, Coverage of existing immunisation schedules, Socio-economic status, Number of deliveries, Training costs & salary of healthcare workers, Distribution of inpatient visits over different levels of health care, Distribution of outpatient visits over different levels of health care, Length of hospitalization, Intussusception case fatality rate, Cost of cooling

Open in a new tab

Sensitivity analysis to identify significant vaccine characteristics

Varying certain vaccine characteristics independently did change the rank order of RVV-2 from second to first position in the rotavirus product selection model. Vaccine characteristics that changed the rank order of RVV-2 are as follows: relative risk of intussusception, number of doses, vaccine efficacy, duration of protection and commodity cost. The best-case scenario of each of these significant parameters could independently make RVV-2 the top ranking vaccine (Table 3). Details on the impact of the best case scenarios used for the one-way sensitivity analysis on the overall vaccine scores can be found in the S5 Table.

Table 3. Sensitivity analysis to identify significant vaccine characteristics for making RVV-2 the highest ranked vaccine.

Vaccine Characteristics	Inputs for RVV-2		Old vaccine Rank	New Vaccine Rank	Related decision criteria
Vaccine Characteristics	Base-case^*	Best-case^**	Old vaccine Rank	New Vaccine Rank	Related decision criteria
Relative risk of intussusception	1.24	1.0	2^nd	1^st	Safety, Budget Impact, Cost-effectiveness
Number of doses	2	1	2^nd	1^st	Safety, Health Impact, Budget Impact, Cost-effectiveness
Vaccine Schedule	DTP-1 (6 weeks after birth)	OPV-1 (1 week after birth)	2^nd	2^nd	Safety, Health Impact, Budget Impact, Cost-effectiveness
Vaccine efficacy	50%	100%	2^nd	1^st	Health Impact, Budget Impact, Cost-effectiveness
Duration of Protection (weeks)	52	156	2^nd	1^st	Health Impact, Budget Impact, Cost-effectiveness
Commodity cost (US$)	2.2	1.1	2^nd	1^st	Budget Impact, Cost-effectiveness
Volume of the vaccine (m³)	17.6	8.8	2^nd	2^nd	Budget Impact, Delivery costs, Cost-effectiveness

Open in a new tab

* The most likely input value

** The best or aspirational input value

Usefulness of TSE approach to inform vaccine R&D: R&D stakeholder perspective

Several themes transpired from the participant’s self-assessment including: potential to strengthen product selection process, limited local capacity, need for technical assistance, concerns of applicability. The profile of respondents is documented in Table 4 and the detailed results of the thematic analysis can be found in S6 Table.

Table 4. Profile of respondents from the self-assessment forms.

	Number
TYPE OF ORGANISATION
GOVERNMENT	8
PRIVATE SECTOR	5
INTERNATIONAL ORGANISATION	3
NOT SPECIFIED	1
COUNTRY OF WORK (THAILAND/OTHER/BOTH)
THAILAND	11
OTHER	3
BOTH	2
NOT SPECIFIED	1

Open in a new tab

The majority of stakeholders stated that characteristics for new vaccine products are being decided unsystematically and with limited visibility of country priorities, and unclear rationale for selection, especially for markets outside of their country. There was broad agreement that TSE could have a pivotal role in providing a platform for discussion between public health and R&D stakeholders to create alignment regarding preferred characteristics of products. As such, it was felt that TSE could potentially be an approach to develop an R&D strategy for countries to drive innovation and increase the likelihood of uptake beyond national markets, by considering attributes that would facilitate recommendations and uptake in other market segments.

One limitation raised was that the TSE approach will only collect information from countries that are implementing the TSE framework for product evaluation and decision-making, which requires developing the technical capacity for TSE implementation. For TSE to be applicable in many settings, there is a need to strengthen the country’s technical capacity for evidence-informed priority setting. Shifts in decision making priorities from the time of initiation of product development to the phase of market translation of the finished product was another concern on the usability of TSE. The R&D stakeholders maintained that it is of great importance that TSE is regularly revisited and revised to reflect changing country specific priorities.

Discussion

This study was based on a proof-of-concept exercise to find out whether an MCDA approach to product selection could analyse trade-offs in characteristics of pipeline products if stakeholder perspectives are known. The findings illustrated that an understanding of national decision-maker priorities for vaccine policy or selection decisions can help to characterize preferred product attributes. Varying either of the following vaccines characteristic; relative risk of intussusception, number of doses, vaccine efficacy, and duration of protection and commodity cost, independently by giving it the best-case value (based on the most optimistic scenario) could propel the second-ranked vaccine into first rank in the rotavirus product selection model. This demonstrates the importance of these characteristics to the Thai stakeholders consulted.

Understanding product preferences from the perspective of country decision makers provides critical input into the Target Product Profile (TPP) for vaccine products. TPPs describe the minimally acceptable and aspirational ranges of product features to guide product development of vaccines, and establish go/no-go criteria for prioritisation and investment by developers. As such, the TSE approach could allow manufacturers to understand country needs, which could be combined with technical considerations such as likelihood of clinical success and manufacturing feasibility to develop more robust TPPs and roadmaps for products with higher chance of country uptake that will ultimately lead to greater public health impact [19].

This study demonstrated the application of MCDA as systematic approach for evaluating the relative value proposition, or trade off, of existing and potential products based on country priorities. It is no longer considered that a single criterion, such as cost-effectiveness or disease burden, is sufficient to rank vaccines in the pipeline; a multitude of factors come into play [20, 21]. MCDA methodology allows for input from a wide range of stakeholders with an array of priorities to support effective vaccine development decisions [21]. Application of criteria weights which represent the relative importance of the decision criteria further imparts flexibility to adapt the approach to different diseases and country setups [22]. The study used equal weighting and did not include varied weights to represent the importance of the decision criteria for the stakeholder group nor perform sensitivity analysis to the equal weights assigned to the criteria due to lack of relevant data collection. However, the authors feel that altering the distribution of the weights in such evaluations would be of significant importance to better understand the robustness of the characteristics. Furthermore, the outcomes associated with each criterion were already pre-determined in the model. Allowing stakeholders to define their own outcomes may lead to a better understanding of expectations for pipeline products from a country perspective.

Special attention needs to be paid when determining the distribution of values identified to test best- and worst-case scenarios. A small range may inflate the minimal advantage offered by a vaccine product over the other products, making it the best scoring vaccine. Similarly, for decision criteria where even small changes are important like deaths avoided, a wide scoring range, may deflate the benefits on the scoring scales. An important requirement of MCDA is to engage multiple stakeholders to identify decision criteria and ensure the results reflect national priorities [23]. TSE can only be applicable if it is rooted in a structured and transparent process with an explicit rationale behind decisions.

TSE takes an early HTA approach to systematically evaluate the value of vaccine products in the pipeline. Utilising HTA during R&D stage, when the major product design decisions are made, can be greatly beneficial to manufacturers. Traditional HTA is applied to existing products on the market. Products that fail this late-stage HTA waste millions of dollars in lost investment, and risk increase market entry costs for future products. Performing early stage HTA, particularly in the form of the TSE framework, provides an opportunity to inform manufacturer TPPs which can mitigate this uncertainty [24], de-risking R&D investment and accelerating product utilisations.

This study uses one-way sensitivity analysis to emphasize how single vaccine characteristics may be significant in favouring one vaccine product over the other. A one-way sensitivity analysis is in line with the standard approach in the literature for early health technology assessment (HTA) [24]. This approach used in this study is named headroom method is a way of estimating the maximum reimbursable price of the new device over a comparator to determine a value-based price ceiling. While multivariate and probabilistic sensitivity analysis are widely accepted in traditional HTA, the authors have not seen any guidelines of how it can be applied to early HTA. This is an area for future research.

The TSE approach is not without its limitations. In order for TSE to inform product selection, and product optimisation, the framework needs to be embedded within, and must complement the country’s existing vaccine evaluation process. Our study used a quantitative model, whereas the TSE framework has been adapted to function in environments with varying data quality and analytical resources.

Also, the current study findings are limited to the local conditions and decision-making mechanisms of Thailand and may not be generalisable to other developing countries in the world. Stakeholders consulted were a small sample of individuals across the Southeast Asian region and predominantly from the public sector. Findings will not be representative of the experiences of all those involved in vaccine innovation and product selection decisions in the region, and further study needs to be undertaken to consider whether information from using the TSE framework in data poor environments yields similar interest from R&D stakeholders.

Conclusion

This studied verified that a critical communication gap between country decision-makers and vaccine developers exists in the South-East Asia region, and findings illustrate that the TSE approach may be used to identify the priority vaccine attributes of country stakeholders through MCDA and early HTA. The authors believe that shifting to a ‘needs-driven’ R&D paradigm can move mountains in addressing preventative health needs and accelerating global health progress towards Universal Health Coverage. Successful implementation of this approach would be a win for all the stakeholders–government accessing products tailored to country needs, manufacturers enjoying a lower risk in R&D and patients attaining a faster access to good quality health benefits. In summary, there was a majority agreement among public and private sector R&D stakeholders in South-East Asia with the overall acceptability of TSE and its role and value in addressing the communication gap between decision makers and manufacturers. Albeit this proof of concept pilot study has shown promising early results, effort needs to be made to ensure TSE can fit within decision-making processes across LMICs and adapt to changing priorities. Moreover, considerable efforts are needed to ensure comprehensive and productive communication of the findings with R&D stakeholders and ensure a quality dialogue between the stakeholders. TSE, under the new name CAPACITI, is in the next phase of the development and issues raised in this pilot study have been critical in its further evolution,

Supporting information

S1 Table. Vaccine characteristics of the five hypothetical rotavirus vaccine products.

(DOCX)

Click here for additional data file.^{(14.1KB, docx)}

S2 Table. Input variables required for populating the TSE model.

(DOCX)

Click here for additional data file.^{(17.2KB, docx)}

S3 Table. Vaccine characteristics variables for the vaccine RVV-2 used in sensitivity analysis.

(DOCX)

Click here for additional data file.^{(13.3KB, docx)}

S4 Table. Scores and ranks for the five hypothetical rotavirus vaccine products using the TSE approach.

(DOCX)

Click here for additional data file.^{(13.2KB, docx)}

S5 Table. Scores and ranks for RVV-2, the second ranking vaccine product, in the base case and best case sensitivity analysis.

(DOCX)

Click here for additional data file.^{(13.2KB, docx)}

S6 Table. Thematic analysis results from self-assessment forms.

(DOCX)

Click here for additional data file.^{(12.5KB, docx)}

S1 File. Open-ended questionnaire distributed at initial stakeholder meeting.

(DOCX)

Click here for additional data file.^{(155.9KB, docx)}

S2 File. Self-assessment form distributed at stakeholder research and development meeting.

(DOCX)

Click here for additional data file.^{(55.3KB, docx)}

Acknowledgments

The authors would like to express their gratitude to Ms. Waranya Rattanavipapong, Ms. Manushi Sharma, and Mr. Md Rajibul Islam who were co-investigators in this project. Publication of study results was not contingent on the sponsor's approval or censorship of the manuscript.

Abbreviations

CAPACITI: Capacity-led Assessment for Prioritisation on Immunisation
HITAP: Health Intervention and Technology Assessment Program
HTA: Health Technology Assessment
LMICs: Low-And Middle-Income Countries
MCDA: Multiple-Criteria Decision Analysis
NITAG: National Immunization Technical Advisory Group
NUS: National University of Singapore
R&D: Research and Development
TPP: Target Product Profile
TSE: Total Systems Effectiveness
UHC: Universal Health Coverage
WHO: World Health Organization

Data Availability

All relevant data are within the paper and its Supporting Information files.

Funding Statement

This pilot study was funded by the Bill and Melinda Gates Foundation. The Health Intervention and Technology Assessment Program (HITAP) is funded by the Thailand Research Fund under a grant for Senior Research Scholar (RTA5980011). HITAP’s International Unit is supported by the International Decision Support Initiative (iDSI) to provide technical assistance on health intervention and technology assessment to governments in low- and middle-income countries. iDSI is funded by the Bill & Melinda Gates Foundation [OPP1202541], the United Kingdom’s Department for International Development, and the Rockefeller Foundation. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. HITAP’s International Unit collaborates with The Access and Delivery Partnership.

References

1.World Health Organisation. Universal health coverage (UHC). Available from: https://www.who.int/news-room/fact-sheets/detail/universal-health-coverage-(uhc). [Accessed April 17, 2019]
2.Strategic Group of Experts in Immunization. 2018 Assessment report of the global vaccine action plan. Available from: http://apps.who.int/. [Accessed April 17, 2019]
3.World Health Organisation. Health Systems Financing: The Path to Universal Coverage. Health Systems Financing: The Path to Universal Coverage. Available from: https://www.who.int/whr/2010/whr10_en.pdf?ua=1. [Accessed May 7, 2019]
4.Kristensen DD, Bartholomew K, Villadiego S, Lorenson K. What vaccine product attributes do immunization program stakeholders value? Results from interviews in six low- and middle-income countries. Vaccine 2016;34(50):6236–42. 10.1016/j.vaccine.2016.10.057 [DOI] [PubMed] [Google Scholar]
5.Popova O, Ibarra de Palacios P. Reaching more children with vaccines in developing countries: key challenges of innovation and delivery. Currrent Medical Research and Opinion 2016; 32(1):177–81. [DOI] [PubMed] [Google Scholar]
6.Giersing BK, Kahn A-L, Jarrahian C, Mvundura M, Rodriguez C, Okayasu H, et al. Challenges of vaccine presentation and delivery: How can we design vaccines to have optimal programmatic impact? Vaccine [Internet]. 2017. December 14 [cited 2019 Apr 17];35(49):6793–7. [DOI] [PubMed] [Google Scholar]
7.Hauck K, Thomas R, Smith PC. Departures from Cost-Effectiveness Recommendations: The Impact of Health System Constraints on Priority Setting. Heal Syst Reform [Internet]. 2016. January 2 [cited 2018 Oct 12];2(1):61–70. [DOI] [PubMed] [Google Scholar]
8.World Health Organisation. Principles and considerations for adding a vaccine to a national immunization programme: from decision to implementation and monitoring. 2014 [cited 2018 Oct 12]; Available from: http://www.who.int/immunization/documents
9.Lee J, Kim C, Shin J. Technology opportunity discovery to R&D planning: Key technological performance analysis. Technol Forecast and Social Change 2017;119:53–63. [Google Scholar]
10.Sheerin D, Openshaw PJ, Pollard AJ. Issues in vaccinology: Present challenges and future directions. European Journal of Immunology 2017;47(12):2017–25. 10.1002/eji.201746942 [DOI] [PMC free article] [PubMed] [Google Scholar]
11.Plotkin S, Robinson JM, Cunningham G, Iqbal R, Larsen S. The complexity and cost of vaccine manufacturing—An overview. Vaccine 2017;35(33):4064–71. 10.1016/j.vaccine.2017.06.003 [DOI] [PMC free article] [PubMed] [Google Scholar]
12.Brooks A, Nunes JK, Garnett A, Biellik R, Leboulleux D, Birkett AJ, et al. Aligning new interventions with developing country health systems: target product profiles, presentation, and clinical trial design. Global Public Health 2012;7(9):931–45. 10.1080/17441692.2012.699088 [DOI] [PMC free article] [PubMed] [Google Scholar]
13.Botwright S, Hutubessy R, Jit M, Meltzer M, Moore KL, El Omeiri N, et al. Country-led Assessment for Prioritisation on Immunisation (CAPACITI): A novel approach to support systematic priority-setting in low and middle income country immunisation programmes. Unpublished manuscript. World Health Organization; 2020 [Google Scholar]
14.Marsh K, Thokala P, Youngkong S, Chalkidou K. Incorporating MCDA into HTA: challenges and potential solutions, with a focus on lower income settings. Cost Effectiveness Resource Allocation 2018;16(S1):43. [DOI] [PMC free article] [PubMed] [Google Scholar]
15.Frazão TDC, Camilo DGG, Cabral ELS, Souza RP. Multicriteria decision analysis (MCDA) in health care: a systematic review of the main characteristics and methodological steps. BMC Medical Informatics and Decision Making 2018;18(1):90 10.1186/s12911-018-0663-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
16.Inotai A, Nguyen HT, Hidayat B, Nurgozhin T, Kiet PHT, Campbell JD, et al. Guidance toward the implementation of multicriteria decision analysis framework in developing countries. Expert Review of Pharmacoeconomics & Outcomes Research 2018;18(6):585–92. [DOI] [PubMed] [Google Scholar]
17.Rattanavipapong W, Kapoor R, Teerawattananon Y, Luttjeboer J, Botwright S, Archer R, et al. Comparing 3 Approaches for Making Vaccine Adoption Decisions in Thailand. International Journal of Health Policy and Management 2020; x(x), 1–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
18.Marsh K, Ijzerman M, Thokala P, Baltussen R, Boysen M, Kaló Z, et al. Multiple Criteria Decision Analysis for Health Care Decision Making-Emerging Good Practices: Report 2 of the ISPOR MCDA Emerging Good Practices Task Force. Value in Health 2016;19:125–37. 10.1016/j.jval.2015.12.016 [DOI] [PubMed] [Google Scholar]
19.Ford AQ, Touchette N, Hall BF, Hwang A, Hombach J. Global Vaccine and Immunization Research Forum: Opportunities and challenges in vaccine discovery, development, and delivery. Vaccine 2016;34(13):1489–95. 10.1016/j.vaccine.2015.11.038 [DOI] [PubMed] [Google Scholar]
20.Knobler S, Bok K, Gellin B. Informing vaccine decision-making: A strategic multi-attribute ranking tool for vaccines—SMART Vaccines 2.0. Vaccine 2017; 35(1):A43–A45 [DOI] [PubMed] [Google Scholar]
21.Madhavan G, Phelps C, Rappuoli R. Bridging the Gap: Need for a Data Repository to Support Vaccine Prioritization Efforts. Vaccine 2015; 33(2):B34–B39. [DOI] [PMC free article] [PubMed] [Google Scholar]
22.Wahlster P, Goetghebeur M, Kriza C, Niederländer C, Kolominsky-Rabas P. Balancing costs and benefits at different stages of medical innovation: A systematic review of Multi-criteria decision analysis (MCDA). BMC Health Services Research 2015;15(1):1–12. [DOI] [PMC free article] [PubMed] [Google Scholar]
23.Angelis A, Kanavos P. Multiple Criteria Decision Analysis (MCDA) for evaluating new medicines in Health Technology Assessment and beyond: The Advance Value Framework. Social Science & Medicine. 2017;188:137–56. [DOI] [PubMed] [Google Scholar]
24.Ijzerman MJ, Koffijberg H, Fenwick E, Krahn Murray. Emerging Use of Early Health Technology Assessment in Medical Product Development: A Scoping Review of the Literature. Pharmacoeconomics 2017;35(7):727–740. 10.1007/s40273-017-0509-1 [DOI] [PMC free article] [PubMed] [Google Scholar]

PLoS One. doi: 10.1371/journal.pone.0233950.r001

Decision Letter 0

Ray Borrow

18 Feb 2020

PONE-D-19-22507

‘It takes two to tango’: Bridging the gap between country need and vaccine product innovation

PLOS ONE

Dear Ms Archer,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised below during the review process.

We would appreciate receiving your revised manuscript by Apr 03 2020 11:59PM. When you are ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter.

To enhance the reproducibility of your results, we recommend that if applicable you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). This letter should be uploaded as separate file and labeled 'Response to Reviewers'.
A marked-up copy of your manuscript that highlights changes made to the original version. This file should be uploaded as separate file and labeled 'Revised Manuscript with Track Changes'.
An unmarked version of your revised paper without tracked changes. This file should be uploaded as separate file and labeled 'Manuscript'.

Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.

We look forward to receiving your revised manuscript.

Kind regards,

Ray Borrow, Ph.D., FRCPath

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

http://www.journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and http://www.journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Please provide additional details regarding participant consent. In the ethics statement in the Methods and online submission information, please ensure that you have specified (1) whether consent was informed and (2) what type you obtained (for instance, written or verbal). If the need for consent was waived by the ethics committee, please include this information.

3. Please include additional information regarding the survey or questionnaire used in the study and ensure that you have provided sufficient details that others could replicate the analyses. For instance, if you developed a questionnaire as part of this study and it is not under a copyright more restrictive than CC-BY, please include a copy, in both the original language and English, as Supporting Information. Moreover, please include more details on how the questionnaire was pre-tested, and whether it was validated.

Moreover, please ensure that the models used are adequately described, and that the coding used is available.

4. "The authors would like to express their gratitude to Ms. Waranya Rattanavipapong, Ms. Manushi Sharma, and Mr. Md Rajibul Islam who were co-investigators in this project"; we would recommend that you consult our authorship requirements page: https://journals.plos.org/plosone/s/authorship, to ensure that everyone who meets our criteria for authorship is listed as an author.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: N/A

Reviewer #2: N/A

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The paper provides interesting insights into workshops that were organized in Thailand with stakeholders from different South East Asian countries, discussing country stakeholders’ preference for vaccine characteristics and piloting an approach called Total System Effectiveness. The authors provide a robust context and rationale for TSE and the pilot study but the methods and results descriptions are very light, making it difficult for readers to fully understand the study findings and conclusions and assess their validity. Several study tools mentioned in the paper are not described and data and data sources are not available. Some of the language used in the discussion and conclusion should be revised to avoid broad statements and better linking findings to study results. I have a list of major and minor comments below:

Major comments

1/ Introduction, lines 150-152 - The study relies on characteristics of existing vaccines and vaccines under development. The authors should comment on how the process is allowing for stakeholders to identify characteristics outside of those already suggested by the use of hypothetical products.

2/ Methods - The model used for the study should be further described, a model sketch would be useful. It’s very hard for the reader to understand what was done in the absence of a clear description or details on calculations of the model. Supplementary Table 1 provides only model variables and their description, the model input and output data (and data sources when relevant) should be made available.

3/ In L156-157 how were model criteria defined? Are these the same criteria referred to later and generated from the stakeholders workshops?

4/ L162 A description of how model outcomes were defined and how they were generated is required.

5/ L176-177 Please define the base and best case values, how they were defined and where they can be found in the article.

5/ Uncertainty in the study is not fully accounted for. A one-way sensitivity analysis is performed on vaccine characteristics of a single product. The authors should consider multivariate or probabilistic sensitivity analysis on all 5 products and characteristics or better justify their choice of a restrictive uncertainty analysis.

6/ Such as for the model, the “open ended survey” and “feedback forms” used should be described if not made available.

7/ L198 The “thematic analysis” mentioned is not described and it’s not clear from the text how it was used and if results from this analysis are shared in the paper.

8/ Results – It seems decision makers identified only criteria relevant for decision-making and not vaccine characteristics. Authors should describe how vaccine characteristics were determined, as identifying preferred vaccine characteristics is one of the objective of the study. How about providing specific vaccine attributes that were identified as ideal for a rotavirus vaccine in the Thai context?

9/ Line 209 – 210: “outcomes measures which represented the criteria are illustrated in Table 2”. Clarify if outcome measures were identified as part of the same process than criteria or if they were suggested to stakeholders. Also why results for each outcome measures and each hypothetical products are not available from the paper?

10/ L220 it is unclear what authors mean by “influencing the vaccine performance on decision criteria”. Defining how vaccine characteristics influence decision criteria is necessary to allow reader to understand the results.

11/ L222-224 If vaccine characteristics were an input to the study and not defined by it, then they should be included in the methods rather than the results.

12/ Table 3 lists decision criteria, vaccine characteristics influencing criteria and other parameters influencing criteria. I have few comments on the table:

1. Some of the items under vaccine characteristics are not really actual characteristics, for example relative risk of intussusception or method of cooling

2. The overlap between some of the vaccine characteristics across different decision criteria should be commented on. For example number of doses or vaccine schedule influencing different decision criteria.

3. The other parameters influencing criteria are broader and it’s not clear where they are coming from.

4. There should be further explanations of how or why are duration of protection or relative risk of intussusception influencing budget impact.

13/ L235-239 The results of the sensitivity analysis include characteristics that would change the ranking order of the preferred product without providing data or extent to which a characteristics would positively or negatively affect the ranking.

14/ Table 4: where are base case and best case values coming from? Does the best case refer to the best product as reported by decision makers? If yes is this a representation of the product characteristics that were shared with R&D stakeholders?

15/ L249-251 weren’t any clinical development related reasons or manufacturing considerations expressed as a trigger to particular vaccine characteristics? If not it would be interesting for the authors to comment on it.

16/ L249- 255 Not having clarity or a description of how data was collected and analyzed and the use of terms such as “stakeholders stated”, “there was broad agreement” or “it was felt that” doesn’t convey a robust scientific basis for the qualitative data results. Authors should instead consider providing results from the thematic analysis mentioned in the methods section.

17/ L267-270 The study doesn’t conclude on preferred product attributes that could inform TPP or at least they are not clearly stated, only prioritized decision criteria and broad vaccine characteristics are provided. Following the pilot experience, a reader would like to see a clear description of the ideal product, which characteristics could inform TPP meaning, how many doses, following what schedule, at what price, etc…as identified by Thai decision makers.

18/ L284 – 285 Best and worst case scenarios are not described in the methods or the results.

Minor comments

1/ Abstract L55-57 disclaimers about the use of information from anonymous stakeholders and ethics review considerations should also be reported in the text.

2/ Line 88-89 it would be interesting listing the six countries and put their relative country context in perspective with the Thai context where the pilot study was carried out.

2/ Line 116-117 “government priorities” seems broad, maybe government vaccine product preferences would be more appropriate.

3/ Line 158 change RRV5 TO RVV5

4/ Line 177 Table 4 is the first table referenced in the text

5/ L179 Three workshops were held but only two are described

6/ Table 1 is not referenced in the text and it’s not clear to which survey/form/workshops the respondents it refers to

7/ Line 208 avoid repeating “criteria” and “identified” in the same sentence

8/ Line 245 the “**” sign is not referenced in the table

9/ Line 252-253 revise the statement “preferred product characteristics of products”

10/ L273 country preferences rather than country needs

Reviewer #2: Vaccine manufacturers are extremely interested in anticipating the market demand during vaccine development. Today, evaluation of product’s desired characteristics is moving toward the earlier stages of development, and R&D activities are increasingly tailored to the target product profile (TPP) of the new vaccine. The limitation of the current approach is that, usually, vaccine manufacturers rely on the opinion of advisory groups in high income countries, which might not align with LMIC needs.

The proposed Total System Effectiveness approach (TSE), based on Multi-Criteria Decision Analysis is long overdue because it brings the tools to consider, in a systematic way, the vaccines characteristics needed by LMICs. The opinions generated by this method could be very valuable for vaccine manufacturers, provided the process is fully transparent. The argumentation of the manuscript is solid, the analytical strategy clearly supports the conclusions, and the authors actively gathered feedback from R&D stakeholders to increase the robustness of their process.

One main consideration concerns the implementation of such approach. A decision-making process should be put in place to define which studies are going to be conducted. Stakeholders from both public and private sector should be made able to influence what topics are of the greatest interest.

Suggestion is to add additional and more relevant references on the use of MCDA in vaccines such as https://doi.org/10.1016/j.vaccine.2016.10.086 and references quoted in this paper.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files to be viewed.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org. Please note that Supporting Information files do not need this step.

PLoS One. 2020 Jun 10;15(6):e0233950. doi: 10.1371/journal.pone.0233950.r002

Author response to Decision Letter 0

27 Apr 2020

RESPONSES TO REVIEWERS

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

Reviewer #1: Partly

Reviewer #2: Yes

RESPONSE: Thank you. We have made revisions to ensure the content is technically sound.

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: N/A

Reviewer #2: N/A

RESPONSE: Thank you.

3. Have the authors made all data underlying the findings in their manuscript fully available?

Reviewer #1: No

Reviewer #2: Yes

RESPONSE: Thank you. We have revised to ensure all data is fully available.

4. Is the manuscript presented in an intelligible fashion and written in standard English?

Reviewer #1: Yes

Reviewer #2: Yes

RESPONSE: Thank you.

5. Review Comments to the Author

The paper provides interesting insights into workshops that were organized in Thailand with stakeholders from different South East Asian countries, discussing country stakeholders’ preference for vaccine characteristics and piloting an approach called Total System Effectiveness. The authors provide a robust context and rationale for TSE and the pilot study but the methods and results descriptions are very light, making it difficult for readers to fully understand the study findings and conclusions and assess their validity. Several study tools mentioned in the paper are not described and data and data sources are not available. Some of the language used in the discussion and conclusion should be revised to avoid broad statements and better linking findings to study results. I have a list of major and minor comments below:

Major comments

RESPONSE:

We are grateful for the reviewer's indispensable comments and thorough reading of our manuscript. The reviewer is right. In this study, we used characteristics of vaccines under development as a baseline. The study allowed for the investigators to use the baseline criteria to predict the future likelihood impact of each hypothetical vaccine product against the decision criteria identified by the stakeholders. The stakeholders directly identified the decision criteria rather than the characteristics. The researchers performed a one-way sensitivity analysis on the vaccine characteristics in order to find out the significant vaccine characteristics which could enable the vaccine product with the 2nd rank to become the most preferred vaccine product (1st rank) against the criteria. This is shown in Table 3 on the paper. The significant vaccine characteristics were found to be relative risk of intussusception, vaccine efficacy, number of doses, duration of protection, commodity cost. The best-case scenario of each of these significant parameters could independently make RVV-2 the top ranking vaccine. We have added this explanation between lines 181 and 186 (pages 7-8).

“A one-way sensitivity analysis was conducted on the vaccine characteristics, to identify the thresholds of each characteristic that could modify the overall ranking of products. It is hypothesised that this could form the basis for identifying minimum and aspirational bounds for target product profiles that meet country needs. We chose the vaccine product which is currently being ranked the second in the MCDA analysis, and via the sensitivity analysis, investigated if by varying the vaccine characteristics, it could become the top ranked vaccine.”

RESPONSE:

The reviewer has made a valid point. An open-access publication by Rattanavipapong et al (2020) provides a detailed description of the model in a supplementary file. The paper is open-access and therefore this description is publically available. We have referenced this paper in the text between lines 179-180 (page 7) stating that “a full description of the model and how the model outcomes were generated is available elsewhere.”

Rattanavipapong, W., Kapoor, R., Teerawattananon, Y., Luttjeboer, J., Botwright, S., Archer, R. A., ... & Hutubessy, R. C. (2020). Comparing 3 approaches for making vaccine adoption decisions in Thailand. International Journal of Health Policy and Management. Doi: 10.15171/ijhpm.2020.01

3/ In L156-157 how were model criteria defined? Are these the same criteria referred to later and generated from the stakeholders’ workshops?

RESPONSE:

Yes the reviewer is correct. The criteria was elected by stakeholders during the first stakeholder meeting convened in May 2018 with representatives from the National Immunization Technical Advisory Group (NITAG) drug and vaccine decision-making committees, academia, pharmaceutical industry and other ministerial departments. During the workshop, stakeholders were asked to complete an open-ended survey to nominate top criteria important for vaccine product selection. Responses were collected from 15 respondents and the top five criteria were selected as the decision-making criteria for the pilot study in Thailand. We have added the following paragraph to clarify this point (lines 159-168, page 7):

“For the purpose of the study, an excel-based model for rotavirus product selection, based on the TSE approach, was modified for the Thai context. The TSE rotavirus product selection model included a generic set of decision criteria, with defined indicators, and enabled comparison between five hypothetical rotavirus vaccine products (RVV1 to RRV5, product characteristics detailed in S1 Table). The final list of decision criteria for the Thai model was identified by stakeholders in Thailand. The final five criteria and associated outcomes for the Thai model are as follows: 1) health outcomes (i.e. cases averted, hospitalisation cases averted, and deaths averted due to the vaccination); 2) cost estimates (total programme costs, healthcare costs, and a five-year budget impact); 3) safety data (intussusception cases); 4) budget impact; and 5) cost effectiveness. Since budget impact and cost-effectiveness had not been included in the generic TSE rotavirus model, they were added for the purpose of the exercise.”

4/ L162 A description of how model outcomes were defined and how they were generated is required.

RESPONSE:

We agree with the reviewer that this needs to be further explained. A published paper by Rattanavipapong et al. (2020) details the methodology behind the outcome estimation. This paper is open-access and therefore this description is publically available. We have referenced this paper in the text between lines to 179-180 (page 7) “a full description of the model and how the model outcomes were generated is available elsewhere.”

5/ L176-177 Please define the base and best case values, how they were defined and where they can be found in the article.

RESPONSE:

Following the reviewer’s suggestion, we have added supplementary file (S3 Table) which provides details of the values and the source rationale for the values for base case and the best case scenario. We have added the following sentence on lines 189-190 (page 8). “The details on the inputs for the vaccine related variables for the base case and the best case can be found in the S3 Table.”

RESPONSE:

The reviewer makes an interesting point here. We opted to use a one-way sensitivity analysis because it is in line with the standard approach for early health technology assessment (HTA). The approach is known as the headroom method and is a way of estimating the maximum reimbursable price of the new device over a comparator to determine a value-based price ceiling. IJzerman et al (2017) cite this as standard methodology used in early HTA.

Whilst multivariate and probabilistic sensitivity analysis are widely accepted in traditional HTA, to our knowledge, the authors are not aware of any guidance on how these two methods can be applied to early HTA. This paper does not explore other potential sensitivity analysis approaches but this is a recommendation for future research. We have added the above justification for the use of a one-way sensitivity analysis in the discussion section on lines 334-340 (page 16).

IJzerman, MJ, Koffijberg H, Fenwick E, Krahn M. Emerging use of early health technology assessment in medical product development: a scoping review of the literature. PharmacoEconomics. 2017;35(7):727–40

6/ Such as for the model, the “open ended survey” and “feedback forms” used should be described if not made available.

RESPONSE:

We wholly agree with the reviewer and have made both instruments available in the supplementary files (S1 File and S2 File).

7/ L198 The “thematic analysis” mentioned is not described and it’s not clear from the text how it was used and if results from this analysis are shared in the paper.

RESPONSE:

This section was updated according to the reviewers’ feedback. The authors have explained the approach to thematic analysis in further detail and have added the detailed results of the thematic analysis in S6 Table. Please see lines 203-205 (page 8), “thematic analysis was deployed to analyze main patterns in the responses within the questionnaires. The forms were read repeatedly to enhance the overall understanding, then coded and classified”.

RESPONSE:

The reviewer is correct. This study used the characteristics of existing or pipeline products as a baseline to predict the future likelihood impact of each vaccine against the decision criteria identified by stakeholders.

We have explained further between lines 181 and 186 (page 7-8) that:

We performed a one-way sensitivity analysis on vaccine characteristics to see whether this was a feasible approach to identifying the preferred product attributes for the stakeholders. We found that varying either of the following vaccines characteristic independently by giving it the best-case value (based on the most optimistic scenario) i. relative risk of intussusception, ii. number of doses iii. vaccine efficacy iv. duration of protection v. commodity cost could turn the second-ranked vaccine into first ranked in the rotavirus product selection model. This demonstrated the significance of these characteristics to Thai stakeholders in this trade-off exercise. We have added the above explanation to the discussion section on lines 288 to 295 (page 14).

From Table 3, the most ideal or desirable vaccine product for the Thai stakeholder would have:

• lowest relative risk of intussusception (best case = 1.0)

• highest vaccine efficacy (best case = 100%),

• lowest number of doses (best case = 1),

• highest duration of protection (best case = 156 weeks),

• lowest commodity cost (best case = 1.1 USD).

This result above can be predicted just by examining the criteria elicited by stakeholders without the modelling exercise. While this is product is most ideal it is fair to say it is probably not realistic. This was a proof-of-concept study to demonstrate whether an MCDA approach to product selection could evaluate trade-offs in characteristics for pipeline products if country stakeholder perspectives (criteria) are known.

The reviewer has made a very good point. When stakeholders were asked to rank criteria in the consultation meeting, the investigators were shown a presentation on what outcome measure would be utilized for each criteria in the generic TSE model (see the answer to major comment /3). The outcome measures for each criteria are illustrated in Table 1. Whilst we did not ask stakeholders to rank outcome measures, implicitly the outcome measures were identified by stakeholders. The outcome measures for each criterion corresponded to indicators in the TSE rotavirus comparison model. Please consult Table 3 in Rattanavipapong et al (2020) which contains the results for each outcome measure. The outcome measures used to evaluate the vaccine performance in each criterion could be adapted to different country settings and has been added in the discussion section. However, the model to evaluate the outcome measures would need to be adapted accordingly. We have added this as a limitation in the discussion section on lines 314-317 (page 15). The paper now states “Furthermore, the outcomes associated with each criterion were already pre-determined in the model. Allowing stakeholders to define their own outcomes may lead to a better understanding of expectations for pipeline products from a country perspective.”

RESPONSE:

Thanks you to the reviewer for this insightful comment. Vaccine characteristics for the five hypothetical rotavirus vaccine products (RVV1 to RRV5) including vaccine efficacy, duration of protection, dosing schedule, safety, cost, components of vaccine constitution, volume, application type, and delivery and storage requirements are noted in S1 Table. Multiple vaccine characteristics were identified which could impact the scores of the vaccine products on the decision criteria nominated by stakeholders. One vaccine characteristic could influence the vaccine performance in one or more decision criteria. Other system or country level factors which may influence overall success of the vaccine program were also recorded. For example, the decision criteria for safety was found to be influenced by the relative risk of intussusception, vaccine efficacy, number of doses and dosing schedule. Health impact was influenced by vaccine schedule, dosage, efficacy, duration of protection and also the coverage and socio-economic status of the population expected from the national program. We have provided further detail on lines 230 and 238.

A detailed description of the calculations of model outcomes and the influence of vaccine characteristics on the different decision criteria could be found in open-access publication by Rattanavipapong et al (2020) which describes the model in a supplementary file. The paper is open-access and therefore this description is publically available. We have referenced this paper in the text between lines 179-180 (page 7) stating that “a full description of the model and how the model outcomes were generated is available elsewhere.”

11/ L222-224 If vaccine characteristics were an input to the study and not defined by it, then they should be included in the methods rather than the results.

RESPONSE:

Thank you for the reviewer for the suggestion. You are right in that the different features (or characteristics) specific to each of the hypothetical vaccine products in S1 Table were inputs in the study as mentioned in the methods. By running the model, we could identify significant characteristics which could influence the scores of the vaccine products against criteria elicited by stakeholders in this Thai case scenario. Table 3 provides a list of these influential vaccine characteristics and therefore have been reported as a result for the Thai case example. The sensitivity analysis was then conducted on the vaccine characteristics that we found to be able to impact the scores of vaccine products against the multi decision criteria. We would therefore like to keep in the results section.

12/ Table 3 lists decision criteria, vaccine characteristics influencing criteria and other parameters influencing criteria. I have few comments on the table:

1. Some of the items under vaccine characteristics are not really actual characteristics, for example relative risk of intussusception or method of cooling

RESPONSE:

Thank you for the reviewer for this comment. We have referred all features specific to the vaccine products as vaccine characteristics. These features are expected to impact the overall cost and health benefits which can be achieved by using that vaccine product in the immunization campaign. Though some of them may not seem as the commonly referred vaccine characteristics but we still feel that they are related directly to the vaccine product and can be referred as vaccine (product) characteristics.

RESPONSE:

The reviewer is correct, some vaccine characteristics were found to impact more than one decision criteria (lines 230-238, page 9-10). This is because the different decision criteria are not mutually exclusive and have some interdependence. Hence, varying one vaccine characteristics may impact the vaccine product ranking on multiple decision criteria. For example, the number of doses of vaccine is found to have an influence on 4 decision criteria- safety, health impact, budget impact and cost-effectiveness. This is because, vaccine which has higher number of doses are expected to not provide complete protection after the first dose and is also expected to have a lower adherence as individuals are required to go to the healthcare institution for multiple doses. A lower initial protection/ adherence to the vaccine doses leads to a suboptimal coverage among the population and is expected to lead to an increase in rotavirus cases in comparison to a comparator with a single dose which imparts full protection in the first dose itself. These increased cases have a direct influence on the health impact achieved, and the overall resource use for the treatment of these cases, which in turn influence the budget impact and the cost-effectiveness analysis. Also, as the risk of side effects of intussusception are highest post vaccine administration, a vaccine with multiple doses may have an overall higher risk of intussusception, which would influence its performance on the criterion of safety.

3. The other parameters influencing criteria are broader and it’s not clear where they are coming from.

RESPONSE:

The model uses some local parameters along with the vaccine characteristics to generate outcomes for the different vaccine products (see detailed explanation on lines 159-174, page 7). These locally relevant parameter inputs have been generated from government reports, published literature and expert opinions. They include relevant parameters included local inputs on socio-economic status, coverage of vaccine programs, vaccine efficacy, and schedule, costs for storage, training and administration, and other epidemiological data on birth cohort, disease burden and resource use (lines 171-174). More details on this can be obtained from the S2 Table and in another related paper which has been published by Rattanavipapong et al (2020).

4. There should be further explanations of how or why are duration of protection or relative risk of intussusception influencing budget impact.

RESPONSE:

The overall budget impact of the vaccine includes the cost of the immunization program and the also the healthcare resources spent on the rotavirus cases and the intussusception cases the population experiences. Hence, if a vaccine product has a lower relative risk of intussusception or a longer duration of protection from the disease, this is expected to reduce the number of cases and hence lead to healthcare savings from the treatment of the diseases. Thus, a vaccine which has the same profile and cost but a higher duration of protection/or a lower intussusception risk from its comparators, is expected to have a relatively lower budget impact. In Table 1 in the paper, we have stated that the 5-year budget impact includes the cost of program and healthcare cost and added a brief footnote explaining this in the table.

13/ L235-239: The results of the sensitivity analysis include characteristics that would change the ranking order of the preferred product without providing data or extent to which a characteristics would positively or negatively affect the ranking.

RESPONSE:

The reviewer has made an excellent point regarding the results of the sensitivity analysis which need to be included in the manuscript. We have added Supplementary table 5 (S5 Table) which provides details on how the scoring of the vaccine product RVV-2 varied with the different best-case scenarios used in one-way sensitivity analysis and its impact on the overall ranking. Also, text has been added in the manuscript to link the reader to the required table in the supplementary (lines 250-251, page 12). Further details on the methodology of the TSE approach can be found in the reference Rattanavipapong et al (2020).

RESPONSE:

We are grateful for the reviewer's constructive comments regarding the best and base case. As previously stated we have added in an additional supplementary table (S3 Table) which explains the base case and best case in further details and the source/ rationale for the values for base case and the best case scenario. We refer to this between lines 189 and 190.

RESPONSE:

The focus for the study was identifying criteria that would be used for decisions on whether to introduce a vaccine (i.e. the criteria leading to a policy decision for the health programme). As such, our study did not consider likelihood of clinical success or manufacturing feasibility. We have modified the following wording to the discussion to highlight this limitation on lines 299-302 (page 15): “As such, the TSE approach could allow manufacturers to understand country needs, which could be combined with technical considerations such as likelihood of clinical success and manufacturing feasibility to develop more robust TPPs and roadmaps for products with higher chance of country uptake that will ultimately lead to greater public health impact [19]”

Improving communication and transparency about government’s priorities from a vaccination program to the manufacturers could help them develop products with higher chances of inclusion in clinical practice. This study focuses on identifying the influential vaccine characteristics based on country priorities, and helping the R&D identify the target product portfolio for their products. As different countries may have different challenges to overcome like lack of trained staff, storage constraints, non-compliance among population etc., it is very important for manufacturers to understand the country needs better while investing huge amount of resources in product development. However, post this the manufacturers would need to perform an internal evaluation and identify the challenges with clinical development, technological expertise and other manufacturing considerations, which may prevent them from developing a product as per the TPP identified earlier. Understanding the country priorities forms the first step for manufacturers and is the focus of our study, and we do not discuss about the challenges experienced by the manufacturer.

RESPONSE:

We thank the reviewer again for this helpful suggestion. As mentioned previously, have described how the self-assessments were thematically analyzed in further detail on lines 203 and 205 and have also added a supplementary file that summarizes the results of the thematic analysis in S6 Table. We direct the reader to this supplementary file in the results section.

RESPONSE:

The authorship team are not quite clear if the reviewer is referring to the ideal product for rotavirus vaccine or all vaccine products. If the latter, then we do not feel this is possible as the preferred product characteristics of a vaccine depends on the type of vaccine and we are against the notion that one size fits all.

Regarding the ideal characteristics of rotavirus vaccine for Thai stakeholders, this is a very important point made by the reviewer, we hope that we have answered this query to you satisfaction in the response to Comment 8/ Results.

18/ L284 – 285 Best and worst case scenarios are not described in the methods or the results.

RESPONSE:

Many thanks. As stated above, we have added a supplementary table (S3 Table) and text in the manuscript lines 189 and 190 (page 8) that describes the best and base case scenarios in further detail.

Minor comments

1/ Abstract L55-57 disclaimers about the use of information from anonymous stakeholders and ethics review considerations should also be reported in the text.

RESPONSE:

Thank you for the suggestion, we have these points in relation to ethics to the text on lines 201-203 (page 8).

2/ Line 88-89 it would be interesting listing the six countries and put their relative country context in perspective with the Thai context where the pilot study was carried out.

RESPONSE:

This paper is focusing primarily on the case of Thailand, so while this is an interesting angle we do not feel like it will strengthen the paper. Moreover, we may not be able to showcase the nuances of these country contexts. The authors believe it would be more useful for readers to refer to the paper cited. We are however appreciative of the reviewers’ suggestion.

2/ Line 116-117 “government priorities” seems broad, maybe government vaccine product preferences would be more appropriate.

RESPONSE:

Thank you. The authorship team have changed the wording as suggested.

3/ Line 158 change RRV5 TO RVV5

RESPONSE:

This correction has been made.

4/ Line 177 Table 4 is the first table referenced in the text

RESPONSE:

Thank you to the reviewer for their detailed review. We have now changed the order of the tables.

5/ L179 Three workshops were held but only two are described

RESPONSE:

Thank you to the reviewer for noticing this. A third meeting was convened to present the results of the pilot studies back to the stakeholders who attended the initial meeting in May. We have added a sentence about the third meeting on lines 206 to 207 (page 8).

6/ Table 1 is not referenced in the text and it’s not clear to which survey/form/workshops the respondents it refers to

RESPONSE:

Thank you. We have now referenced the table in the text and added the following title to the table “Profile of respondents from self-assessment forms”. This is now Table 4.

7/ Line 208 avoid repeating “criteria” and “identified” in the same sentence

RESPONSE:

We wholly agree with the reviewer that this sentence does not flow well as it stands. We have revised the sentence to read as follows “the top 5 decision-making criteria identified by stakeholders for choosing between rotavirus vaccine products were as follows: health impact, safety, budget impact, cost-effectiveness and delivery cost.” (Lines 213-214, pages 8-9)

8/ Line 245 the “**” sign is not referenced in the table

RESPONSE:

Thank you. We have revised accordingly.

9/ Line 252-253 revise the statement “preferred product characteristics of products”

RESPONSE:

The authorship have corrected this following the reviewers’ suggestion.

10/ L273 country preferences rather than country needs

RESPONSE:

We changed this as advised.

REVIEWER #2:

Vaccine manufacturers are extremely interested in anticipating the market demand during vaccine development. Today, evaluation of product’s desired characteristics is moving toward the earlier stages of development, and R&D activities are increasingly tailored to the target product profile (TPP) of the new vaccine. The limitation of the current approach is that, usually, vaccine manufacturers rely on the opinion of advisory groups in high income countries, which might not align with LMIC needs.

Suggestion is to add additional and more relevant references on the use of MCDA in vaccines such as https://doi.org/10.1016/j.vaccine.2016.10.086 and references quoted in this paper.

RESPONSE:

We thank the reviewer for their invaluable feedback and optimistic stance. We have added the reference Knobler et al (2017) in addition to Madhavan et al (2015) to the discussion section between lines 305 and 308 (page 15).

Attachment

Submitted filename: Response to Reviewers.docx

Click here for additional data file.^{(107.4KB, docx)}

PLoS One. doi: 10.1371/journal.pone.0233950.r003

Decision Letter 1

Ray Borrow

18 May 2020

‘It takes two to tango’: Bridging the gap between country need and vaccine product innovation

PONE-D-19-22507R1

Dear Dr. Archer,

We are pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it complies with all outstanding technical requirements.

Within one week, you will receive an e-mail containing information on the amendments required prior to publication. When all required modifications have been addressed, you will receive a formal acceptance letter and your manuscript will proceed to our production department and be scheduled for publication.

Shortly after the formal acceptance letter is sent, an invoice for payment will follow. To ensure an efficient production and billing process, please log into Editorial Manager at https://www.editorialmanager.com/pone/, click the "Update My Information" link at the top of the page, and update your user information. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to enable them to help maximize its impact. If they will be preparing press materials for this manuscript, you must inform our press team as soon as possible and no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

With kind regards,

Ray Borrow, Ph.D., FRCPath

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #2: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

Reviewer #1: Yes

Reviewer #2: (No Response)

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: (No Response)

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

Reviewer #1: Yes

Reviewer #2: Yes

**********

6. Review Comments to the Author

Reviewer #1: (No Response)

Reviewer #2: (No Response)

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

PLoS One. doi: 10.1371/journal.pone.0233950.r004

Acceptance letter

Ray Borrow

1 Jun 2020

PONE-D-19-22507R1

‘It takes two to tango’: Bridging the gap between country need and vaccine product innovation

Dear Dr. Archer:

I am pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please notify them about your upcoming paper at this point, to enable them to help maximize its impact. If they will be preparing press materials for this manuscript, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

For any other questions or concerns, please email plosone@plos.org.

Thank you for submitting your work to PLOS ONE.

With kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Prof. Ray Borrow

Academic Editor

PLOS ONE

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

S1 Table. Vaccine characteristics of the five hypothetical rotavirus vaccine products.

(DOCX)

Click here for additional data file.^{(14.1KB, docx)}

S2 Table. Input variables required for populating the TSE model.

(DOCX)

Click here for additional data file.^{(17.2KB, docx)}

S3 Table. Vaccine characteristics variables for the vaccine RVV-2 used in sensitivity analysis.

(DOCX)

Click here for additional data file.^{(13.3KB, docx)}

S4 Table. Scores and ranks for the five hypothetical rotavirus vaccine products using the TSE approach.

(DOCX)

Click here for additional data file.^{(13.2KB, docx)}

S5 Table. Scores and ranks for RVV-2, the second ranking vaccine product, in the base case and best case sensitivity analysis.

(DOCX)

Click here for additional data file.^{(13.2KB, docx)}

S6 Table. Thematic analysis results from self-assessment forms.

(DOCX)

Click here for additional data file.^{(12.5KB, docx)}

S1 File. Open-ended questionnaire distributed at initial stakeholder meeting.

(DOCX)

Click here for additional data file.^{(155.9KB, docx)}

S2 File. Self-assessment form distributed at stakeholder research and development meeting.

(DOCX)

Click here for additional data file.^{(55.3KB, docx)}

Attachment

Submitted filename: Response to Reviewers.docx

Click here for additional data file.^{(107.4KB, docx)}

Data Availability Statement

All relevant data are within the paper and its Supporting Information files.

[pone.0233950.ref001] 1.World Health Organisation. Universal health coverage (UHC). Available from: https://www.who.int/news-room/fact-sheets/detail/universal-health-coverage-(uhc). [Accessed April 17, 2019]

[pone.0233950.ref002] 2.Strategic Group of Experts in Immunization. 2018 Assessment report of the global vaccine action plan. Available from: http://apps.who.int/. [Accessed April 17, 2019]

[pone.0233950.ref003] 3.World Health Organisation. Health Systems Financing: The Path to Universal Coverage. Health Systems Financing: The Path to Universal Coverage. Available from: https://www.who.int/whr/2010/whr10_en.pdf?ua=1. [Accessed May 7, 2019]

[pone.0233950.ref004] 4.Kristensen DD, Bartholomew K, Villadiego S, Lorenson K. What vaccine product attributes do immunization program stakeholders value? Results from interviews in six low- and middle-income countries. Vaccine 2016;34(50):6236–42. 10.1016/j.vaccine.2016.10.057 [DOI] [PubMed] [Google Scholar]

[pone.0233950.ref005] 5.Popova O, Ibarra de Palacios P. Reaching more children with vaccines in developing countries: key challenges of innovation and delivery. Currrent Medical Research and Opinion 2016; 32(1):177–81. [DOI] [PubMed] [Google Scholar]

[pone.0233950.ref006] 6.Giersing BK, Kahn A-L, Jarrahian C, Mvundura M, Rodriguez C, Okayasu H, et al. Challenges of vaccine presentation and delivery: How can we design vaccines to have optimal programmatic impact? Vaccine [Internet]. 2017. December 14 [cited 2019 Apr 17];35(49):6793–7. [DOI] [PubMed] [Google Scholar]

[pone.0233950.ref007] 7.Hauck K, Thomas R, Smith PC. Departures from Cost-Effectiveness Recommendations: The Impact of Health System Constraints on Priority Setting. Heal Syst Reform [Internet]. 2016. January 2 [cited 2018 Oct 12];2(1):61–70. [DOI] [PubMed] [Google Scholar]

[pone.0233950.ref008] 8.World Health Organisation. Principles and considerations for adding a vaccine to a national immunization programme: from decision to implementation and monitoring. 2014 [cited 2018 Oct 12]; Available from: http://www.who.int/immunization/documents

[pone.0233950.ref009] 9.Lee J, Kim C, Shin J. Technology opportunity discovery to R&D planning: Key technological performance analysis. Technol Forecast and Social Change 2017;119:53–63. [Google Scholar]

[pone.0233950.ref010] 10.Sheerin D, Openshaw PJ, Pollard AJ. Issues in vaccinology: Present challenges and future directions. European Journal of Immunology 2017;47(12):2017–25. 10.1002/eji.201746942 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0233950.ref011] 11.Plotkin S, Robinson JM, Cunningham G, Iqbal R, Larsen S. The complexity and cost of vaccine manufacturing—An overview. Vaccine 2017;35(33):4064–71. 10.1016/j.vaccine.2017.06.003 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0233950.ref012] 12.Brooks A, Nunes JK, Garnett A, Biellik R, Leboulleux D, Birkett AJ, et al. Aligning new interventions with developing country health systems: target product profiles, presentation, and clinical trial design. Global Public Health 2012;7(9):931–45. 10.1080/17441692.2012.699088 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0233950.ref013] 13.Botwright S, Hutubessy R, Jit M, Meltzer M, Moore KL, El Omeiri N, et al. Country-led Assessment for Prioritisation on Immunisation (CAPACITI): A novel approach to support systematic priority-setting in low and middle income country immunisation programmes. Unpublished manuscript. World Health Organization; 2020 [Google Scholar]

[pone.0233950.ref014] 14.Marsh K, Thokala P, Youngkong S, Chalkidou K. Incorporating MCDA into HTA: challenges and potential solutions, with a focus on lower income settings. Cost Effectiveness Resource Allocation 2018;16(S1):43. [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0233950.ref015] 15.Frazão TDC, Camilo DGG, Cabral ELS, Souza RP. Multicriteria decision analysis (MCDA) in health care: a systematic review of the main characteristics and methodological steps. BMC Medical Informatics and Decision Making 2018;18(1):90 10.1186/s12911-018-0663-1 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0233950.ref016] 16.Inotai A, Nguyen HT, Hidayat B, Nurgozhin T, Kiet PHT, Campbell JD, et al. Guidance toward the implementation of multicriteria decision analysis framework in developing countries. Expert Review of Pharmacoeconomics & Outcomes Research 2018;18(6):585–92. [DOI] [PubMed] [Google Scholar]

[pone.0233950.ref017] 17.Rattanavipapong W, Kapoor R, Teerawattananon Y, Luttjeboer J, Botwright S, Archer R, et al. Comparing 3 Approaches for Making Vaccine Adoption Decisions in Thailand. International Journal of Health Policy and Management 2020; x(x), 1–9. [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0233950.ref018] 18.Marsh K, Ijzerman M, Thokala P, Baltussen R, Boysen M, Kaló Z, et al. Multiple Criteria Decision Analysis for Health Care Decision Making-Emerging Good Practices: Report 2 of the ISPOR MCDA Emerging Good Practices Task Force. Value in Health 2016;19:125–37. 10.1016/j.jval.2015.12.016 [DOI] [PubMed] [Google Scholar]

[pone.0233950.ref019] 19.Ford AQ, Touchette N, Hall BF, Hwang A, Hombach J. Global Vaccine and Immunization Research Forum: Opportunities and challenges in vaccine discovery, development, and delivery. Vaccine 2016;34(13):1489–95. 10.1016/j.vaccine.2015.11.038 [DOI] [PubMed] [Google Scholar]

[pone.0233950.ref020] 20.Knobler S, Bok K, Gellin B. Informing vaccine decision-making: A strategic multi-attribute ranking tool for vaccines—SMART Vaccines 2.0. Vaccine 2017; 35(1):A43–A45 [DOI] [PubMed] [Google Scholar]

[pone.0233950.ref021] 21.Madhavan G, Phelps C, Rappuoli R. Bridging the Gap: Need for a Data Repository to Support Vaccine Prioritization Efforts. Vaccine 2015; 33(2):B34–B39. [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0233950.ref022] 22.Wahlster P, Goetghebeur M, Kriza C, Niederländer C, Kolominsky-Rabas P. Balancing costs and benefits at different stages of medical innovation: A systematic review of Multi-criteria decision analysis (MCDA). BMC Health Services Research 2015;15(1):1–12. [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0233950.ref023] 23.Angelis A, Kanavos P. Multiple Criteria Decision Analysis (MCDA) for evaluating new medicines in Health Technology Assessment and beyond: The Advance Value Framework. Social Science & Medicine. 2017;188:137–56. [DOI] [PubMed] [Google Scholar]

[pone.0233950.ref024] 24.Ijzerman MJ, Koffijberg H, Fenwick E, Krahn Murray. Emerging Use of Early Health Technology Assessment in Medical Product Development: A Scoping Review of the Literature. Pharmacoeconomics 2017;35(7):727–740. 10.1007/s40273-017-0509-1 [DOI] [PMC free article] [PubMed] [Google Scholar]

PERMALINK

‘It takes two to tango’: Bridging the gap between country need and vaccine product innovation

Rachel A Archer

Ritika Kapoor

Wanrudee Isaranuwatchai

Yot Teerawattananon

Birgitte Giersing

Siobhan Botwright

Jos Luttjeboer

Raymond C W Hutubessy

Roles

Abstract

Background

Methods

Results

Conclusions

Introduction

Methods

Results

Criteria selected by decision makers in Thailand

Table 1. Top 5 decision making criteria for Thailand stakeholders for vaccine policy making.

Vaccine characteristics influencing the vaccine performance on decision criteria

Table 2. A breakdown of the influential vaccine characteristics.

Sensitivity analysis to identify significant vaccine characteristics

Table 3. Sensitivity analysis to identify significant vaccine characteristics for making RVV-2 the highest ranked vaccine.

Usefulness of TSE approach to inform vaccine R&D: R&D stakeholder perspective

Table 4. Profile of respondents from the self-assessment forms.

Discussion

Conclusion

Supporting information

Acknowledgments

Abbreviations

Data Availability

Funding Statement

References

Decision Letter 0

Ray Borrow

Roles

Author response to Decision Letter 0

Decision Letter 1

Ray Borrow

Roles

Acceptance letter

Ray Borrow

Roles

Associated Data

Supplementary Materials

Data Availability Statement

ACTIONS

PERMALINK

RESOURCES

Similar articles

Cited by other articles

Links to NCBI Databases