Skip to main content
F1000Research logoLink to F1000Research
. 2024 Oct 21;13:518. Originally published 2024 May 20. [Version 2] doi: 10.12688/f1000research.146025.2

The development of a roadmap for Health Technology Assessment implementation in Moldova

Gavin Surgey 1,2,a, Ghenadie Curocichin 3, Maria Garabajiu 3, Tanja Novakovic 4, Adrian Pana 5, Wija Oortwijn 2
PMCID: PMC11502998  PMID: 39464776

Version Changes

Revised. Amendments from Version 1

Some additional details are given in the paper, mostly to add to the clarity of the document. Additional points are given in the discussion section. An additional supplementary file and link are provided.

Abstract

Background

Moldova strives for universal health coverage (UHC) and considers health technology assessment (HTA) an important policy instrument to inform the choice of services that should be covered to progressively realize UHC. It plays a key role in determining which technologies are coved, considering various dimensions often including cost-effectiveness, budget impact, and feasibility. This paper reports on work undertaken to develop a roadmap for HTA implementation, using evidence-informed deliberative processes (EDPs) as the guiding framework.

Methods

Between 2020 and 2022, we undertook several activities that informed the roadmap. We conducted a needs assessment and an assessment of European HTA best practices using a combination of desk research, interviews and surveys. We then conducted a document review of six selected HTA systems, complimented by expert interviews from three neighbouring countries.

Results

The roadmap provided a suggested HTA process, which included instructions on how to execute the different steps of the EDP framework to enhance the legitimacy of HTA-informed decision-making. The roadmap encompasses several aspects such as how to organize stakeholder involvement, how to identify and operationalize decision criteria, and how to make the decision process transparent. Guidance was given to the country on establishing a legal framework for HTA; a communication strategy; strengthening capacity and leveraging HTA expertise. The country was also given tailored advice on the positioning of the HTA-agency, first as an entity within the Ministry of Health or the National Agency of Public Health but with the long-term vision for it to be independent of any state institution.

Conclusions

Despite several challenges, including a partial disruption because of the Covid-19 pandemic, the development of the roadmap for HTA implementation was completed and agreed upon by the Ministry of Health in Moldova. This work has helped motivate and support the country in HTA institutionalization.

Keywords: HTA, Health Technology Assessment, UHC, Priority-setting; Roadmap; Moldova; EDPs; Evidence-informed deliberative processes

Introduction

Like many countries worldwide that aim to achieve universal health coverage, Moldova views health technology assessment (HTA) as a significant policy tool as it plays a key role in determining which technologies are coved, considering various dimensions often including cost-effectiveness, budget impact, and feasibility.

HTA is widely used internationally to inform resource allocation decisions. It is defined as “a multidisciplinary process that uses explicit methods to determine the value of a health technology at different points in its lifecycle. The purpose is to inform decision-making to promote an equitable, efficient, and high-quality health system.” 1

While HTA is still in its infancy in Moldova, the country has been supported by partners for the past years to improve their capacity for HTA. This paper reports on the process used to develop a roadmap for HTA implementation and activities with the overall objective to establish the most suitable HTA system in Moldova. The activities executed between October 2020 and June 2022 were funded by the World Bank and supported by the Ministry of Health (MOH) and the National Health Insurance Company (CNAM) in Moldova.

We employ evidence-informed deliberative processes (EDPs) as the guiding framework 2 as well as experience with frameworks tested in other countries. The practical guidance on EDPs was developed by Radboudumc 3 and provides recommendations of how to implement each step of the decision-making process of benefits package design. The guide takes the current decision-making context in a country as the starting point, offering practical support depending on the country’s level of HTA development.

Country context

Moldova is one of the lowest income countries in Europe, with a Gross National Income (GNI) per capita of $4,570 in 2020. The country heavily relies on the agricultural sector, and its GDP per capita is estimated to be $5,231 in current US dollars. 4 The constrained economic situation in Moldova can lead health system challenges such as funding shortages, limitations in service provision and increased out–of–pocket payments for users.

In 2019, the portion of GDP allocated to healthcare spending was 3.8%, which fell below the average of 6% for the European Union (EU) and 5% for South-Eastern Europe (SEE). However, it exceeded the average of 2.7% for lower-middle-income countries (LMIC) within the WHO European Region. 5 Public spending on health as a share of GDP increased to 4.8% in 2020 likely due to an increase in spending related to Covid-19. The annual GDP change for the Republic of Moldova was –7% between 2019 and 2020 (World Bank, 2022). 6

A significant number of people in the Republic of Moldova still lack access to affordable and high-quality essential healthcare services. To address this issue, the Moldovan government established a national purchasing agency (CNAM), which aimed to pool individual and state contributions. The objective was to create a strong foundation for gradually expanding the coverage of healthcare services and medications provided by CNAM to a larger portion of the population. 7

While this initiative has resulted in improved accessibility to healthcare services and a decrease in unmet needs, Moldova still experiences a higher incidence of catastrophic health expenditure compared to other countries in the WHO European Region. This is primarily due to persistent gaps in healthcare coverage. 8 Although CNAM has been progressively increasing the number of medicines it covers, there are still essential medications that are not included, and outpatient medicine coverage remains limited.

The entitlement to publicly funded benefits under mandatory health insurance in Moldova is defined by the 1998 Law on mandatory health insurance (No.1585-XIII). The purpose of mandatory health insurance is to guarantee equal access to healthcare for all residents, who are obliged to be covered by CNAM.

Methods

The framework that we used for developing an HTA roadmap concerned evidence-informed deliberative processes (EDPs), combined with frameworks tested in other countries. 9 EDPs 3 can be used to guide countries to improve their decision-making processes, 10 , 11 and have been implemented successfully in a range of low, middle and high-income countries. 12 , 13 The core concept of EDPs centers around legitimacy and is broken down into four key elements, with stakeholder involvement and transparency being of utmost importance. The EDP framework offers a pragmatic and systematic approach tool for priority setting in health. Its primary objective is to enhance the credibility of benefit package choices and associated results, such as population health and financial risk mitigation. The activities encompassed in this project includes a needs assessment, an assessment of best practices and the development of the roadmap.

Needs assessment

We conducted a needs assessment to understand the decision-making context in which HTA could be used, stakeholders and their (potential) roles and, the capacity and skills for HTA within the country. The needs assessment consisted of three building blocks namely: a situational analysis, stakeholder analysis and a capacity and skills assessment through which we conducted interviews and surveys. To conclude the phase of the needs assessment, a virtual workshop was held with participants to show preliminary results from the needs analysis and to use the opportunity to conduct some capacity strengthening around HTA. Data collection instruments were developed and approved both by the Moldova National Ethical Committee (Date 22.12.2020 Nr. 1023) and Nicolae Testemitsanu SUMPh Committee for Research Ethics (Date 28.12.2020 Nr. 2). Data was collected between April– May 2021.

A list of potential participants was compiled through an online search and our knowledge of the situation in the country. The online search consisted of reviewing the MOH website, National Health Insurance (NHIC) website and a general Google search. We classified all stakeholders according to the 7Ps (policymakers, payers, product makers, principal investigators, patients and the public, providers, and purchasers) to ensure that we had a representative sample of each stakeholder. 14 Subsequently, our list of potential respondents was reviewed and revised by the World Bank and the MOH and an official letter of request to participate in the study was provided.

Situational analysis

The situational analysis aimed to establish the feasibility of EDP application in Moldova. We conducted a desktop review using standard health indicators and the list of services currently included in the MHI to match supply and demand. This was to help understand data availability relevant to HTA development, priority setting, and evidence-informed decision making; to help understand the current policy landscape that could support the institutionalisation of HTA, and to describe relevant stakeholders and the political landscape in Moldova.

In parallel, we collected views through semi-structured interviews with identified stakeholders to understand their knowledge of, position regarding, and interest in HTA.

The interview protocol 15 was based on Oortwijn et al. 16 which consisted of elements that reflect each step of the EDP framework and the contextual factors for HTA development.

Stakeholder analysis

The stakeholder analysis aimed to identify and analyse potential stakeholder roles. We adapted and applied a stakeholder mapping tool 15 developed by Vlad. 17

The different perspectives and roles of various stakeholders would help understand who might influence and shape governance structures and legislative measures to support HTA institutionalisation and how far evidence-building tools such as HTA would be taken into account in decision-making.

Capacity and skills assessment

Through a survey, 15 we assessed the available capacity and skills for HTA in Moldova of both ‘do-ers’ and users of HTA, although many of the respondents completed the survey in the presence of researchers. The survey was based on an existing set of tools tested out in other country assessments, 18 complemented with literature. 19

Assessment of best practices

To draw lessons in implementing HTA in Moldova we assessed best practices in HTA which involved a mapping of HTA systems in Europe. For this purpose, we undertook literature reviews and conducted interviews with key stakeholders.

Review of HTA systems

We first conducted a review of ‘comparative analyses’ from published literature in the last 5 years and then a focused analysis of six countries’ HTA systems. We did not use the information to compare countries, but rather to give an overview of the HTA landscape in Europe.

For the focused analysis we selected three early adopters (or mature) and three ‘recent’ (or nascent) adopter countries based on our review and experience. These were England, Scotland and Germany (early adopter countries) and Romania, Poland and Ukraine (recent adopters). 20

The focused analysis examined each country’s HTA system according to each step of the EDP framework: A. installing an advisory committee; B. selecting decision criteria; C. selection of health technologies for HTA; D1. scoping; D2. assessment; D3. appraisal; E. communication and appeal; and F. monitoring and evaluation. 3 For this, we reviewed key documents and websites of HTA bodies to assess practices both in terms of HTA institutional design and the HTA process.

Interviews with selected country key experts

For a selection of countries that could serve as an inspiration for Moldova, we conducted in-depth, semi-structured interviews with key experts and policymakers who were knowledgeable about their countries’ HTA system and to consider the transferability of the lessons learned to Moldova.

In choosing these countries, we started to develop a selection tool that includes the following criteria: similar health system financing and organization of health services delivery, although the perspective used in HTA may differ (health system verses societal perspective), level of HTA development (intermediate/mature level, e.g. Croatia, Poland, Romania, Ukraine), level of EDP development or interested in EDP uptake (e.g. Kazakhstan, Ukraine).

We identified 3-5 stakeholders per country, via our networks, and invited them for a telephone interview.

The interview focused on the establishment of HTA in the respective country, and the steps of EDPs where we validated and/or updated the findings from the literature review of HTA systems.

Development of a roadmap

Semi-structured in-depth interviews were conducted with senior Moldova representatives including three from the Ministry of Health, and four from the National Agency for Public Health to learn more about the Moldavian health system and the vision that for HTA in the country.

A workshop with stakeholders was held to validate the lessons to be drawn from the review and the neighbouring countries. The workshop also aimed to discuss key elements from other countries that would work in Moldova and where more work would be needed, both in terms of HTA institutionalisation and HTA processes.

The workshop also aimed to validate the recommendations developed and help create buy-in and legitimacy for the implementation plan to be developed.

Using a synthesis of findings from the needs assessment, assessment of best practices, and further discussions with key stakeholders from Moldova, a roadmap for HTA in Moldova was developed.

Results

Overview of respondents

There were 61 participants initially identified, and 55 were agreed to be contacted by e-mail. Potential participants who did not respond to the e-mail were sent a reminder 2 weeks after the initial invitation. On the second follow-up, six participants explicitly abstained from participation – due to several reasons including changing positions and a feeling of not being entitled to make decisions as their positions were solely technical in nature. A further six potential participants did not reply to the email nor phone calls. Respondents who accepted the invitation by e-mail were contacted by phone for scheduling the interview or agreed to complete the survey independently. An additional 3 technology producers were contacted through our networks as we had difficulty securing participants from this stakeholder category. Of these, 2 were successfully interviewed.

The number of respondents for each (7P) stakeholder category who participated are shown in Table 1.

Table 1. Overview of respondents.

Stakeholder category Activity participated in
Situational analysis Stakeholder analysis Capacity and skills assessment
Policymakers 7 6 4
Payers 5 5 5
Product makers 2 2 2
Principal investigators 5 5 7
Patients and the public 2 2 2
Providers 3 3 4
Total respondents 24 23 24

Concerning the interviews with selected country key experts, we aimed to get 3-5 respondents (in Poland, Romania and Ukraine). However, it was challenging to get a timely response from more than 5 so we decided to interview at least 3 experts in each country, representing different stakeholder groups ( Table 4). We did not reach out to additional stakeholders once we had a minimum of 3 stakeholders per country.

Table 4. Stakeholders interviewed for assessment of best practices.

Poland Romania Ukraine
Policymakers 2 2 2
Payers 1 - -
Principal investigators 1 1 2
Total 4 3 4

Situational analysis

Very few respondents felt that they were very familiar (33%) with HTA, although some (58%) indicated they had partial familiarity. When questioning respondents regarding the potential involvement of their organization in HTA, 17 respondents (71%) indicated that they or their organizations would have involvement in HTA indicating that this was the correct target audience for the survey.

Respondents indicated that Moldova is in the nascent phase of applying (elements of) EDPs for legitimate health benefits package design. There were, however, several elements that were shown to already be in place in Moldova which is in line with the HTA process including decision-making (advisory) committees with a level of legitimacy through government (legal) orders. The advisory committee for medicines called the “Council for Compensated Medicines” makes decisions on which medicines to include in their essential medicines list.

For all the steps and elements stipulated in the EDP framework, we probed whether there was a need for further guidance. Unsurprisingly, the majority of respondents felt that guidance was needed for all steps.

Stakeholder analysis

We received complete responses from a total of 23 individuals who were either interviewed or completed a survey form before the interview which we followed up on afterwards.

The stakeholder analysis helped identify relevant stakeholders and their role or level of importance in priority setting in Moldova. This is shown in Table 2.

Table 2. Stakeholders and importance in priority setting in Moldova.

Stakeholder category Key role Important Not important
Ministry of Health 87% 9% 0%
Ministry of Finance 39% 35% 13%
Any other central government body 20% 60% 10%
Elected decision-making committee 30% 61% 0%
Health insurers (government/private) 52% 35% 4%
Academic or research institutions 9% 70% 0%
External donors 13% 65% 13%
NGOs 0% 77% 5%
Healthcare providers (e.g. public/private hospitals) 18% 59% 9%
Clinicians 9% 70% 9%
Professional organisations (e.g. medical associations) 9% 78% 0%
Patient and/or carer organisations 4% 70% 13%
Pharmaceutical and/or devices industry 9% 74% 4%
Civil Society 0% 70% 13%
Any other group not mentioned 0% 22% 9%

We found that respondents reported several stakeholders involved in priority setting and that there is seemingly a collaborative environment between stakeholders.

In trying to understand the role of politicians/appointed decision-makers in defining the benefits package, respondents stated that their main role is around decision-making and approval of decisions. They noted that they have the responsibility for applying changes to the benefits packages such as updating with new medicines. There were no stated political tensions or disagreements. Civil servants and non-health professional decision-makers had similar roles, but there their specific role in the process was not clear. When it came to an understanding of the process of defining the benefits package it was reported that the MOH was responsible for coordinating the procedure with CNAM (the health insurer) and that there were appointed committees responsible for this, but the overall procedure was not clear to respondents.

The Center for Centralized Public Procurement in Healthcare (CAPCS) was stated as having responsibility for the procurement of medical products. The Agency for Medicines and Medical Devices (AMDM) was noted as having responsibility for negotiating prices, although there was no clear consensus among respondents. Respondents mentioned several stakeholders as having responsibility for pricing.

When it comes to decision-making, respondents felt that Patients or carer groups and the public were given a voice either partially (33%) or fully (27%). However, voting on decisions is restricted to only a few key policy-makers and this means that there is mostly consultation of stakeholders and very little participative decision-making.

Capacity and skills assessment

The majority of respondents to the survey had expertise in the field of public health and management. There was an equal split between those who classified themselves as decision-makers (users) or researchers (producers/do-ers).

We found that half of the respondents (50%) were part of, or had previously belonged to a government advisory committee of some sort and thus showed an average or above-average level of competence in related leadership or management skills. There was a good percentage of respondents (81%) who had experience with, and specific skills in conducting academic research and writing publications. This could partly be explained by the fact that there was a significant proportion of respondents (21%) who were employed by a university/research organisation.

Respondents rated their technical skills related to HTA as being moderate with more than half showing that they had specific experience in the majority of categories. Some respondents (31%) stated that they had read HTA reports from other countries, although there was a lower amount (15%) who stated that they had used the results from other countries.

However, when it came to specific technical skills (such as economic evaluations) the majority indicated a low level of confidence. Few showed confidence in conducting appraisals which is a key element of the HTA process where the results of the assessment need to be interpreted from a broader perspective and where recommendations are developed to inform decision-making.

Results relating to the capacity and skills assessment can be found in Table 3.

Table 3. Familiarity, experience and knowledge related to HTA.

Familiarity of HTA
Not at all Low Moderate Fairly familiar Very familiar
How familiar are you with HTA? 4.00% 20.00% 60.00% 8.00% 8.00%
Level of experience in HTA n %
Produced or used/referred to an hta report No, never produced or used an HTA report 8 30.80%
Yes, I only read the report from another country 8 30.80%
Yes, I read and used the results from the report from another country 4 15.40%
Yes, I have produced/contributed to an HTA report 6 23.10%
Ever used, or referred to a systematic review or other types of evidence synthesis (e.g. rapid review, meta-analysis)? Yes 20 76.90%
No 6 23.10%
Ever undertaken a (systematic) review or a meta-analysis of the clinical/medical literature No 9 34.60%
Yes, 1-5 reviews 9 34.60%
Yes, 6-10 reviews 3 11.50%
Yes, more than 10 reviews 5 19.20%
Ever undertaken an economic evaluation? Yes 14 53.80%
No 12 46.20%
How many economic evaluations? 1-3 8 4-5%
4-5 2 14.30%
6-7 1 7.10%
≥ 8 3 21.40%
Type of analysis Cost-benefit analysis (CBA) 8 57.10%
Cost-effectiveness analysis (CEA) 10 71.40%
Cost-utility analysis (CUA) 3 21.40%
Other (please specify) 0 0.00%
Ever used the results of an economic evaluation? Yes 10 38.50%
No 16 61.50%
Ever undertaken health-related quality-of-life studies? If yes, how many? No 12 46.20%
1-3 13 50.00%
4-5 1 3.80%
Ever used the results of health-related quality-of-life studies? Yes 17 65.40%
No 9 34.60%
Ever undertaken studies to assess ethical, social, cultural and legal issues; organizational and environmental aspects and/or implications for the patients, relatives, caregivers and the population None, 0 11 42.30%
1-3 13 50.00%
4-5 2 7.70%
Ever used the results of studies that assess ethical, social, cultural and legal issues; organizational and environmental aspects and/or implications for the patients, relatives, caregivers and the population Yes 18 69.20%
No 8 30.80%
Level of knowledge and confidence in conducting/using HTA activities
No knowledge and confidence Heard of it but not confident in doing any of it Slightly confident: Have some understanding in this area Confident: Can interpret results already produced in this area Expert: Can produce research and analysis in this area
Determining if the PICO (TS) (population, intervention comparator, outcome, (type of study, timing and setting) presented is correct 23.10% 15.40% 34.60% 19% 8%
Systematic literature reviews 3.90% 23.10% 26.90% 27% 19%
Differentiation between cost minimisation, cost-effectiveness, cost-utility and cost-benefit 0.00% 30.80% 30.80% 31% 8%
Determination of the impact of decisions on health inequity 11.50% 34.60% 19.20% 31% 4%

Workshop

Twenty-five individuals attended the workshop primarily from the MOH and CNAM. Preliminary results from the needs assessment were shown and discussed. The role of stakeholders was highlighted and there was discussion about how to further engage other stakeholders to actively participate in the process. Participants agreed that the level of understanding around HTA was too weak which might explain the current lower level of engagement around HTA.

Capacity strengthening was also undertaken to further sensitize on UHC, and HTA concepts and methods. EDPs were presented as a practical approach to implementing HTA in Moldova, and how these processes could be used to promote legitimate decision-making on the path to UHC.

Review of HTA systems

A review of ‘comparative analyses’ from published literature in the last 5 years shows that the majority (29 out of 31 countries, 94%) of countries (where information was available) use some elements of HTA to support decision-making about the use of pharmaceuticals. HTA activity was found to be lower for non-pharmaceutical health technologies with 71% of countries (22 out of 31 countries) using HTA to support decision-making.

The comparison of the six selected countries, early adopters (England, Scotland and Germany) and recent adopters (Poland, Romania and Ukraine) show that:

  • In early adopter countries, stakeholders are involved in nearly all HTA processes which is not the case in recent adopter countries. It appears that there is no stakeholder engagement in most steps;

  • Early adopters (Germany and England) conduct explicit monitoring and evaluation of decisions, while recent adopters seem not to monitor or evaluate the results of HTA decisions;

  • Training of stakeholders exists in early adopter countries and Poland, but not in Ukraine and Romania;

  • Assessment and appraisal phases are the most developed processes across all countries, with methodological HTA guidelines in place in England, Poland and Ukraine. The Polish appraisal process seems to have the highest level of transparency compared to the other two recent adopters (Romania and Ukraine).

Interviews with selected country key experts

Discussions with key experts gave insight as to some of the reasons the country had introduced HTA and some conducive factors. How HTA was set up and other recommendations for newly establishing HTA in a country.

The main reasons respondents gave for introducing HTA were economic concerns such as budget around spending on health and the need for cost-containment. Some conducive factors to introducing HTA was political will supported by new legislation to enforce HTA. Other key success factors that respondents noted included building support for various stages in the establishment of HTA and having a close collaboration with all stakeholders including industry and patients throughout the process.

Experts also gave some recommendations to Moldovan policymakers on establishing HTA. These included ensuring the involvement of multiple stakeholders and building support for the various stages of HTA and establishing a solid methodology and processes containing all the steps for HTA. Experts also advised on the importance of creating an HTA assessment body independent of state authorities to make decisions regarding reimbursement and building the right legislation around this. It was also recommended to continuously invest and build the institutional capacity of those involved in HTA in the country.

Development of a roadmap

A final workshop was held with approximately 25 individuals primarily from the MOH and CNAM. Experiences and best practices were shared by presenters who lead HTA activities in Poland and Ukraine. Presentations and discussions were held on the current situation of Moldova (issues and challenges) and how the country can move towards the establishment of an HTA system.

During the workshop it was discussed where the HTA unit should be based - it was agreed that for now it should start with either the MOH or the National Agency of Public Health but the long-term vision would be for it to be independent of any state institutions. The main purpose agreed upon during the workshop was for the HTA unit to focus on developing the benefits package and keeping the Essential Medicines List up to date.

The development of the roadmap took into consideration all the findings and other discussions that were held with Moldavian stakeholders instrumental in the design of HTA. The roadmap covered the HTA institutional design in terms of the responsibilities of participating organizations in implementing HTA evidence into policy. The roadmap also gave details of a proposed HTA process including guidance on how to implement the steps of the EDP framework. This included providing suggestions on how to effectively organize stakeholder involvement, how to potentially identify and operationalize decision criteria, and how to best make the decision process transparent. Specific guidance was given to the country on establishing a legal framework for HTA; a communication strategy; strengthening capacity and leveraging HTA expertise. Recommendations given were in line with the 2020 WHO report “Principles of health benefit packages”, 21 and the “roadmap for systematic priority setting and HTA” by Castro, Kumar, Suharlim et al. 22

Discussion

This paper outlines the components that can support the successful development of a roadmap for HTA implementation in Moldova. HTA leads to better informed benefit package and improved access to essential services which is critical for the achievement of UHC. 23 , 24 Through this research, we find that there is a high demand and appetite by all stakeholders for HTA in the country. It showed that there is a strong political will, but this is driven by very few within the government making the success of Moldova very dependent on a few individuals who have already stretched agendas. There are a few stakeholders that have received some training on HTA however, there is still a lot more sensitization needed across all stakeholder groups. There are limited resources including human and financial resources but also a limited amount of country-specific data. However, the country has some decision-making systems and processes that can be harnessed for HTA. We believe that any additional guidance on existing processes will only strengthen the processes for Moldova.

Currently, there are a handful of activities taking place to establish HTA in the country which were known by respondents including demonstration projects and some training courses in the field of HTA. However, given the number of stakeholders (individuals and departments) who were reported to have kickstarted the establishment of HTA, we might expect to see more activities currently taking place.

The situation described in the paper largely aligns with what we would expect ex-ante for a country with Moldova's history and government. As a lower-middle-income country and one of the poorest in Europe, Moldova faces significant challenges in healthcare provision and resource allocation. Moldova has had a difficult economic transition following the dissolution of the Soviet Union, which has negatively impacted its health system. This is consistent with the experiences of many post-Soviet states. The nascent state of HTA in Moldova is also in line with expectations. Many post-Soviet countries have struggled to implement modern health policy tools like HTA, often due to limited resources, institutional capacity, and the need to overhaul legacy systems (which include decision making structures) from the Soviet era.

Compared to these (other nascent) countries, Moldova is at an earlier stage of HTA implementation. Our analysis show that Moldova is still in the process of developing capacity and establishing governance structures. There is now a legal act that supports institutionalisation of HTA in the country. By learning from both regional peers and more established HTA systems, the new roadmap aims to accelerate Moldova's progress in this area.

The surveys conducted did have some shortfalls, for example when measuring skills, we did not provide a definition of terms meaning respondents could have interpreted these differently. Results are therefore prone to measurement bias. We saw this when asking knowledge questions, i.e. asking a respondent to rate their knowledge on a topic and then asking them a verification question on that topic. We found that they did not always rate their knowledge correctly. As such, there is a need for creating awareness, training and education in HTA. Stronger technical skills are needed in using different types of evidence (reviewing evidence) and interpreting economic evaluations.

Involving multiple stakeholders is expected to enhance the credibility of Moldova's decision-making process. 3 Two successful workshops were conducted with stakeholders to ensure alignment of objectives and consensus on advancing Health Technology Assessment (HTA) in the country. In these sessions, stakeholders reached a mutual understanding regarding the rationale, goals, and scope of HTA initiatives. Institutionalizing HTA and enhancing the legitimacy of this process will reinforce the overall credibility of decision-making and foster greater transparency, given the comprehensive involvement and informed status of numerous stakeholders.

The overall project and development of the roadmap did face a challenge in that almost all of the work was developed remotely given the disruption due to COVID-19. Implementation of the roadmap will likely also experience delays due to pressing issues related to COVID-19 and the current situation in Ukraine. Moldova has, however, started to take some of these recommendations forward in developing capacity-strengthening options with local universities. Building skills in this area will help drive the process of institutionalization of HTA in Moldova.

There were some additional limitations, the desk review conducted for this project primarily relied on publicly available documents in English, which may have excluded important local materials in Romanian. As a result, the understanding of Moldova's current HTA landscape may be incomplete. Furthermore, Moldova's rapidly changing health policies mean that some of the information gathered could have become outdated by the time of publication.

One of the strengths of this study is the multi-stage sequential mixed-method approach, which allowed us to assess needs methodically and offer solutions. It allowed for iterative development but also presented the risk of earlier stages influencing later findings in ways that may not have been fully addressed. Additionally, the rapid pace of the project, combined with the constraints of the COVID-19 pandemic, may have limited the team's ability to fully validate all findings with local stakeholders.

An important contribution of this paper is to demonstrate how countries can institutionalise HTA for the achievement of UHC. Critical to the success is the continuous political will and determination of the government 25 of coupled with comprehensive stakeholder engagement. 26 , 27 From here, HTA should be linked to policy decision-making supported by the HTA legislation with a permanently financed core team focused on HTA. 28 At the same time, developing formal HTA training capacity so that over time, skills and expertise are built. This needs to be accompanied by continuous awareness raising among users of HTA. International collaboration with other countries that are further in the HTA process as well as international networks such as HTAi and INAHTA can support Moldova in the implementation of the roadmap.

Ethical considerations and consent

Ethical approval for this study was obtained from the National Committee for Ethical Expertise of Clinical Trial (NCEECT/1023/22.12.2020) on 22.12.2020 and Nicolae Testemitanu State University of Medicine on and Pharmacy (94) on 21.12.2020. All participants gave either verbal or written informed consent depending on the nature of the survey. Verbal informed consent was used when participants were supported in completing a survey form, or when the survey form was completed on their behalf.

Acknowledgements

We would like to thank to officials in the Republic of Moldova from the Ministry of Health and CNAM. We are also sincerely grateful to Rob Baltussen who helped shape the project in the development phase and supported the workshops. Finally, we appreciate the continuous support of Volkan Cetinkaya and Ilie Volovei and the rest of the World Bank Project Team.

Funding Statement

This research was funded by the World Bank Group, Contract No. 7197841 financed through TF0B1249 - SDC support towards Universal Health Coverage in Moldova

The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

[version 2; peer review: 3 approved]

Data availability

OSF: Roadmap for HTA implementation in Moldova [dataset] OSF: DOI 10.17605/OSF.IO/NPKBF

File ‘Moldova HTA Roadmap data’ contains individual underlying data for each 3 surveys

File ‘1. Instrument - Situational Analysis.docx’ contains situational analysis instrument

File ‘2. Instrument - Stakeholder Analysis.docx’ contains stakeholder analysis instrument

File ‘3. Instrument - Capacity Skills Assessment.docx’ contains capacity skills assessment instrument

File ‘4. Review of early and recent adopter countries against EDP steps.docx’ contains individual country level results

File ‘5. Sources - Review of HTA systems

All data underlying the results are available as part of the article and no additional source data are required.

Data are available under the terms of the CC0 licence CC0 1.0 Universal

References

  • 1. O’Rourke B, Oortwijn W, Schuller T: The new definition of health technology assessment: A milestone in international collaboration. Int. J. Technol. Assess. Health Care. 2020 Jun;36(3):187–190. 10.1017/S0266462320000215 [DOI] [PubMed] [Google Scholar]
  • 2. Oortwijn W, Jansen M, Baltussen R: Evidence-informed deliberative processes for health benefit package design–part II: a practical guide. Int. J. Health Policy Manag. 2022 Oct;11(10):2327. 10.34172/ijhpm.2021.159 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3. Oortwijn W, Jansen M, Baltussen R: Evidence-informed deliberative processes. A practical guide for HTA bodies for legitimate benefit package design. Version 2.0. Nijmegen: Radboud University Medical Center;2021. [Google Scholar]
  • 4. World Bank. Overview - Moldova [Internet]. [cited 2023.03.01]. : Reference Source
  • 5. World Health Organization: Health systems in action: Republic of Moldova: 2022 edition. 2022.
  • 6. World Bank: World Development Indicators. Washington, DC: World Bank;2022. [Google Scholar]
  • 7. Garam I, Zadnipru M, Doronin V, et al. : Can people afford to pay for health care? New evidence on financial protection in the Republic of Moldova. 2020.
  • 8. Cylus J, Thomson S, Evetovits T: Catastrophic health spending in Europe: equity and policy implications of different calculation methods. Bull. World Health Organ. 2018;96(9):599–609. 10.2471/BLT.18.209031 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9. Jeffery M, Chi YL, Stewart M: iDSI Health Technology Assessment Toolkit [version 1; not peer reviewed]. F1000Res. 7(1545). 10.7490/f1000research.1116126.1 [DOI] [Google Scholar]
  • 10. Baltussen R: Value assessment frameworks for HTA agencies: The organisation of evidence-informed deliberative processes. Value Health. 2017;20:256–260. 10.1016/j.jval.2016.11.019 [DOI] [PubMed] [Google Scholar]
  • 11. Oortwijn W, Jansen M, Baltussen R: Use of evidence-informed deliberative processes by health technology assessment agencies around the globe. Int. J. Health Policy Manag. 2020;9(1):27–33. 10.15171/ijhpm.2019.72 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12. Nouhi M, Baltussen R, Razavi S, et al. : The use of evidence-informed deliberative processes for health insurance benefit package revision in Iran. Int. J. Health Policy Manag. 2022;11(11):2719–2726. 10.34172/ijhpm.2022.6485 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13. Oortwijn W, Surgey G, Novakovic T, et al. : The Use of Evidence-Informed Deliberative Processes for Health Benefit Package Design in Kazakhstan. Int. J. Environ. Res. Public Health. 2022 Sep 10;19(18):11412. 10.3390/ijerph191811412 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14. Concannon T, Meissner P, Grunbaum J: A new taxonomy for stakeholder engagement in patient-centered outcomes research. J. Gen. Intern. Med. 2012;27(8):985–991. 10.1007/s11606-012-2037-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15. Surgey G, Oortwijn W, Curocichin G, et al. : Roadmap for HTA implementation in Moldova.[dataset].2024 [cited 2024 Mar 21]. Reference Source [DOI] [PMC free article] [PubMed]
  • 16. Oortwijn W, Oosterhout S, Kapiriri L: Application of evidence-informed deliberative processes in health technology assessment in low-and middle-income countries. Int. J. Technol. Assess. Health Care. 2020;36(4):440–444. 10.1017/S0266462320000549 [DOI] [PubMed] [Google Scholar]
  • 17. Vlad I: Stakeholder checklist for priority-setting in low and middle-income countries [version 1; not peer reviewed]. F1000Res. 2018;7:1718. [Google Scholar]
  • 18. Surgey G, Mori AT, Baltussen R: Health Technology Assessment in Tanzania: Capacity and Experience of HTA Committee Members. J. Glob. Health Econ. Policy. 2022;2:e2022004. 10.52872/001c.33116 [DOI] [Google Scholar]
  • 19. Pichler F: Defining Capacity Building in the Context of HTA: A Proposal by the HTAi Scientific Development and Capacity Building Committee. Int. J. Technol. Assess. Health Care. 2019;35(5):362–366. 10.1017/S0266462319000631 [DOI] [PubMed] [Google Scholar]
  • 20. Beletsi A, Koutrafouri V, Karampli E, et al. : Comparing use of health technology assessment in pharmaceutical policy among earlier and more recent adopters in the European Union. Value Health Reg Issues. 2018;16:81–91. 10.1016/j.vhri.2018.08.002 [DOI] [PubMed] [Google Scholar]
  • 21. World Health Organization: Principles of Health Benefit Packages. Geneva: World Health Organization;2021. [Google Scholar]
  • 22. Castro HE, Kumar R, Suharlim C: A Roadmap for Systematic Priority Setting and Health Technology Assessment (HTA). Arlington, VA: USAID/MSH;2020. [Google Scholar]
  • 23. Bertram M, Dhaene G, Tan-Torres ET: Institutionalizing Health Technology Assessment Mechanisms: A How-to Guide. Bertram M, Dhaene G, Tan-Torres Edejer T, editors. Geneva: World Health Organization;2021. [Google Scholar]
  • 24. Norheim O, Watkins D: The Role of HTA for Essential Health Benefit Package Design in Low or Middle-Income Countries. Health Syst. Reform. 2023;9(3):2273051. 10.1080/23288604.2023.2273051 [DOI] [PubMed] [Google Scholar]
  • 25. Mueller D: Addressing the challenges of implementing a Health Technology Assessment Policy framework in South Africa. Int. J. Technol. Assess. Health Care. 2020 Aug;36(4):453–458. 10.1017/S0266462320000562 [DOI] [PubMed] [Google Scholar]
  • 26. Mitton C, Smith N, Peacock S, et al. : Public participation in health care priority setting: a scoping review. Health Policy. 2009 Aug 1;91(3):219–228. 10.1016/j.healthpol.2009.01.005 [DOI] [PubMed] [Google Scholar]
  • 27. Kapiriri L, Baltussen R, Oortwijn W: Implementing evidence-informed deliberative processes in health technology assessment: a low income country perspective. Int. J. Technol. Assess. Health Care. 2020;36(1):29–33. 10.1017/S0266462319003398 [DOI] [PubMed] [Google Scholar]
  • 28. Surgey G, Chalkidou K, Reuben W, et al. : Introducing health technology assessment in Tanzania. Int. J. Technol. Assess. Health Care. 2020;36(2):80–86. 10.1017/S0266462319000588 [DOI] [PubMed] [Google Scholar]
F1000Res. 2024 Nov 25. doi: 10.5256/f1000research.173342.r334159

Reviewer response for version 2

Eleanor Grieve 1

Thank you for the opportunity to review this research into the process of initiating HTA in a country new to these systems.  A very thorough needs assessment was undertaken (situational analysis, stakeholder analysis and capacity / skills assessment); an assessment of European HTA best practices using a combination of desk research, interviews and surveys;  a document review of six selected HTA systems; and expert interviews from three neighbouring countries.  The country was given advice on the positioning of the HTA-agency.

I don’t have too much to add to the other two thoughtful reviews.  My main thoughts are around the benefit of taking a deeper dive into each type of stakeholder, or perhaps specifically industry and academia/research (they are both listed in Table 2). 

The emphasis is mostly on government clearly as policy/decision makers but for a country embarking on an HTA journey, it would be good to understand HTA from the perspective of the country’s relationships with pharma/industry and research. There is some albeit brief mention of both. 

For pharma, the Center for Centralized Public Procurement in Healthcare is stated as having responsibility for the procurement of medical products, and the Agency for Medicines and Medical Devices for price negotiations.  More information here would be good, for example, are price negotiations based purely on budgetary concerns or value / cost-effectiveness too?  What is the process?  Where would HTA sit in this process?

Likewise, with research, it is briefly state that there is a limited amount of country-specific data.  Is this referring to routinely collected health monitoring data or to research generating country-specific evidence.

HTA needs to be linked into both research/evidence generation and industry as well as government and so perhaps more on the former two could be added or explored if out of scope in this study.

Is the work clearly and accurately presented and does it cite the current literature?

Yes

If applicable, is the statistical analysis and its interpretation appropriate?

Not applicable

Are all the source data underlying the results available to ensure full reproducibility?

No source data required

Is the study design appropriate and is the work technically sound?

Yes

Are the conclusions drawn adequately supported by the results?

Yes

Are sufficient details of methods and analysis provided to allow replication by others?

Yes

Reviewer Expertise:

Health Economics and Health Technology Assesment

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

F1000Res. 2024 Oct 24. doi: 10.5256/f1000research.173342.r333444

Reviewer response for version 2

Christian Suharlim 1

No further comments. Thank you for the clarification

Is the work clearly and accurately presented and does it cite the current literature?

Yes

If applicable, is the statistical analysis and its interpretation appropriate?

Partly

Are all the source data underlying the results available to ensure full reproducibility?

No

Is the study design appropriate and is the work technically sound?

Yes

Are the conclusions drawn adequately supported by the results?

Yes

Are sufficient details of methods and analysis provided to allow replication by others?

Partly

Reviewer Expertise:

Health Technology Assessment in LMICs; economic evaluation

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

F1000Res. 2024 Jun 25. doi: 10.5256/f1000research.160056.r288867

Reviewer response for version 1

Kanchan Mukherjee 1

This paper documents the process used for developing a road map for HTA implementation in Moldova. The methodology involved using the EDP framework, which has been applied before, and thus provides transparency with scope for replicability. The steps included needs assessment (situational analysis, stakeholder analysis, capacity assessment), which provided an internal contextual assessment of the HTA situation in Moldova. This was complemented by assessment of best practices, and neighboring countries key expert interviews, which provided the external contextual analysis. Overall, the above activities suggest a holistic approach to the situation, while acknowledging the contextual influence. Contextualisation of HTA is extremely important and has also been reported in literature in creating a road map for HTA in India, which is a low-middle-income country (like Moldova) (Mukherjee K et al, 2017) [Ref 1]

The methodology followed is comprehensive and justifiable given the objective of this exercise. It also reflects a well thought out process which is systematic and specific, while acknowledging the limitations of such a process. The conclusions and discussion reflect the positive aspects (high demand by stakeholder) as well as the challenges (few policy champions for HTA, low HTA capacity).

India also encountered a similar situation like Moldova, when the process of HTA was first conceived at the political level and continues to face similar challenges in its implementation. From my HTA experience in the  Indian context, I am sharing a few learnings, which maybe useful for Moldova:

  1. Political institutionalization is extremely important, but the HTA body should be an autonomous entity to avoid political interference in the HTA process.

  2. Academic institutionalization of HTA through local universities is a sustainable model for capacity building in HTA. Hence, it may also be useful to consider academic institutionalization of HTA in local universities in the form of Masters/PhD programme or short term training programmes, in which policy makers/implementers, can also be nominated to be trained in the HTA process. This model has been implemented in India since 2010 and has resulted in a demand creation in this niche area as well as increased sensitization and awareness among policy makers about HTA (Mukherjee K, et al 2020) [Ref 2].

  3. It may be useful to have small pilot HTA projects (‘mini-HTA’) at the initial stages to provide credibility, legitimacy and acceptance by various stakeholders. Hence, simple incremental steps in HTA based on existing resources and decision making mechanisms maybe useful before using HTA for complex issues.

  4. The HTA ecosystem should be an open system receptive to learnings (internal and external), adaptive and responsive to evolving challenges over time. Hence, it would be useful to have a follow up review to assess the extent of implementation of HTA in Moldova.

To conclude, I congratulate the team for their efforts in this exercise, which by itself would have resulted in sensitization of stakeholders and started the first step towards HTA implementation in Moldova.

Is the work clearly and accurately presented and does it cite the current literature?

Yes

If applicable, is the statistical analysis and its interpretation appropriate?

Not applicable

Are all the source data underlying the results available to ensure full reproducibility?

Yes

Is the study design appropriate and is the work technically sound?

Yes

Are the conclusions drawn adequately supported by the results?

Yes

Are sufficient details of methods and analysis provided to allow replication by others?

Yes

Reviewer Expertise:

HTA, Health Policy, Public Health

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

References

  • 1. : Health technology assessment: A potential roadmap for India. International Journal of Medical Science and Public Health .2017; 10.5455/ijmsph.2017.1269225012017 10.5455/ijmsph.2017.1269225012017 [DOI] [Google Scholar]
  • 2. : A SMART framework for HTA capability development: Lessons from India. Health Policy and Technology .2020;9(1) : 10.1016/j.hlpt.2019.12.001 42-44 10.1016/j.hlpt.2019.12.001 [DOI] [Google Scholar]
F1000Res. 2024 Jun 14. doi: 10.5256/f1000research.160056.r288862

Reviewer response for version 1

Christian Suharlim 1

Thank you for the opportunity to review this paper. The authors did an excellent job in describing, in detail, the activities conducted in order to assess the needs, propose improvements, and develop the HTA Roadmap. I suggest a few revisions below:

  1. Authors described in [Results-Stakeholder analysis] "However, voting on decisions is restricted to only a few key policy-makers and this means that there is mostly consultation of stakeholders and very little participative decision-making." I would have liked to see a deeper dive into this aspect in the Discussion section, e.g., Does this align with what researchers would expect ex-ante for a country with Moldova's history and government? Does the comparative analysis shed insight on how Moldova compares in this regard to other post-Soviet states? How is this improved in the new roadmap?

  2. Authors put quotation marks around the term comparative analyses. I wonder why this is the case. I would have liked to have more detail on how the 5-year published literature desk review was conducted under [methods] and a table of the findings under [results] and [Supplementary data]

  3. In the introduction, the Authors specifically mention how Moldova is lower on the development level in Europe- as proxied by GNI/capita. However, the six focused countries are mostly high-income countries with vastly different development levels. Under [Discussion], I would like to see how authors synthesize the impact of the development level difference between these countries and Moldova and how it may impact the generalizability and applicability of a country with a much lower development level. In particular, it would be excellent if authors could comment on human resource capacity, e.g., human flight and brain drain, and how that may impact a country with such issues to meet the unfair expectations of the gold standards set by early adopters.

  4. Under [Results-development of a roadmap] "Experiences and best practices were shared with presenters" - or "by presenters"?

  5. Under [Discussion], the authors state, "It showed that there is a strong political will, but this is driven by very few within the government, making the success of Moldova very dependent on a few individuals who have already stretched agendas." I would have expected this to be the case in most countries with lower development levels.Does this align with the Authors' desk review / comparative analyses? If this is indeed the typical lay of the land in LMICs, is there a precedent to more positive wording (e.g., 'local champions'?)

  6. The authors state their limitations in the survey. I would have liked to see if authors could synthesize their thinking on the current limitations with the desk review, interviews, sample size and selection, and interpretation of data.

  7. The authors made a point about improving stakeholder involvement. However, I would have liked to see the intensity and extent of these 'new stakeholders' contributions. What (if any) decisions would have been different had they not been a part of the contribution? Beyond the cited theory and paper, in Moldova, how would their involvement improve the legitimacy of decisions? Were there explicit buy-ins and endorsements by the new stakeholders' affiliated organizations?

  8. Lastly, this is a really well-done paper and process. The authors would benefit from more explicit statements that one of the strengths of this study is the multi-stage sequential mixed-method approach, which allowed them to assess needs methodically and offer solutions.

Congratulations for the excellent work and a job well-done!

Best regards,

Chris Suharlim

Is the work clearly and accurately presented and does it cite the current literature?

Yes

If applicable, is the statistical analysis and its interpretation appropriate?

Partly

Are all the source data underlying the results available to ensure full reproducibility?

No

Is the study design appropriate and is the work technically sound?

Yes

Are the conclusions drawn adequately supported by the results?

Yes

Are sufficient details of methods and analysis provided to allow replication by others?

Partly

Reviewer Expertise:

Health Technology Assessment in LMICs; economic evaluation

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

F1000Res. 2024 Oct 10.
Gavin Surgey 1

Thank you for the comments. We have responded to each of the comments in turn, and indicated where we have made additions to the paper. Thank you for the suggestions – we believe they have added to the overall paper.

1. Authors described in [Results-Stakeholder analysis] "However, voting on decisions is restricted to only a few key policy-makers and this means that there is mostly consultation of stakeholders and very little participative decision-making." I would have liked to see a deeper dive into this aspect in the Discussion section, e.g., Does this align with what researchers would expect ex-ante for a country with Moldova's history and government? Does the comparative analysis shed insight on how Moldova compares in this regard to other post-Soviet states? How is this improved in the new roadmap?

Ans: We have added additional information related to:

  • Alignment with expectations for Moldova – a paragraph was added in the discussion section

  • Comparison to other post-Soviet states

  • Improvements in the new roadmap and how learning from both regional peers and more established HTA systems, the new roadmap aims to accelerate Moldova's progress in this area.

2.  Authors put quotation marks around the term comparative analyses. I wonder why this is the case. I would have liked to have more detail on how the 5-year published literature desk review was conducted under [methods] and a table of the findings under [results] and [Supplementary data]

Ans: We attempted to make this more clear by adding additional information in the paper including: We did not use the information to compare countries, but rather to give an overview of the HTA landscape in Europe.

We did not include a table of findings for the comparative analysis as this mainly served as background information for selecting countries for the focused analysis – and this was not the aim of the analysis. The table of findings for the focused analysis are presented in supplementary data file:

File ‘4. Review of early and recent adopter countries against EDP steps.docx’ contains individual country level results

An additional supplementary file was added (#5) to give a listing of the of the resources we used to gather the findings.

3. In the introduction, the Authors specifically mention how Moldova is lower on the development level in Europe- as proxied by GNI/capita. However, the six focused countries are mostly high-income countries with vastly different development levels. Under [Discussion], I would like to see how authors synthesize the impact of the development level difference between these countries and Moldova and how it may impact the generalizability and applicability of a country with a much lower development level. In particular, it would be excellent if authors could comment on human resource capacity, e.g., human flight and brain drain, and how that may impact a country with such issues to meet the unfair expectations of the gold standards set by early adopters.

The countries were selected to serve as inspiration.

Ans: We added information: We will select 2-3 countries that could serve as an inspiration for HTA development in Moldova. In choosing these countries, we started to develop a selection tool that includes the following criteria: similar health system financing and organization of health services delivery, although the perspective used in HTA may differ (health system verses societal perspective),  level of HTA development (intermediate/mature level, e.g. Croatia, Poland, Romania, Ukraine), level of EDP development or interested in EDP uptake (e.g. Kazakhstan, Ukraine).

4. Under [Results-development of a roadmap] "Experiences and best practices were shared with presenters" - or "by presenters"?

Ans: Thanks for picking this up: “by presenters” is correct

5. Under [Discussion], the authors state, "It showed that there is a strong political will, but this is driven by very few within the government, making the success of Moldova very dependent on a few individuals who have already stretched agendas." I would have expected this to be the case in most countries with lower development levels. Does this align with the Authors' desk review / comparative analyses? If this is indeed the typical lay of the land in LMICs, is there a precedent to more positive wording (e.g., 'local champions'?)

Ans: The terms "political will" and "local champions" are related but distinct. A "local champion" usually refers to an individual, not necessarily in government, who drives forward a specific agenda. In contrast, "political will" refers to the collective determination of a group of individuals within the government to pursue a particular course of action. While a local champion may lead or inspire efforts, political will reflects a broader, shared commitment within the governing body. We have not made any changes to this

6. The authors state their limitations in the survey. I would have liked to see if authors could synthesize their thinking on the current limitations with the desk review, interviews, sample size and selection, and interpretation of data.

Ans: Some additional details relating to the limitations were added to the discussion section relating to the desk review. We also added additional paragraph on the multi-stage approach.

7. The authors made a point about improving stakeholder involvement. However, I would have liked to see the intensity and extent of these 'new stakeholders' contributions. What (if any) decisions would have been different had they not been a part of the contribution? Beyond the cited theory and paper, in Moldova, how would their involvement improve the legitimacy of decisions? Were there explicit buy-ins and endorsements by the new stakeholders' affiliated organizations?

Ans: We appreciate the suggestion to further analyze the impact of expanded stakeholder involvement. You raise good questions about the tangible effects of including new stakeholders. We did not assess how specific decisions differ. We can at this time only refer to theoretical arguments and examples of other countries where stakeholder involvement improves legitimacy of decisions as well as buy-ins and endorsements. However, we agree this could be more thoroughly analysed as a separate piece of future work.

8. Lastly, this is a really well-done paper and process. The authors would benefit from more explicit statements that one of the strengths of this study is the multi-stage sequential mixed-method approach, which allowed them to assess needs methodically and offer solutions.

Ans: Thanks for this suggestion, it has been added to the paper.

We appreciate these insightful comments which will help strengthen both this paper and our ongoing work supporting HTA development in Moldova and other countries.

  • Are sufficient details of methods and analysis provided to allow replication by others?

All the instruments and source data is available online. This is detailed in the “Data availability section”: OSF: Roadmap for HTA implementation in Moldova [dataset] OSF: DOI  10.17605/OSF.IO/NPKBF

 

  • If applicable, is the statistical analysis and its interpretation appropriate?

Partly

Descriptive analysis techniques were used. These methods are detailed in the paper with the individual results in the source data.

OSF: Roadmap for HTA implementation in Moldova [dataset] OSF: DOI  10.17605/OSF.IO/NPKBF

 

  • Are all the source data underlying the results available to ensure full reproducibility?

No

All the instruments and source data is available online. This is detailed in the “Data availability section”: OSF: Roadmap for HTA implementation in Moldova [dataset] OSF: DOI  10.17605/OSF.IO/NPKBF

 

  • Are the conclusions drawn adequately supported by the results?

Yes

Associated Data

    This section collects any data citations, data availability statements, or supplementary materials included in this article.

    Data Citations

    1. Surgey G, Oortwijn W, Curocichin G, et al. : Roadmap for HTA implementation in Moldova.[dataset].2024 [cited 2024 Mar 21]. Reference Source [DOI] [PMC free article] [PubMed]

    Data Availability Statement

    OSF: Roadmap for HTA implementation in Moldova [dataset] OSF: DOI 10.17605/OSF.IO/NPKBF

    File ‘Moldova HTA Roadmap data’ contains individual underlying data for each 3 surveys

    File ‘1. Instrument - Situational Analysis.docx’ contains situational analysis instrument

    File ‘2. Instrument - Stakeholder Analysis.docx’ contains stakeholder analysis instrument

    File ‘3. Instrument - Capacity Skills Assessment.docx’ contains capacity skills assessment instrument

    File ‘4. Review of early and recent adopter countries against EDP steps.docx’ contains individual country level results

    File ‘5. Sources - Review of HTA systems

    All data underlying the results are available as part of the article and no additional source data are required.

    Data are available under the terms of the CC0 licence CC0 1.0 Universal


    Articles from F1000Research are provided here courtesy of F1000 Research Ltd

    RESOURCES