Classification grid and evidence matrix for evaluating digital medical devices under the European union landscape

Abstract

A uniform and harmonised taxonomy of Digital Medical Devices (DMDs) and their evidence-based evaluation are essential to ensure their integration into healthcare systems across the European Union (EU). As part of the Taskforce for Harmonised Evaluation of DMDs, a Common European Classification Grid for DMDs (CEUGrid-DMD) associated with an Evidence Matrix is developed. These tools are based on the mapping of existing frameworks, a survey of Health Technology Assessment (HTA) practices, consensus meetings and workshop. The survey was sent to 32 national representatives of HTA bodies from 18 EU countries. Ten HTA bodies from nine countries completed the survey while others could not, in the absence of the effective implementation of a DMD evaluation framework. This work results in the CEUGrid-DMD including four taxonomy categories, associated with an evidence-based matrix. Overall, this first version should help to converge scientific assessments of DMDs in the context of HTA Regulations across the EU.

Introduction

The use of Digital Medical Devices (DMDs) by patients or healthcare providers is increasing but still varies between European Union (EU) countries, and even within a single country according to its healthcare organisation and according to public and private nature of healthcare providers¹. The World Health Organization (WHO) Global Strategy on Digital Health 2020–2025² refers to the assessment and deployment of digital health technologies in its short, medium and long-term strategic objectives, including the development of benchmarking tools and assessment frameworks for digital health solutions. To date, few EU Member States have institutionalized assessment frameworks of DMDs in place. Furthermore, the specific frameworks currently implemented differ in terms of the elements that are evaluated and the methodologies used¹. This can result in inconsistencies in assessment criteria and evidence requirements among assessment bodies. For instance, Deprexis® (a digital psychotherapy complementary to conventional treatment for adult patients suffering from mild depressive episodes) and Hellobetter® (a digital therapy for patients with diabetes suffering from possible depression) are two DMDs that have been evaluated differently by France and Germany: both have been accepted and reimbursed in Germany, but they are not reimbursed in France after having been assessed. These disparities have already led to a fragmented European landscape of market access and reimbursement or financing procedures for DMDs, a trend which requires continuous input to foster the ongoing harmonization process in the EU³. Furthermore, these inconsistencies in assessment criteria and evidence requirements are also related to the lack of harmonization in classificatory systems and taxonomy regarding DMDs (e.g., each country proposing their own taxonomy and classification as the US Food Drug and Administration - FDA⁴ - and the British’s National Institute for Health and Care Excellence - NICE⁵).

A harmonized approach to the taxonomy of DMDs and their evidence-based evaluation is therefore essential to ensure that valuable technologies can be integrated across EU countries, while being adapted to each country’s healthcare system. Increasing transparency in existing assessment processes could help developers get better visibility and save time and resources in their application procedures. Collaboration amongst governments could facilitate the adaptation of assessment requirements to inform decision-making processes on the inclusion of DMDs in national healthcare pathways.

In this spirit of harmonisation, since 2022, more than 20 national representatives have gathered in a Taskforce for Harmonised Evaluation of DMDs (EvalEUDMD)⁶. In addition to the Taskforce members, an external advisory group was gathered to be involved in the working process to contribute to the final suggestions by sharing perspectives of different stakeholders and experiences from real-world examples. The advisory board included DMDs developers (from private or academic sector), national digital health associations, medical officers, market access experts and more. The Taskforce aims at proposing robust elements for an EU alignment of a DMD taxonomy as well as of evidence-based requirements for their evaluation. Specifically, the first work package of the Taskforce focuses on establishing a common taxonomy for such technologies and developing a DMD European classification grid, based on national experiences where such grids already exist, to serve as a guiding instrument for authorities and developers. This grid is then used in the second work package, which sought to define and map evidence requirements according to the type of technology and its potential impact. The outputs of the work packages are considered as an important contribution to collaborative efforts at the European level to shape the much-needed harmonization in this field. The socio-economic factors that impact the uptake of DMDs by different actors of healthcare systems, such as the design of the healthcare system and practices of prescribing doctors, are addressed in another working group of the Taskforce.

The objectives of this paper are to present a European classification grid of DMDs and to structure evidence requirements thereof. This paper explains the methodology to develop the grid and how it can be used to measure the value of innovative digital health technologies at the EU level.

Results

Definition of the scope of the grid

The selected criteria included in the scope of the grid were the following: (a) CE marking devices, (b) devices whose main function is based on a digital technology, (c) devices whose functionalities can serve a variety of purposes: prevent, diagnose, monitor, treat (therapeutics), clinical decision management (i.e., health care pathway), treatment management decision-making, and (d) device whose users can be one of the following: HealthCare Professionals (HCP), patients, caregivers, health system users. Exclusion criteria were the following: devices not intended to support medical purposes, software qualified as an accessory for a hardware and administrative software. Including only CE-marked devices permitted to clearly differentiate wellness applications from other applications having a medical purpose, an approach coherent with assessment procedures where clinical evidence is required solely for devices with a medical purpose.

These criteria also enable to propose a specific terminology for the technologies in scope and defined as DMDs. These can thus be considered as a sub-category of Digital Health Technologies (encompassing technologies that are not medical devices such as wellness applications) which have a medical purpose and are therefore medical devices. Applying all inclusion and exclusion criteria has led to the adoption by consensus of the following definition of the scope: Digital Medical Devices are health technologies falling into the definition of Medical Devices as outlined in the Regulation (EU) (2017/745)⁷ and whose main function is based on digital technologies intended to support one or more of the following medical purposes:

• Prevention, diagnosis, monitoring, treatment or alleviation of disease.

• Diagnosis, monitoring, treatment, alleviation of, or compensation for, an injury or disability.

These devices could include software, hardware, static and self-learning algorithms (e.g. those based on artificial intelligence). DMDs can be used by people or the wider health and social care system. They may include smartphone applications, standalone software, online tools for treating or diagnosing conditions, preventing ill health, or for improving system efficiencies as well as programs that can be used to analyse data from medical devices such as scanners, sensors or monitors. They do not include devices that are not intended to support medical purposes (e.g., wellness applications), software qualified as an accessory for a hardware and administrative software.

This approach ensures an EU-wide application of the grid as it remains in line with the existing national classification schemes and could be applicable regardless the maturity stage of a country in regulating DMDs. Furthermore, coherence with the Medical Device Regulation (EU) 2017/747⁷ is considered essential.

The Common European Classification Grid for Digital Medical Devices (CEUGrid-DMD)

The CEUGrid-DMD presents the following classification items: DMD category, its function (without distinction of the algorithm nature), the intended beneficiary (patient and/or HCP), context of use, either stand-alone setting or as part of a recognised integrated care pathway (Table 1). Categories for DMDs are established according to their main intended use: “A-Inform”, “B-Diagnose”, “C-Manage”, “D-Monitor” and “E-Treat”. Each category corresponds to one or more sub-categories based on the DMD functionality, the intended beneficiary and the context of use. As such, the category “A-Inform” corresponds to a DMD having prevention and health education as its main function. The Category “B-Diagnose” targets DMDs serving as diagnostic aid by providing information to HCP to take an immediate or near-term action to diagnose, screen or detect a disease or condition. This category does not include DMDs whose objective is to provide to patients a complete diagnosis, as we consider that a DMD never gives a diagnosis directly to a patient. The category C-Manage comprises two distinct DMD functionalities. The first aims to provide support to the organisation of care and enhance efficiency. Examples are triage systems or teleconsultations. The second includes DMD supporting self-management systems which build on personalized information, such as guidance using behaviour change techniques or assistance to mitigate disability and disease symptoms and to promote good health and healthy lifestyles. In these cases, no input from HCPs is required and the information provided by the DMD will not trigger an immediate or near-term action by the HCP. In the category “D-Monitor”, two functional sub-categories are identified. The first includes DMDs enabling self-monitoring of a disability or a disease. Examples are DMDs helping individuals with a diagnosed and treated condition to manage their treatment autonomously, essentially in the context of chronic, somatic or psychiatric illnesses. Other DMDs falling into these sub-categories are those managing a non-pathological health state or DMDs that can be used for rehabilitation by the patient itself. If alerts are provided by these DMDs, they would only target the patients themselves. This is in contrast to the second sub-category of remote monitoring assisting HCP in the management of care and the monitoring of the treatment. Finally, in the category “E-Treatment”, three functional sub-categories distinguish DMDs according to whether they are “support self-treatment”, “treatment aids” or “therapeutic decision-making”. While the first sub-category enables patients to improve the management of their condition, the two others support the HCP. A lot of DMDs have a main intended use, and therefore a main category and functionality, but also one or more other functionalities. Moreover, regarding the type of algorithm embedded, it is a useful information to understand what is being assessed. However, no distinction has been made in the CEUGrid-DMD as it has no consequences on how to assess the clinical or organisational interest of a DMD.

Table 1.

Common Classification European Grid for Digital Medical Devices (CEUGrid-DMD)

Categorisation	Function (with or without autonomous of the DMD)	Intended beneficiary	Used in stand-alone setting (Yes/No)	Used in recognised integrated care pathway (Yes/No)	Examples
A - Inform	1. Prevention and health education The DMD provides information on living conditions, lifestyle and dietary /physiological information that encourage behaviour to promote good health. The DMD may also provide information on specific conditions or on any health state standard care protocols.	Patient			System offering lifestyle and dietary advice (diet, smoking cessation, sports or physical activities, skin protection, etc.).
B - Diagnose	1. Diagnostic aid The DMD provides information used by HCPs to take an immediate or near-term action to diagnose, screen or detect a disease or condition.	HCP			• Software for detection of tumours using imaging techniques. • Software associated with a chest band to detect breathing pauses aiming to diagnose sleep apnoea.
C - Manage	1. Support organisation of care The DMD is intended to improve organisation of care and efficiency.	Patient and HCP			• Triaging systems based on individual patient health data that impact care decisions or access to care. • Tool of teleconsultation.
	2. Self-management - Personalised information The DMD provides personalised information as guidance or assistance to mitigate disability and disease symptoms and to promote good health and healthy lifestyles. The guidance or assistance can be based on, for example, behaviour change techniques. The information provided by the DMD will not trigger an immediate or near-term action by HCPs and does not require any input from them.	Patient			Systems offering targeted lifestyle and dietary advice (smoking, diet, alcohol, physical exercise) based on the user's data, for the purposes of preventing/managing chronic illnesses/addictions/health states.
D - Monitor	1. Self-monitoring of a disability or a disease The DMD aims to: • Assist persons with a diagnosed and treated condition to manage their treatment autonomously, essentially in the context of chronic, somatic or psychiatric illnesses. • Manage a non-pathological health state • Support rehabilitation therapy whereby the device is used by the patients themselves The DMD enables patients to receive alerts or advice supporting them in the management of their condition. The alert/advice features are managed by the patients themselves.	Patient			• Continuous interstitial glucose monitoring system coupled with an insulin pump or not. • App proposing physiotherapy exercises.
	2. Remote monitoring The DMD monitors parameters of a patient in a home-setting and can send alerts to HCPs enabling them to: • Interpret the data and assess the patient condition remotely; • Adapt and optimise patient care and monitoring of the treatment.	Patient/HCP			• App for tracking the mood of patients with depression sending an alert to the healthcare professional if any issue is detected. • Upper arm blood pressure monitor connected to a telemonitoring platform and coordination of care.
E - Treat	1. Self-treatment The DMD enables the patient: • to adapt its treatment according to the medical prescription and • to achieve the care objectives defined with the healthcare professional; • to optimise treatment compliance by the patient; to send alerts or provide advice facilitating disease management. These alerts/advice features are managed by the patients themselves.	Patient			• External neurostimulator for managing pain and epileptic seizures, etc. • Gamification solution applied to the treatment of psychiatric illnesses.
	2. Treatment aid The DMD enables treatment, determination of parameters for implementation or guidance of the medical decision. The technology is used during care provision or beforehand to optimise treatment implementation by the HCP	HCP			• App calculating insulin dose based on blood glucose data. • Dosage adaptation software based on kidney function.
	3. Therapeutic decision-making aid The DMD offers therapeutic decision-making support to the HCP by for example: • Suggesting one or more therapeutical options, based on the patient's data, relative to a diagnosed condition • Providing information or alerts on for example medicinal product interactions, contraindications or pharmacovigilance-related issues. The users are those who prescribe and dispense the treatment.	HCP			• Prescription aid software. • Dispensing aid software. • Drug interactions calculation software

The two columns (“Used in stand-alone setting” and “Used in recognised integrated care pathway”) are left blank as they can each take the values “Yes” or “No” for each DMD category and function, depending on the DMD being evaluated. The term “autonomy” means that the DMD has an AI component that makes it autonomous.

State of the art: responses to the survey by Health Technology Assessment (HTA) bodies and evidence generation workshop

In July 2023, a survey was sent, with the help of the remaining European network for HTA “EUnetHTA”, to 32 national representatives of HTA bodies (18 countries). From July 2023 to April 2025, only Belgium, Germany and France have made DMDs reimbursable. Other countries such as Italy, Finland or Austria are in the process of developing their HTA regulations. Ten HTA bodies in nine European countries answered the survey (Table 2). Some HTA bodies have declined to answer to the survey as their DMD evaluation process was not yet defined. HTA bodies that could respond provided concrete information on the context of assessments (the scope of the assessment, the existence of a specific framework for the assessment of DMDs, their assessment domains) and on the evidence requirements. Most countries that responded to the survey applied traditional Medical Devices HTA frameworks. Only few countries had a specific framework for DMDs in place i.e., Belgium⁸, Germany⁹ and France^10,11. The latter, however, did not target all DMDs, but regarded only one or two types (e.g., telemonitoring solutions and therapeutic applications). The scope of these specific frameworks was therefore limited. The answers also showed that the assessment domains were not similar between European countries. The only domain evaluated in all countries was the clinical domain. Economic aspects were explored by 8/10 HTA bodies; organisational issues by 7/10; technical and social domains by 5/10; ethics by 4/10; and environmental issues by 2/10 (Table 2). Most of the responses received did not include an assessment of areas not covered by the EUnetHTA core model, specific to digital aspects such as: technical stability, interoperability, ease of use and accessibility, security and data protection. Several countries have taken these aspects into account, either directly in the HTA assessment or as a prerequisite.

Table 2.

Evidence characteristics and requirement analysing the survey responses from 9 EU countries HTAs and one non-EU

	Germany	Germany	France	Austria	Denmark	Portugal	Finland	Norway	Sweden	Belgium	Iceland
Name of the organisation	BfArM	IQWiG	HAS	AIHTA	DHTC	INFARME	FinCCHTA	NOMA	TLV	KCE	-
Dedicated organization(s) for HTA	Yes	Yes	No	Yes	NA	No	Yes	No	Yes	No	No
Assessment linked to reimbursement	Yes	Yes	Yes	Yes	Other	Yes	Other	Other	Yes	Yes	NA
CE marking a pre-requirement	Yes	Yes	Yes	NA	Yes	Yes	No	Yes	Yes	Yes	No
Assessment domains
Clinical	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	NA
Technical	Yes	No	Yes	Yes	NA	Yes	Yes	No	No	NA	Yes
Economic	No	No	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	NA
Organisational	Yes	No	Yes	Yes	Yes	No	Yes	Yes	No	Yes	Yes
Social	Yes	No	No	Yes	NA	No	No	Yes	Yes	Yes	NA
Ethical	No	No	No	Yes	NA	No	No	Yes	Yes	Yes	Yes
Environmental	No	No	Yes	No	NA	No	No	Yes	No	No	NA
Type of DMD assessed	DiGA	NA	DMD with therapeutic purposes and remote monitoring	CE-marked, support to chronic diseased	No specific	No specific	NA	Medical device, organizational and medical procedures	No specific	Mobile health apps	EHR based analysis
Framework including a categorisation DMDs (e.g., based on risk or in a different way, final user, medical purpose)	Yes	Yes	No	Yes	No	No	Yes	No	No	No	No
Clinical requirements related to the type DMD	Yes	No	No	Yes	No	No	No	Yes	NA	No	No
Usage of EU Regulation (2017/745)	Yes	NA	Yes	Yes	Yes	Yes	Yes	No	NA	No	No

Some countries are not present in this table as a specific framework was not yet defined or in the process of definition for the clinical evaluation requirement for DMDs.

Regarding evidence requirements (Table 3), the six responses available were in complete agreement on the need for further clinical studies. Five out of six respondents answered questions about eligible types of studies. Some HTA bodies required randomized clinical trials (RCTs). These RCTs could be associated with different assumptions such as superiority or non-inferiority. For example, one HTA body (Germany) did not accept single arm studies while four HTA bodies accepted these. The responses did underline that for DMDs, new types of clinical study designs could be accepted, such as adaptive interventional studies or real-world evidence.

Table 3.

Clinical evidence framework and clinical study type analysing the survey responses from 9 EU countries HTAs

Table 3	Germany	Germany	France	Austria	Danmark	Portugal	Finland	Norway	Sweden	Belgium
Trigger mechanism for the 1st evaluation
• Initiated by the company	Yes	Yes	Yes	No	Yes	Yes	Yes	Yes	Yes	Yes
• Initiated by the agency	No	No	No	Yes	Yes	Yes	Yes	Yes	Yes	No
• Initiated by the country decision maker	No	Yes	No	Yes	Yes	No	Yes	Yes	NA	No
Framework including re-assessment in time	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	No	No
Post-listing studies after a first assessment	Yes	Yes	Yes	Yes	NA	Yes	Yes	NA	Yes	NA
Clinical evidence or organisational benefit
• Need of clinical study	Yes	Yes	Yes	Yes	NA	Yes	Yes	NA	NA	NA
• Interventional or/and based on RWD study	Yes	Yes	Yes	NA	NA	Yes	Both	Yes	NA	NA
Type of study design acceptable or required
• RCT mandatory?	No	Yes	No	NA	NA	Yes	No	NA	Both	NA
• Superiority mandatory?	Yes	No	No	NA	NA	No	No	NA	NA	NA
• Non-inferiority acceptable?	No	Yes	Yes	NA	NA	Yes	Yes	NA	NA	NA
• Single arm study acceptable?	Yes	No	Yes	NA	NA	Yes	Yes	NA	NA	NA
• Alternative innovative designs acceptable?	Yes	Yes	Yes	NA	NA	Yes	Yes	NA	NA	NA
Acceptable Outcomes/Impacts
• Efficacy	Yes	Yes	Yes	Yes	NA	NA	Yes	NA	NA	NA
• Performance	Yes	Yes	Yes	Yes	NA	Yes	Yes	NA	NA	NA
• Safety	Yes	Yes	Yes		NA	NA	Yes	NA	NA	NA
• Quality of Life (QoL)	Yes	Yes	Yes	Yes	NA	Yes	Yes	NA	NA	NA
• Efficiency / cost-effectiveness	NA	NA	Yes	NA	Yes	Yes	Yes	Yes	NA	NA
• Budget impact	NA	NA	Yes	NA	Yes	Yes	Yes	Yes	NA	NA
• Care system management / re-organisation	NA	NA	Yes	NA	Yes	Yes	Yes	Yes	NA	NA
• Specific requirements for AI based DMD	No	No	Yes	NA	No	No	Yes	Yes	NA	NA
• Technical information provided	No	NA	Yes	NA	Yes	NA	Yes	Yes	NA	NA

There was general consensus on the acceptance of clinical evidence data collected in other EU countries. Nevertheless, some agencies (i.e., France’s Haute Autorité de Santé – HAS - and Portugal’s Autorida de Nacional do Medicamento e Produtos de Saúde I.P. - INFARMED) required similarity of healthcare contexts. Others (i.e., Germany’s Bundesinstitut für Arzneimittel und Medizinprodukte - BfArM) emphasised their preference for national data, even though foreign clinical data could be accepted in some cases.

Description, usage and interpretation of the Evidence Matrix

The Evidence Matrix presented in Table 4 is based on the scope of DMDs defined for the European classification grid as well as input received through the survey and the workshop. The matrix is divided in the following fourmain sections: (a) “usage” – use of the DMD as stand-alone device and/or as part of a recognised integrated care pathway, (b) CE marking - risk classification (I, IIA, IIb, III), (c) “regulatory setting landscape” - DMD falling under HTA evaluation policy, (d) “DMD evidence core model”- evidence domains for assessment: CUR (Current use), TEC (Technical), TEC AI (Technical for AI based DMDs), PERF (Performance), SAF (Safety), EFF (Efficacy), ECO (Economics), ORG (Organisational), ETH (Ethics), SOC (Social), SEC, (Security) INT (Interoperability).

Table 4.

Description of DMD evidence core model associated with the Evidence Matrix

Categorisation	Function (with or without autonomous of the DMD)	Intended beneficiary	Used in stand-alone setting (Yes/No)	Used in recognised integrated care pathway (Yes/No)	MD CE risk class (I, IIa, IIb, III)	HTA evaluation needed (Yes/No)	Evidence: * using EUnetHTA classification + specific classification regarding the digital part of the DMD (Yes/No)										Data security and Interoperability
							CUR*	TEC*	TEC AI	PERF	SAF*	EFF*	ECO*	ORG*	ETH*	SOC*	SEC	INT
A - Inform	1. Prevention and health education	Patient
B - Diagnose	1. Diagnostic aid	HCP
C - Manage	1. Support organisation of care	HCP and patient
	2. Self-management - Personalised information	Patient
D - Monitor	1. Self-monitoring of a disability or a disease	Patient
	2. Remote monitoring	Patient/HCP
E - Treat	1. Self-treatment	Patient
	2. Treatment aid	HCP
	3. Therapeutic decision-making aid	HCP

The term “autonomy” means that the DMD has an AI component that makes it autonomous.

The domains are based on EUnetHTA’s HTA Core Model¹², (i.e., CUR-TEC-ORG, EFF-SAF-ECO and SOC-ETH) to which the following domains are added: TEC-AI, PERF, SEC and INT. Each of these items is detailed and described in Table 5. The HTA Core Model domains need to be interpreted and applied to the DMD context. The additional novel domains are added based on the findings in the literature, the answers to the survey and the evidence generation workshop, to ensure the framework would fit all device types¹³, their respective functionalities and capacity to respond to specificities of DMDs.

Table 5.

Evidence Matrix to be associated with the CEUGrid-DMD

Evidence: * using EUnetHTA classification + specific classification regarding the digital part of the DMD (Yes/No)	CUR*	Description of the indication and population
	TEC*	Description and Technical characteristics of the DMD
	TEC AI	If DMD includes AI algorithm; Specific information requirements regarding data, model (training, validation, testing) before and after DMD deployment, functional characteristics (robustness, performance and qualification, resilience, explainability and interpretability).
	PERF	DMD performance evaluated in experimental contexts or in real world context
	SAF*	Clinical Safety evaluation of the DMD on final user or patient
	EFF*	Clinical Efficacy evaluation of the DMD on final user or patient
	ECO*	Cost effectiveness evaluation of the DMD
	ORG*	Evaluation of individual and/or collective impact on the health organizational by the direct or indirect usage of the DMD
	ETH*	Ethical evaluation regarding the usage of the DMD regarding its use and if relevant patients' data usage under GPRD regulation. If the DMD includes AI are ethical aspect under AI act should be detailed.
	SOC*	Evaluation of the DMD literacy, acceptability, adherence and/or compliance by patients, health care professionals, health care systems or any other user.
Data security and Interoperability	SEC	Evaluation of IT security; data storage, cloud security, data usage, etc.
	INT	Evaluation of interoperability according to regulatory requirement if DMD interact with other DMDs and/or hardware, DMs, or any health care systems

It is important to highlight that these DMD domains should be read and interpreted considering not only the software itself but also how their use will be integrated into the patient care pathway and in the overall health care system.

Discussion

DMDs are already transforming our healthcare processes, organizations and healthcare systems as a whole, not only by introducing alternatives for treatment or follow-up, but also by providing new ways of developing prevention strategies. In the past two years, the Taskforce has largely contributed to the discussions and developments in this field within the EU. As such, the work on the scope, classification grid and Evidence Matrix has brought together many actors seeking to advance the integration of DMDs in the healthcare systems. It has also contributed to raising awareness regarding the importance of developing specific DMD regulatory and HTA pathways at national and European levels.

Developing evidence to accelerate the uptake of DMDs is a challenge, both for users to help them make their choices towards performant and useful technologies, and for companies to succeed in their market access strategy. The Evidence Matrix, combining DMD classification and evidence generation needs, is a first step to fostering an evidence generation plan to compensate the lack of visibility for companies regarding evidence requirements for market access in Europe. In addition, other measures could also help clarify HTA requirements applicable to companies (e.g., by developing transparency on the conclusions and principles of evaluation). No assessment principles have yet been identified to help developers anticipate the evidence required. Such principles already exist in other assessment channels in France¹⁴ and are similar to existing processes of (joint) scientific advice for the development of pharmaceuticals or medical devices¹⁵. The CEUGrid-DMD could facilitate such initiatives aimed at clarifying expectations in terms of evidence.

The CEUGrid-DMD creates a comprehensive taxonomy allowing the classification of DMDs according to their specificities. This classification facilitates the definition of the evidence requirements for assessment of the devices and offers the possibility to have a common reference on a European level against which national classifications and evidence requirements can be mapped. The Evidence Matrix can provide DMD developers, assessors and regulators with a grid of evidence specifications to be considered in the development and evaluation of a new DMD and in particular when planning clinical studies for registration and reimbursement purposes. This Evidence Matrix is therefore complementary to the CEUGrid-DMD.

In addition, the Evidence Matrix could stimulate discussions between HTA bodies in Europe to share their experience, particularly for those countries that have already started to create dedicated pathways for DMDs such as Belgium, France and Germany. The CEUGrid-DMD and the Evidence Matrix could also provide support to voluntary collaborative efforts of Member States in developing joint assessments of DMDs (e.g. within the framework of the EU Regulation on Health Technology Assessment - HTAR¹⁶).

The work has, however, some limitations. At the start of the Taskforce activities, no clear DMD definition or taxonomy existed. The analysis of the survey responses should therefore be interpreted with care. Furthermore, the disparities in DMDs assessment approaches identified in the survey, must be placed in the context of each country’s healthcare organization, financing arrangements and assessment procedures. For example, in France, the ethical interoperability and data safety of DMDs are not part of the HAS criteria (as the national HTA body), as they are already dealt with beforehand or in parallel by the national agency specialized in these issues. Some areas may also be implicitly considered without being explicitly visible (social acceptance and environmental issues, for example).

Moreover, it is important to underscore that the matrix focuses on identifying domains where evidence could be requested during the DMD evaluation process.

Despite these limitations, the input provided highlighted main guidelines in the clinical domain that EU countries need to take into account or follow in their assessment. As any DMD can be mapped into a specific category of the grid, transposition into national frameworks becomes possible and does not require specific adaptation. The grid is also foreseen as a tool for EU Member States who currently have not implemented an assessment framework in order to develop national approaches.

In conclusion, upcoming developments in the field of digital health will affect the integration of DMDs in national health systems. Deployments of Electronic Health Record (EHR) systems to which DMDs as data-producing technologies can be connected¹⁷ is an example hereof. Indeed, the availability and use of AI-enabled DMDs, remote monitoring tools, digital therapeutics and wellness applications represent new sources of health data and affect the design of healthcare systems. In view of the European Health Data Space¹⁸ (EHDS) and associated requirements for digital health infrastructures, data use, reuse and exchange, solid assessment procedures and frameworks will further contribute to facilitate decision-making with regards to the certification and referencing of DMDs connected to EHR systems. Within this context, the development of funding mechanisms for DMDs through national solidarity should be addressed. Belgium, Germany and France have been the first countries in the EU to make DMDs reimbursable by public funds^10–13. Many other countries are investigating similar approaches. In parallel, many other European and national regulations affect the development of digital health tools, including the EU Artificial Intelligence AI Act¹⁹ providing a risk classification for AI-enabled tools, and the medical devices regulation framing conditions of their market access.

These EU regulations will support the fast development and deployment of innovative digital technologies, but they also must be complemented by specific national and European HTA frameworks. These will indeed be essential to assess the scientific value of DMDs and ensure their safe and effective integration in the national healthcare systems. During the development of the HTAR¹⁶, adopted in December 2021, the issue of DMDs was not addressed since specific assessment frameworks for DMD were not yet identified as a priority in most national contexts. Although the focus of this regulation, which came into effect in January 2025, is first and foremost on subjects whose implementation is mandatory (pharmaceuticals, oncology, class III medical devices, etc.), attention is given to joint assessment of DMDs by means of voluntary collaboration based on article 23 of the HTAR. The HTAR¹⁶ could indeed offer the appropriate setting to foster EU collaboration in this field and develop a common approach to DMD HTA assessments. This approach is complementary to past or present EU-funded projects. For instance, Next Generation HTA (HTx) project has developed methods delivering more customized information on the effectiveness and cost-effectiveness of complex and personalized combinations of health technologies²⁰. At the intersection of regulatory developments lie the recently initiated EU-funded projects EDiHTA²¹ and ASSESS DHT²², both of which build upon the framework of the HTAR to establish a methodological approach for the comprehensive assessment of digital health technologies. These projects emphasize stakeholder engagement across the healthcare ecosystem to ensure relevance and applicability. While the European Health Data Space (EHDS) provides the foundational data infrastructure, EDiHTA and ASSESS DHT focus on defining specific criteria and methodologies for the evaluation of digital health technologies, leveraging real-world data and patient-reported outcomes since early 2024. The proposed CEUGrid-DMD and the Evidence Matrix have been incorporated into the early stages of these projects, facilitating their integration into the broader methodological framework.

These first tools are important building blocks in the construction of a common approach to DMD assessment in Europe. Therefore, they may contribute to fostering the integration of DMDs which have demonstrated real value for patients, HCP and the healthcare systems.

Methods

Three main steps were conducted: (1) Definition of the scope of the grid via a comparative analysis of existing national scopes in EU countries, (2) definition of the categories and subcategories of the grid via a comparative analysis of existing assessment frameworks, and (3) development of the Evidence Matrix via an overview of actual practices of EU HTA bodies followed by a workshop with experts. The methodology for the development of the CEUGrid-DMD and the associated Evidence Matrix is presented in Fig. 1.

Fig. 1. Development methodology of the Common European Classification Grid for Digital Medical Devices (CEUGrid-DMD) and the associated Evidence Matrix.

Three main steps were conducted: (1) definition of the scope of the grid via a comparative analysis of existing EU national scopes, (2) definition of the categories and subcategories of the grid via a comparative analysis of existing assessment frameworks, and (3) development of the Evidence Matrix via an overview of actual practices of EU HTA bodies followed by a workshop with experts (EU: European Union; HTA: Health Technology Assessment).

Definition of scope - mapping of EU terminology

Recent comprehensive reviews on digital health terminology have highlighted the coexistence of multiple denominations sometimes referring to the same or similar notions^23,24 (Supplementary Table 1). The scope of concepts such as eHealth, mHealth or digital health is often unclear and can refer to a broad range of digital tools such as digital health applications, digital therapeutics, software as medical device or digital clinical decision support systems²⁵. The definition of the scope used in the present analysis has resulted from a comparative analysis of existing national assessment frameworks of countries (France, Germany, United Kingdom - UK, United States of America - USA) which determined the selection criteria for technologies to be included. When this comparative analysis was carried out, the UK, the USA, France, Belgium and Germany were the only five countries to have an assessment framework for DMDs implemented and enforced by public authorities, and therefore comparable. The Belgian framework for the reimbursement of DMDs was at the time undergoing a revision and therefore it was excluded from our analysis. The selection criteria have been validated by an iterative review process. The latter mobilised experts of EU Member States where HTA procedures were in place for digital health technologies (i.e. France, Germany, Belgium, Finland) and experts of countries who were still analysing the feasibility of developing such procedures, the above-mentioned Taskforce members and an external advisory group (Fig. 1).

Definition of the categories and subcategories of DMDs

A comparative analysis of existing assessment frameworks has served to define the categories of the grid. The classification of digital solutions made by the French HAS²⁶ was mapped against the ones used by the US FDA⁴ and the UK NICE⁵, as well as German digital health applications (DiGA) criteria²⁷ (BfArM) (Supplementary Table 2). The grid has been validated by the same methodology used for the definition of the scope including iterative reviews. Several cycles of review and amendments to the grid were conducted between 2022 and 2024. Once a consensus was achieved, a working proposal of the classification grid with a common mapping exercise between the Taskforce and its external advisory group was organised. The purpose of the mapping exercise was to assess whether the initial classification grid was inclusive and comprehensive. The exercise was based on 136 marketed DMD for which information was publicly available. Members of the Taskforce and the external advisory group participated in the exercise which required to map randomly distributed DMDs within categories of the classification grid (17 groups evaluated 8 DMDs each). If the DMD did not correspond to any category in the grid, an explanation was requested. Sometimes this was due to a lack of information or comprehensibility, or categories and functionalities overlapped, or a category had to be added. Suggestions were also requested. After receiving the completed evaluation forms, a revised grid was proposed based on feedback. The mapping exercise allowed also to test the usability of the classification validity for multiple function device products²⁸ and borderline devices²⁹. For devices with multiple functions, it was considered that the same DMD could fall into several categories depending on its functionalities (i.e. approach identical to the one adopted by the HAS in the 2021 with the functional classification, according to their intended use, of digital solutions used in the context of medical and paramedical care). This approach was in line with a perspective for HTA of DMDs that will require a specific type of evidence according to a specific functionality³⁰.

Construction of the Evidence Matrix as decision-making aid for DMD development and assessment

Once the CEUGrid-DMD was defined, the next step was to link it to an Evidence Matrix (Fig. 1). This matrix has been constructed in two stages: (1) establishing an overview of existing DMD evaluation practices and guidelines among EU HTA bodies, and (2) holding an evidence generation workshop.

A survey has been implemented to provide input for the analysis of DMD assessment practices by EU HTA bodies. The survey focused on: (a) understanding the context of HTA agencies evaluation processes and their HTA frameworks for DMDs, if any; (b) contextualizing clinical evidence requirements in a national environment; (c) identifying relevant domains for a comprehensive assessment of a specific DMD and (d) collecting clinical evidence requirements for specific DMD categories based on examples provided (Supplementary Data no. 1). Countries were also asked to indicate if they did not have or were in the process of developing evaluation criteria. Regarding closed questions, respondents could answer “yes”, “no” or “not applicable” and add comments to explain their answer if needed.

Based on the responses to the survey from HTA agencies across the EU, the Taskforce has chaired an evidence generation workshop in December 2023 dedicated to the analysis of the responses and setting the basis of an evidence matrix. Members of the Taskforce were invited to this workshop as well as other Horizon Europe funded initiatives in this field as representatives of the European Digital HTA “EDiHTA”²¹ and “ASSESS DHT”²² consortia and HTA bodies. The workshop was chaired by a representative from the National Agency for Regional Health Care Italy and a representative from the French Ministry of Health.

Author contributions

M.B., A.R., L.S, C.C., and S.Z. wrote the manuscript and developed the grid and the evidence matrix. L.S.M., J.K., F.M., J.F., S.B., A.P., R.T., F.P., and B.H. edited the text, read and approved the final version of the manuscript.

Data availability

The survey form is available on the EIT Health web page dedicated to the Taskforce.

Competing interests

The authors declare no competing interests.

Supplementary information

The online version contains supplementary material available at 10.1038/s41746-025-01697-w.