Identifying essential COVID-19 indicators for primary healthcare through Delphi analysis in 31 European countries: Eurodata eDelphi study

Abstract

Background

The COVID-19 pandemic has underlined the essential role of primary healthcare (PHC) in epidemiological surveillance and public health decision-making. Across Europe, the integration of electronic health records (EHRs) and the sentinel networks have been pivotal in monitoring COVID-19. However, the lack of standardized PHC indicators for COVID-19 hinders the comparability of data among countries.

Objective

To establish a consensus on a set of standardized PHC activity indicators related to the COVID-19 pandemic for 31 countries, enhancing the capability of health authorities to make informed decisions and prepare for future health crises.

Methods

A two-round eDelphi study was conducted using a structured web-based survey, following the CREDES guidelines, to achieve consensus among a panel of 164 experts from the Eurodata study. 86 Indicators were selected based on their availability during the current pandemic, with participants rating the relevance and utility of proposed indicators.

Results

Of the 22 initial indicators, seven received consensuses for inclusion, while two remained contentious after the second round. The study found significant discrepancies in the awareness of sentinel networks and accessibility to PHC data. The consensus emphasized the necessity for indicators to be standardized, reproducible, and easily extractable from databases, with recommendations for disaggregation by age, sex, and vaccination status.

Conclusion

Key COVID-19 indicators for PHC were identified, reflecting a consensus among healthcare professionals. Further cooperation between PHC providers and national public health authorities is warranted both on the national and the international level to harmonized healthcare indicators in response to future health emergencies.

Introduction

Epidemiological surveillance serves not only to detect cases and quantify disease trends but also as a foundation for health policy decision-making to face health crisis [1]. The COVID-19 pandemic underscored the importance of current robust surveillance systems, for COVID-19 and also for influenza and other respiratory viruses. The European Centre for Disease Prevention and Control (ECDC) and the World Health Organization (WHO) have emphasized the importance of gathering data from both primary and secondary healthcare settings. This approach is pivotal for detecting and responding to increases and outbreaks of acute respiratory infections throughout Europe [2].

The majority of European countries have adopted and integrated electronic health records (EHRs) in PHC. These EHRs played a relevant role as valuable sources of data for both collection and monitoring of the COVID-19 pandemic's progression in some European countries [3], especially since the majority of COVID-19 patients received outpatient care in PHC [4]. They provided insights into the volume of acute respiratory infection (ARI) consultations [5], the healthcare worker attending patients [6], sick leave statistics [7, 8], and prescribed treatments. At the same time, the role of primary care sentinel networks has been well-established. Originally focused on detecting influenza-like illness (ILI), the definition has now expanded to include ARI, enabling the collection of data on COVID-19 and other viral infections [9, 10]. Nowadays, 29 European countries are reporting influenza data of ILI and ARI consultations. Currently, the general practitioners (GPs) participating in these sentinel networks represent 1-5% of the total number of GPs in their countries [9]. Despite their importance, support for these networks is inconsistent; some countries, like the Czech Republic, operate on voluntary participation without funding. The coexistence of sentinel networks and EHRs in Europe has marked a new era for monitoring COVID-19 in PHC [11]. Many countries have leveraged both sources to report the pandemic's progression within their communities, resulting in the creation of over 40 distinct COVID-19 PHC indicators across Europe [4]. At an European level, the ECDC is collecting data based from primary care mainly from the sentinel networks [12], public European data concerning other PHC COVID-19 indicators remains scarce. It is important to remember the lack of a common application programming interface, which limits the interoperability of EHRs, as well as the infrequent use of open data standards. The objective of this study is to establish a consensus on a set of PHC indicators that describe the workload in PHC related to diagnosing, follow-up, and responding to patients' needs during the COVID-19 pandemic across 31 European countries through a web-based Delphi study [13]. The overarching goal is to equip European health authorities with a cohesive PHC scorecard, thus enhancing their capacity to make informed policy decisions regarding organizing work in health centers, allocating resources, and assessing PHC team in a collaborative response within national health systems to future health crises across the continent.

Methods

Study design

The Delphi technique employs a multistage self-administered questionnaire with individual feedback, to ascertain consensus from a larger group of experts [14, 15]. To ensure methodological rigor, we used the CREDES guideline [16], available in Supplement S1, along with the invitation letter included in Supplement S2.

Development of eDelphi survey

We selected PHC indicators available in 31 European countries during the ongoing COVID-19 pandemic identified in a previous Eurodata project study [4]. In addition to this, a comprehensive literature review was performed to enrich the list. Following thorough discussions within the research team, a definitive list of indicators was selected for inclusion in the eDelphi survey. This survey consisted of 85 items, organized into 11 distinct sections as detailed in Supplement S5. Each eDelphi round consisted of an introductory part on the purpose of each round. To avoid confusion among the different levels of decision-making (practice, regional, national, or European level). The process of constructing the eDelphi survey is depicted in Fig. 1A. Then, the research team discussed and decided on the final list of indicators to consider for the eDelphi survey.

Figure 1.

(A) Process steps for constructing the eDelphi survey. (B) Participants invited to answer the eDelphi and countries of origin.

An initial version of the survey was shared with the Eurodata research network, aiming to achieve a consensus on the included items [17], clarify the various decision-making levels, and define each indicator precisely. Subsequent to a review and integration of all the feedback, a revised draft was prepared. To ensure its validity, a pilot survey was conducted with a selection of Eurodata researchers invited to provide feedback. Incorporating this input, the final questionnaire was then configured on the online platform. Once the final questionnaire was designed, it was shared with the participants so they could respond.

We defined consensus as at least 70% of respondents agreeing or strongly agreeing on a given item within the team. Additionally, it was required that at least 40% of this 70% rate the item as “strongly agree” to achieve consensus [18]. Non-consensus was defined as less than 70% of respondents agreed or strongly agree, and no major change was suggested for the item. Those indicators with less than 70% and more than 50% in consensus ratings were sent to the second round. Those with less than 50% in consensus ratings and without any suggestions for change by the panel after two rounds were rejected for the set of indicators.

Study setting and expert panel

We recruited 164 panelists from 31 European countries with prior involvement in or connections to the Eurodata project, allowing us to form a larger group in a cost-effective manner (Fig. 1B). Selection criteria were (i) to be a front-line family doctors involved in PHC COVID-19 pandemic, (ii) belonging to WONCA Europe Organization or EGPRN (European General Practitioner Research Network), and (iii) Public health technical officer linked to PHC in any European countries participating in the study and being proposed by a family doctor linked to the project.

Data collection and analysis

The online form was built up in the Delphi Studies Platform of Miguel Hernandez University of Elche (Spain). To collect our data, we used the Qualtrics XM Platform. A personal link was sent via email to each panelist. The researchers on the team did not have access to the responses, which were coded anonymously, preventing the extraction of stratified data that could break anonymity. This allowed following up response rates and sending reminders to specific members. Due to high workload caused by the COVID-19 pandemic, each round lasted 8 weeks. We opted for a flexible approach towards the panelists to increase the response rate of each round. Reminders were sent weekly sent to members that had not completed the survey. Data collection took place between October 2022 and March 2023. For analyzing quantitative data, we calculated descriptive statistics of every item using SPSS 27 (IBM SPSS Statistics 27). We used Microsoft Excel to list and categorize qualitative data. Panelists’ comments were anonymously and literally registered. For analyzing qualitative data, we used content analysis.

Role of research team to prevent bias

The research team made methodological decisions in line with the available literature. We predefined and stipulated methodological steps before commencing the study. We applied, monitored, and evaluated these steps during the study. The results of each round were discussed by the research team, while qualitative data were interpreted by two researchers for researcher triangulation (Supplement S1).

Results

A two-wave eDelphi consensus survey has been performed. The first round of participation included 117 respondents (71.3%) between October and December 2022, while the second round saw 101 respondents (86.3%) between January and March 2023, despite two personalized email reminders sent during each round. Mean age was 44.1 years, with 69.1% males, 62.2% were GPs and 30.9% were public health officers (Table 1). Regarding years of work experience, 61% had more than 3 years of experience. Country of origin are provided in Fig. 1B and Supplement S3.

Table 1.

Demographic information of eDelphi participants and utility of health indicators in pandemic monitoring and data disaggregation levels (eDelphi first round, n: 117)

Demographic information of eDelphi participants
Sex	n	Professional background (n)
Male	84	General Practice				76
Female	32	Public Health				38
Unknown	1	Other				8
Age groups		Unknown				5
<35 years	23	Years of expertise (n)
35–49 years	54	<5 years				28
>50 years	36	5–9 years				16
Unknown	4	10–19 years				40
		20–29 years				21
		>30 years				12
Utility of health indicators in pandemic monitoring and data disaggregation levels
Level of agreement on values relevant for a health indicator and level of disaggregation			Strongly disagree (%)	Disagree (%)	Neither agree nor disagree (%)		Agree (%)	Strongly agree (%)
Values considered relevant to create an indicator in primary care
Health indicators attempt to describe and monitor a population’s health status or condition			1.7	0.8	3.4		31.9	62.2
An indicator is a measurement that reflects health characteristics in a given population			0.8	4.2	11.8		35.3	47.9
Indicators are dynamic, reflecting a specific time-linked period			1.7	3.4	10.1		38.7	46.2
Health indicators can be used to describe disease burden in a specific population group			0.8	1.7	12.6		34.5	50.4
Health indicators can be used to forecast the risk of disease outbreaks and helping to prevent epidemic/pandemic			2.6	5.1	18.8		37.6	35.9
Indicators are used in public health to drive decision-making for the health of the community			1.7	3.5	8.7		33.0	53.0
Regular monitoring indicators can provide feedback to improve decision-making in healthcare systems			0.9	0.0	6.1		34.8	58.3
Health indicators should have a common and clear definition for all primary care providers			0.9	1.7	3.5		18.3	75.7
Health indicators should be reproducible			0.0	0.0	2.6		23.5	73.9
Health indicators should be feasible and designed to allow for easy extraction from database			0.0	1.8	5.3		23.7	69.3
Population disaggregation level of indicators
COVID-19 primary care indicators should be disaggregated by sex			0.8	9.2	16.8		16.8	56.3
COVID-19 primary care indicators should be disaggregated by age			0.0	0.8	4.2		12.6	82.4
COVID-19 primary care indicators should be disaggregated by vaccination status			0.0	2.5	6.8		19.5	71.2
COVID-19 primary care indicators should be disaggregated by group ethnic and/or migrant situation			5.9	10.9	22.7		25.2	35.3
COVID-19 primary care indicators should be disaggregated by vulnerable populations (low socioeconomic status, health illiteracy, homeless people, etc.)			0.8	5.0	10.9		22.7	60.5
Would you like to add other indicator regarding this section?			Urban/rural; Local/Regional; Comorbidities; Functional status; war; PHC system (free of charge or co-payment)

Bold type denotes ≥70% of agreement throughout participants.

In questions regarding the conceptualization of health indicators, the majority of participants concurred on the importance of indicators and the necessity for a common definition (Table 1). A majority of participants, 71.6%, were not aware of a sentinel doctor network in their countries. Additionally, 67.5% reported a lack of public PHC data availability, while 57.9% indicated access to PHC data from insurance companies. For 61% of the participants, PHC data were available at the practice level, encompassing both public and insurance company sources. However, only 40% had access to regional and national PHC data from both public and private providers.

Table 1 summarizes the results of the first and second round of Delphi survey with the accepted or rejected indicators. In the first round of the Delphi survey, four items were accepted and seven were rejected. Subsequently, 11 items advanced to the second round, where 3 were accepted, 6 rejected, and 2 remained unclear for acceptance. In terms of content, all items related to the sentinel network and suspected COVID-19 cases were accepted. However, items pertaining to nurse follow-up of patients, as well as those concerning home visits and sick leaves, were uniformly rejected.

Regarding the chosen level of data disaggregation, the majority of participants (80-96%) indicated having access to regional and national-level information. In terms of publication frequency, most respondents mentioned daily reporting during pandemic peaks and weekly reporting during other periods (96%). As for proposed new indicators, the suggestions primarily revolved around revising some from the survey. Suggestions such as gathering information during home visits, conducting follow-ups, documenting the types of symptoms observed when patients were referred to the hospital, or detailing the types of complementary tests conducted were shared. Comments and proposal of new indicators were recorded (Supplement S4 (S4-1 and S4-2)).

In terms of indicators for estimating suspicious COVID-19 cases, the majority of comments were focused on the definition of denominators and how to collect the data (Tables 1 and 2). In terms of follow-up, one noteworthy aspect was the number of COVID-19 patients who received follow-up care solely in PHC, without requiring hospitalization. The accepted and uncertain indicators are described in Table 3.

Table 2.

Round 1 and round 2 from the consensus of eDelphi items with mean scores

	Results Round: 1						Results Round: 2
	N	Mean	VC (%)	%>=4	%=5	Decision	N	Mean	VC (%)	%>=4	%=5	Decision
Indicators regarding role centinel network
Positive cases for SARS-CoV-2 (COVID-19) seen by the Sentinel networks. Definition: Numerator: Positive cases for SARS-CoV-2 (COVID-19) seen by the Sentinel networks. Denominator: Total population of a country or region	116	4.1	22.8	73.3	42.2	Accepted
Positivity rates to SARS-CoV-2 (COVID-19) among all the respiratory infections by the Sentinel networks. Definition: Numerator: Positivity rates to SARS-CoV-2 (COVID-19) among all the respiratory infections by the Sentinel networks. Denominator: Total population of a country or region	115	4.1	22.8	80.9	40.0	Accepted
Estimated incidence of COVID-19 cases per 100 000 population with respiratory signs observed in general practice through the Sentinel networks. Definition: Numerator: Number of COVID-19 cases with respiratory signs from Sentinel network. Denominator: Region or country's total population	116	4.1	24.6	76.7	39.7	Second Round	102	4.1	20.3	74.51	39.2	Accepted
Indicators regarding suspicious COVID-19 cases
Percentage of cases of COVID-19 among all respiratory infection cases in PHC. Definition: Numerator: Number of cases of COVID-19 in primary care, region or country. Denominator: Number of all respiratory infection cases in primary care, region or country	116	4.1	22.1	78.4	37.9	Second Round	100	4.2	21.5	86	44	Accepted
Total COVID-19 cases with positive test in primary care. Definition: Numerator: Number of primary care COVID-19 cases with positive test in practice, region or country. Denominator: Total primary care COVID-19 Tests performed in practice, region or country	117	4.1	24.3	75.2	40.2	Accepted
Indicators regarding primary care follow-up to COVID-19 patients
Number of COVID-19 patients who were followed-up in primary care (nurse and/or GP) for all reasons. Definition: Numerator: Number of COVID-19 patients who were followed-up in primary care in a period. Denominator: Total number of patients visited in primary care in a period	117	3.9	28.8	70.1	38.5	Second Round	100	3.8	28.6	65	35	Doubtful
Primary care follow-up ratio (nurses and/or GP): follow-up ratio of cases and contacts. Definition: Numerator: Family medicine follow-up COVID-19 cases. Denominator: Family medicine follow-up COVID-19 contacts	117	3.4	39	54.7	24.8	Second Round	101	3.2	42.1	46.53	20.8	Rejected
Indicators regarding the follow-up of primary care nurses to COVID-19 patients
Number of any contacts with nurse with COVID-19 recorded as reason for the contact. Definition: Numerator: Number of any contacts with nurse with COVID-19 recorded as reason for the contact. Denominator: Total contacts with nurse in a period	116	3.3	38.7	49.1	18.1	Rejected
Number of nurse home visits with COVID-19 recorded as reason for home care. Definition: Numerator: Number of nurse home visits with COVID-19 recorded as reason for home care. Denominator: Total nurse home visits with nurse in a period	115	3.3	38.8	47.8	20.0	Rejected
Number of nurse telephone consultations with COVID-19 recorded as the reason for consultation. Definition: Numerator: Number of nurse telephone consultations with COVID-19 recorded as the reason for consultation. Denominator: Total telephone contacts with nurse in a period	115	3.2	40.8	47	17.4	Rejected
Number of nurse control home visit with COVID-19 recorded as the reason for home visit	113	2.9	46.5	37.2	14.1	Rejected
Indicators regarding the follow-up of COVID-19 patients in primary care
Number of phone consultations to patients with COVID-19 or patients close family member (by GP). Definition: Numerator: Number of phone consultations to patients with COVID-19 or patients close family member by GP. Denominator: Total number of phone consultations to patients by GP	117	3.6	33.7	60.7	27.4	Second Round	101	3.5	33.6	52.48	20.8	Rejected
Number of email consultations to patients with COVID-19 or patients close family member (by GP). Definition: Numerator: Number of email consultations to patients with COVID-19 or patients close family member (by physician). Denominator: Total number of email consultations to patients by GP	117	3.1	42.6	41	17	Rejected
Number of face-to-face visits to GP with COVID-19 recorded as reason for the visit. Definition: Numerator: Number of face-to-face visits to GP with COVID-19 recorded as reason for the visit. Denominator: Total number of face-to-face consultations to patients by GP	117	4.0	25.2	72.6	40.2	Accepted
Number of first visits (examinations) with COVID-19 recorded as reason for the visit (by GP)	113	3.8	27.1	65.5	31.9	Second Round	101	3.9	25.2	74.26	28.7	Accepted
Number of control visits (examinations) with COVID-19 recorded as reason for the visit (by GP). Definition: Numerator: Number of first visits (examinations) with COVID-19 recorded as reason for the visit (by GP). Denominator: Total number of first visits to patients by GP	114	3.5	34.2	59.6	23.7	Second Round	97	3.5	31.5	56.7	18.6	Rejected
Number of first home visits with COVID-19 recorded as reason for home visit (by GP). Definition: Numerator: Number of first home visits with COVID-19 recorded as reason for the home visit (by GP). Denominator: Total number of first home visits to patients by GP	113	3.6	32.7	60.2	25.7	Second Round	96	3.4	33.4	53.13	20.8	Rejected
Number of follow-up home visits with COVID-19 recorded as reason for home visit (by GP). Definition: Numerator: Number of control home visits with COVID-19 recorded as reason for home visit (by physician). Denominator: Total number of control home visits to patients by GP	114	3.3	37.1	48.2	21.1	Rejected
Indicators regarding number of procedures in PHC to COVID-19 patients
Total number of procedures to patients in primary care with COVID-19 recorded as reason for procedures. Definition: Numerator: Total number of procedures to patients in primary care with COVID-19 recorded as reason for procedures. Denominator: Total number of procedures to all patients in the practice in a period	116	3.4	38.6	51.7	25.0	Second Round	100	3.4	34.7	52	17	Rejected
Number of COVID-19 patients who were examined in PHC (X-ray or/and phlebotomy). Definition: Numerator: Number of COVID-19 patients who were examined in primary care (X-ray or/and phlebotomy). Denominator: Total number of patients examined in PHC (X ray or/and phlebotomy) by GP	116	3.2	40.1	47.4	17.2	Rejected
Indicators regarding number of procedures in PHC to COVID-19 patients
Sick leaves processed by GPs of patients in COVID-19 quarantine. Definition: Numerator: Sick leaves processed by GPs of patients in COVID-19 quarantine. Denominator: Total of sick leaves by GPs in a period	116	3.6	35.6	58.6	33.6	Second round	100	3.6	35.4	60	32	Rejected
Sick leaves processed by GPs of COVID-19 patients in isolation. Definition: Numerator: Sick leaves processed by GPs of COVID-19 patients in isolation. Denominator: Total of sick leaves by GPs in a period	117	3.7	34.2	59.8	32.5	Second round	100	3.8	30.7	66	31	Doubtful

Low-medium scores (<70% of respondents agreed or strongly agree, and no major change was suggested for the item); medium-high scores (<70% and more than 50% in consensus ratings were sent to the second round; those with <50% in consensus ratings and without any suggestions for change by the panel after two rounds were rejected);  high scores (70% of respondents agreed or strongly agreed, and at least 40% agreed to rate strongly agreed for the item).

VC: variation coefficient.

Table 3.

Accepted and Uncertain Indicators based on the eDelphi Process

Indicators	Decision after the eDephi
Indicators regarding role sentinel network
Positive cases for SARS-CoV-2 (COVID-19) seen by the Sentinelles network.	Accepted in the first round
Positivity rates to SARS-CoV-2 (COVID-19) among all the respiratory infections by the Sentinelles network.	Accepted in the first round
Estimated incidence of COVID-19 cases per 100 000 population with respiratory signs observed in general practice through the Sentinelles network.	Accepted in the second round
Indicators regarding suspicious COVID-19 cases
Total COVID-19 cases with positive test in primary care.	Accepted in the first round
Percentage of cases of COVID-19 among all respiratory infection cases in PHC.	Accepted in the second round
Indicators regarding primary care follow-up to COVID-19 patients
Number of COVID-19 patients who were follow-up in primary care (nurse and/or GP) for all reasons.	Doubtful
Indicators regarding primary care follow-up to COVID-19 patients
Number of face-to-face visits to GP with COVID-19 recorded as reason for the visit.	Accepted in the first round
Number of first visits (examinations) with COVID-19 recorded as reason for the visit (by GP)	Accepted in the second round
Indicators regarding number of procedures in PHC to COVID-19 patients
Sick leaves processed by GPs of COVID-19 patients in isolation.	Doubtful

Discussion

Main findings

The results of this comprehensive eDelphi study provide a pivotal step towards the standardization of PHC indicators in the context of the COVID-19 pandemic across Europe. PHC data availability is scarce across countries participants. There is a need to define health indicators integrating PHC workload in order to tailor public health policies. Most countries adopted health measures during the COVID-19 pandemic based upon data collected from secondary care (hospital admissions being the most important measure in most countries), whereas data from PHC which took care of the majority of COVID-19 patients were not used as effectively. Seven indicators have been defined, considering COVID-19 estimated cases. Robust agreement on indicators related to case detection and management within the sentinel networks were found. These indicators are valid not only for the EHR systems but also for sentinel networks.

These proposed indicators for COVID-19 should enable us to develop robust indicators for surveillance of respiratory illness in PHC (not only during pandemics) and mainly connect those to the national public health authorities, which collect data for international surveillance. The robust agreement on indicators related to case detection and management within the sentinel networks underscores the critical role these networks play in national surveillance systems. These networks facilitate real-time data collection, which is vital for the timely response to infectious disease outbreaks [9, 10]. However, variations in the denominators used for these indicators highlight the challenges of harmonizing data across diverse European healthcare systems. Although there is strong agreement on the usefulness of sentinel networks, not all GPs were aware of the existence of the sentinel network, highlighting the need to promote the network and make the results public. Sentinel networks must be valued and given sufficient support (both financial and technical); otherwise, the quality of the collected data will not be adequate to draw valid conclusions.

The number of face-to-face visits to GPs by COVID-19 patients and the count of physical examinations were selected as key indicators. These indicators are essential for assessing the burden of disease and the capacity of PHC systems to manage patient loads. However, indicators related to the frequency of home visits, and the role of nurses were rejected. These results highlight the varying contexts in which different health systems and PHC services operate, such as differences in patient access and the need for sick leave policies. European PHC systems showed variation, especially in nursing roles, that were not involved in clinical encounters with COVID-19 patients in countries such Austria, Czech Republic, Germany, Greece, Italy, Luxemburg, Bulgaria, and Belgium [6]. However, they are in charge of long-term facilities and home visits in regular demand instead of GPs in many European countries in a daily basis [6, 19]. This finding warrants further investigation, as nursing services have been integral to the pandemic response, especially in community settings and for patients requiring long-term care [19]. Nevertheless, community settings and long-term care are not considered as part of PHC facilities services in many European countries. As PHC increasingly adopts multi-professional team structures, monitoring the activities of other professionals becomes pertinent, particularly during staffing shortages experienced at the peak of the pandemic.

Other indicators that were rejected pertained to sick leaves. The ethical dilemma of self-declared sick leave from self-testing, as France proceed, has emerged and it opens again the discussion of self-empowerment due to the fact that a self-declaration of a positive test provided an online sick leave [20]. As studies highlight a decrease in essential PHC activities during the pandemic [21–24], there is a pressing need to enhance legislation across Europe. This is crucial to encourage self-declared sick leave for mild COVID-19 cases, preventing the overburdening of PHC with low-priority tasks and promoting high-value care activities [25].

Also, the burden of administrative work provided by PHC has become no longer bearable and unwarranted with regard to uncomplicated patients in whom a sick-leave could be managed without the need for a GP consultation.

It also notes the importance of tracking hospital referred patients (percentage of COVID-19 patients treated who are admitted to the hospital). The referred activity from hospital to PHC could be defined as the percentage of COVID-19 patients treated who are discharged [COVID-19 rate (cases and/or contacts) per 10 000 assigned inhabitants].

Indicators related with complementary test have been evaluated too. The study suggests separating X-ray and blood test demands indicators and stratifying by COVID-19 severity. It might offer a proxy of the complexity of COVID-19 patients treated in PHC. Nevertheless, there is a high variability on the emergency X-ray access for GPs among European countries [6].

The inclusion of mandatory reporting of COVID-19 cases in PHC as an accepted indicator in the initial round holds significance as it marked the foundational step toward constructing a PHC scorecard related to COVID-19 activity, interconnected with public health information. The ease of collecting this indicator stemmed from certain countries' readiness to integrate their PHC Information Systems with PHC EHR and public health information systems, allowing for electronic mandatory disease reporting [26]. Including this indicator as the primary measurement to monitor the pandemic could be an excellent initial step in a comprehensive scorecard for tracking the pandemic [27, 28]. This could complement metrics like the number of COVID-19 hospitalizations or ICU admissions. It is essential to establish European legislation that mandates this value, along with national regulations incorporating additional indicators to support decision-making during ARI/ILI outbreaks.

Indicators on COVID-19 cases tracing and vaccination need to be considered. When a pandemic occurs, there is a need to evaluate the impact of contact tracing to reduce virus spread. Integrating information about COVID-19 cases is crucial to establish health strategies to deal with a health emergency [2]. On the other hand, health information integrated in a same system would help to monitor vaccinations roll out and detection of possible side effect as the vaccines are still under surveillance by the European Medicines Agency (EMA). PHC is an essential place to detect potential vaccination side effects.

The high level of consensus for disaggregating data by age and vaccination status is particularly relevant in the context of a pandemic that disproportionately affects different age groups and has seen varying rates of vaccine uptake [29]. Detailed information might help to create concrete actions to specific groups to reduce disparities and avoiding compliance of the inverse care law in PHC [30]. All the surveillance data need to be propelled back to the PHC setting and be explained how they may help in providing healthcare on a daily basis. However, during the COVID-19 pandemic many countries do not share this information [4].

Strengths and limitations

One of the main strengths lies in the participation of 31 countries, ensuring that the indicators are feasible for all participants, despite variations in the organization of PHC systems. Additionally, a high level of agreement exists for some accepted indicators, providing multiple options for monitoring the pandemic in PHC. Limitations of this study include the potential for selection bias given the recruitment of participants with prior involvement in the Eurodata project, which may not fully represent the broader European PHC context. Additionally, the response rate, although reasonable, may not capture the full spectrum of opinions among European GPs and public healthcare professionals, particularly as the survey was only in English. The exclusion of nurses and other healthcare professionals from the panel could overlook critical insights, given their significant role in PHC, especially during the pandemic. This could be a clear limitation especially as nurses’ indicators were not accepted.

While the study identifies a set of indicators, the validation of these indicators in real-world settings is not discussed. Future research should focus on testing the applicability and impact of these indicators on PHC practices and patient outcomes. Continuous evaluation and adaptation of the indicator set will be necessary to ensure they remain aligned with the current state of the pandemic and the needs of a strong PHC systems linked to public health system.

Implications for research and policy

The study's findings have the potential to significantly influence how European health authorities collect, analyze, and utilize PHC data during pandemics, improving response strategies. There is a need for subsequent studies to validate the identified indicators in various European healthcare settings. Future iterations of this study should consider including a broader range of healthcare professionals. Another option to solidify agreement could involve designing a quantitative study with a representative sample. A robust electronic reporting system is a key element. EHR systems can help collect valuable data and build indicators to measure practicés workload. Health policy should be more flexible, enabling decision-making based on the epidemiological situation of ARI/ILI outbreaks. This flexibility could include hiring additional personnel, adjusting schedules, and prioritizing pathologies during periods of high workload. Unfortunately, such adaptive measures are not routinely implemented, limiting the ability of PHC to respond effectively to sudden increases in patient demand. The network of PHC providers included in surveillance reporting should be given extra support (both financial and technical) and education, be in close contact with the responsible national public health authorities and also receive feedback. Epidemiologic data should be routinely explained to PHC providers through commented reports so that the data could be used effectively to tailor health policies with regard not only to COVID-19, but ARI/ILI in general.

This eDelphi study has highlighted the complexities and variabilities inherent in the European PHC systems during the COVID-19 pandemic. The strong agreement on the need for common definitions and reproducibility of health indicators underscores the pursuit of a unified approach to pandemic data handling across Europe and the need to coordinate ECDC surveillance with PHC systems across Europe. A major challenge is the lack of national legislation for standardized PHC data collection. While some countries use EHRs and sentinel networks, many lack legal mandates, hindering disease tracking and policy decisions. A unified national PHC information system could ensure consistency and interoperability. Governments and the EU should invest in these systems and strengthen legal and technological frameworks to enhance data-driven decision-making and reinforce the role of PHC in public health preparedness and crisis response. The identified indicators can serve as a foundation for a Europe-wide PHC surveillance dashboard that could streamline data reporting, facilitate cross-country comparisons, and aid in the allocation of resources during health emergencies. Such a dashboard would also support the ECDC and WHO in their efforts to coordinate international responses to pandemics and to enhance the relationship among public health officers and PHC systems in each European country.

Conclusions

This eDelphi study contributes to a more cohesive understanding of PHC's role in pandemic surveillance and has highlighted both the potential and the challenges of standardizing PHC activity indicators for COVID-19 across Europe. The consensus reached on the selected indicators offers a pathway towards a more unified and effective surveillance system that can significantly contribute to the management of current and future pandemics, providing a foundation for the enhancement of Europe’s preparedness to face them.

Authors’ contributions

M.P.A.P., R.G.B., M.G.C., and S.A.B. developed the research idea. M.P.A.P. and J.J.M. carried out the body of the analysis with support of R.G.B., M.G.C., S.A.B., and T.F. M.P.A.P., R.G.B., M.G.C., S.A.B., and T.F. wrote the manuscript. All other authors contributed to the discussion and comments on the manuscript.

Supplementary data

Supplementary data are available at EURPUB online.

Conflict of interest

None declared.

Funding

This publication will be funded by open access publication fund of the Martin-Luther-University Halle-Wittenberg. This study was supported by the European General Practice Research Network (EGPRN) Grant (2022/01).

Ethical consideration

The study had the approval of the Research and Ethics Committee of the Hospital Universitario de la Paz (PI-5030), Madrid, Spain.

Data availability

The data supporting this study's findings are available on request from the corresponding author. The data are not publicly available due to privacy or ethical restrictions.

Key points.

• The study identified key PHC activity indicators related to COVID-19, focusing on case detection, management, and sentinel network reporting. These indicators are intended to standardize surveillance and inform public health policy across Europe.

• Sentinel networks and electronic health records are essential for real-time data collection.

• The findings advocate for stronger support for PHC surveillance systems, including financial and technical resources, and for integrating PHC data with national public health systems to improve pandemic preparedness and management across Europe.