Defining Excellence: The First Core Set of Quality Indicators for the European Paediatric Surgical Audit on Hirschsprung's Disease Care

ABSTRACT

Aim

This study aimed to develop a universally applicable core set of quality indicators for Hirschsprung's disease care through a consensus‐driven process, to standardise and improve care quality across Europe.

Methods

A modified Delphi method was used to achieve consensus among healthcare professionals (HPs) and patient representatives (PRs) across Europe. Participants completed three rounds of anonymous surveys, rating quality indicators for Hirschsprung's disease care. A systematic literature review informed the initial item list. Results were analysed using predefined criteria, and a final consensus meeting established the core set of indicators.

Results

An international panel of 8 PRs and 96 multidisciplinary health care professionals representing 59 European hospitals completed all questionnaires, eventually including 12 baseline characteristics and 39 indicators. Six of the top 10 indicators were commonly prioritised by both groups. The remaining items were refined through debate and finalised during a consensus meeting.

Conclusion

This study established a core set of 12 baseline characteristics and 14 quality indicators for evaluating Hirschsprung's disease care. These indicators will support benchmarking and continuous quality improvement within the European Paediatric Surgical Audit framework, ultimately enhancing outcomes and care for children with Hirschsprung's disease.

Summary.

• A universally accepted set of quality indicators for Hirschsprung's disease care was required to address inconsistencies in care and improve standardisation across European healthcare systems.

• This study developed a consensus‐driven core set of 12 baseline characteristics and 14 quality indicators with input from healthcare professionals and patient representatives.

• The indicators aim to support benchmarking, continuous quality improvement, and research to improve outcomes in Hirschsprung's disease care.

Abbreviations

ePAG

European Patient Advocacy Group

EPSA

European Paediatric Surgical Audit

ERN

European Reference Network

ERNICA

European Reference Network for Inherited and Congenital Anomalies

eUROGEN

European Reference Network on Rare Urogenital Diseases and Complex Conditions

HCP

healthcare provider

HP

healthcare professional

HSCR

Hirschsprung's disease

IQR

interquartile range

NETS

next stage in evidence‐based paediatric surgery treatment strategies

PR

patient representative

1. Introduction

Hirschsprung's disease (HSCR) is a rare congenital disorder characterised by complete absence of ganglia in the enteric nervous system of the rectum and/or sigmoid colon, with variable involvement of the rest of the colon and small bowel. HSCR affects 1 in 5000 living newborns and is three to four times more common in males than in females [1, 2]. This condition typically presents with symptoms of functional intestinal obstruction, such as failure to pass meconium within the first 24 h of life, vomiting, abdominal distension and failure to thrive. Early diagnosis is most common due to these symptoms, though in some cases, patients may only present at a later stage with severe constipation. As outlined in recent guidelines, the diagnosis primarily depends on rectal biopsies confirming an absence of ganglion cells and hypertrophied extrinsic nerve fibres [3, 4]. Treatment strategies focus on restoring bowel function, usually by removing the aganglionic segment of the bowel. Various surgical techniques, such as Swenson [5], Duhamel [6] and Soave‐Boley [7, 8] have been employed and can be performed in single‐ or multi‐stage fashion with open, laparoscopic, or totally transanal approach [9]. Despite surgical excision of the affected bowel segment being the cornerstone of treatment, nearly half of the patients may continue to experience symptoms such as constipation or faecal incontinence following the procedure. This emphasises the critical need for development of novel therapeutic strategies and underscores the significance of holistic care management by multidisciplinary teams in expertise centers, particularly in this rare congenital malformation associated with considerable variation in clinical practice [10, 11].

Clinical auditing emerges as a crucial strategy in ensuring high quality of care through the identification, monitoring, and evaluation of variations in clinical practice and outcomes [12]. The European Pediatric Surgical Audit (EPSA) represents such an audit [13]. Initiated in 2014 as a Dutch, nationwide, prospective quality assurance system, it includes seven congenital anomalies: oesophageal atresia, congenital diaphragmatic hernia, HSCR, anorectal malformation, gastroschisis, omphalocele and malrotation. By 2020, the audit broadened its horizons internationally with the support of the European Reference Network for Inherited and Congenital Anomalies (ERNICA), a venture by the European Commission to enhance the understanding and management of rare diseases. As all 24 European Reference Networks (ERNs) established their own registries, ERNICA selected the EPSA due to its capacity to benchmark quality indicators and provide a feedback infrastructure aimed at enhancing the quality of care at the Health Care Provider (HCP) level [13]. Presently, it integrates 30 hospitals across 17 European countries, permitting registration of patients born with abovementioned conditions.

Quality indicators are critical tools used to measure and improve the quality of care and provide benchmarked information. They are categorised into structure, process, and outcome indicators, each assessing different aspects of the care pathway [13]. An optimal core set of quality indicators incorporates both process and outcome indicators, reflecting the consensus of all stakeholders, including patient groups and healthcare professionals (HPs), on what constitutes good quality care [13]. While numerous quality indicator sets have been designed for more prevalent conditions, such as hip fractures [14] or oesophageal cancer [15], the rarity of conditions like HSCR has led to a lack in evidence‐based quality indicators tailored for these unique disease‐specific challenges. Initially, to bridge this gap, quality indicators for all conditions under the EPSA, including HSCR, were developed through an expert consensus meeting among Dutch paediatric surgeons during the audit's inception in 2014 [16]. These indicators, reflective of Dutch care standards, may not be entirely relevant or applicable across other nations involved in the audit and are possibly outdated, as they focus primarily on the perspectives of paediatric surgeons, without incorporating input from patients, specialised nurses, paediatricians or paediatric gastroenterologists. This implies that the required methodology was not adopted at the start of the EPSA initiative; therefore, international approval from paediatric surgeons, all other specialists involved in the care of infants and children with HSCR and patient representatives (PRs) is paramount.

This study aims to create a universally applicable and supported comprehensive set of quality indicators (structure‐, process‐ and outcome indicators) [17] for HSCR care. The aim is to facilitate monitoring, evaluation, and comparison of the quality of care for HSCR patients in Europe. The HSCR Core Quality Indicator Set will include patient characteristics and treatment‐ and care process characteristics to aid in the interpretation and adjustment of indicator results. The future implementation of this quality indicator set within a clinical audit framework, such as the EPSA/ERNICA registry, aspires to deepen the understanding of HSCR care and, consequently, improve the quality of care across Europe for every child with this condition.

2. Methods

2.1. The Modified Delphi Study Design (See Study Flowchart in Figure 1)

FIGURE 1.

Study flow chart.

We employed a modified Delphi method, which is a consensus method designed to achieve the most reliable agreement among experts through a system of anonymous voting. This approach intends to reduce the influence of any individual participant's opinion [18, 19, 20]. Based on the idea that the opinions of experts tend to align when exposed to the perspectives of their colleagues, the process involves several rounds of survey dissemination to predetermined experts. Following each round, the aggregate of all (anonymous) responses is shared with the participants, providing an opportunity to revise their initial responses in the light of collective feedback. If consensus has not been reached after three rounds, a final consensus meeting is held to address and resolve any discrepancies and disagreements, ending with a decisive vote on the to‐be‐determined studied parameters [19, 21].

2.2. Generating Item List

A thorough literature search for existing quality indicators in HSCR care was performed through Medline, Embase and the Cochrane Library with the help of an experienced clinical librarian. As the search revealed no relevant results aside from one indicator (‘Percentage of patients with Hirschsprung disease receiving a temporary enterostomy before or during the removal of the aganglionic segment’), proposed by Dutch paediatric surgeons at the start of the audit in 2014, we performed a systematic review of literature [16]. This review focused on the literature related to the primary care process for HSCR from 2015 to 2021, collecting data on patient characteristics, treatment and care processes, as well as outcomes [22]. Parameters that were discussed in more than 5% of all the reviewed articles were added to the preliminary long list of items. Our team then converted these care process characteristics and outcomes into quality indicators, categorising them into structure‐, process‐ and outcome indicators as per Donabedian's model [17]. The conversion involved describing underlying concepts and definitions, outlining a numerator and denominator for each indicator, and setting a specific time for measurement. In addition, we translated the recommendations from the NETS^1HD study [9], which resulted in a core outcome set, and the ERNICA consensus statements on HSCR [4], into quality indicators. These were then added to the list of quality indicators.

2.3. Panel Selection

To establish a broad representation of all involved stakeholder groups, the panel for this study included both HPs experienced in HSCR care (1) and PRs involved in HSCR patient organisations across Europe (2). Aiming for a comprehensive European perspective, we aimed at enrolling participants from at least five different European nations to enhance the international relevance and acceptance of the findings [23]. Additionally, a steering committee of HSCR experts, including HPs of various specialties and PRs from different countries, was formed by mutual agreement among the lead researchers of this study to validate definitions, timing of measurements, and, if necessary, narrow down the long list of characteristics and indicators for the Delphi process. This step was crucial to ensure the study's timely execution and reduce participant dropout.

Representatives from HSCR expert centers within ERNICA and eUROGEN were invited via email, which included a brief study overview and a pre‐registration link, available for 8 weeks. To ensure comprehensive involvement, the invitation was extended to all key stakeholders in HSCR care, such as paediatric surgeons, paediatric gastroenterologists, paediatric urologists, neonatologists, specialised nurses and PRs. The study was also promoted at European paediatric surgery conferences (e.g., EUPSA congress) and EPSA, ERNICA, and eUROGEN meetings to engage participants outside ERN networks, targeting HCPs treating at least five new HSCR patients annually. Participants were encouraged to share the invitation with colleagues specialising in HSCR care. While no minimum experience was required, most healthcare providers were affiliated with ERNICA and eUROGEN centers, recognised by national healthcare systems and the European Commission. Participation was limited to four HPs per institution to account for local practices. PRs linked with ERNICA through the European Patient Advocacy Group (ePAG) were similarly approached, with the requirement that they had direct involvement in at least two HSCR cases as patients, caregivers, or advocates. An official Delphi participation invitation, providing detailed study information, was sent to those who pre‐registered.

2.4. The Round ‘Zero’, Three‐Round Delphi Questionnaires and Consensus Meeting

Before the formal beginning of the Delphi process, a preliminary round ‘zero’ was conducted, during which the steering committee evaluated all patient, treatment, and care process characteristics and indicators to refine the initial long list into a concise set. In the first round, quality indicators were categorised into structure (regarding structure of the health system), process (regarding care processes) and outcome (regarding outcomes of provided care) [17]. These three categories of quality indicators represent three interacting, but substantially different aspects of quality of care [23]. Detailed definitions, numerator, denominator, timing for measurement, and the source of each quality indicator were documented in corresponding ‘help texts’. Participants were asked to rate on a Likert scale from 1 to 9 (where 1 = ‘Totally disagree’ and 9 = ‘Totally agree’) whether each indicator should be used to evaluate HSCR care and had the possibility to share insights on each indicator's significance through a comment box. The same scale was applied to evaluate previously identified patient baseline and treatment characteristics for their relevance for case‐mix correction. Solely during the first round, participants were given the opportunity to suggest new quality indicators and characteristics that they felt were missing, which would be considered for inclusion in the next round.

For an indicator or characteristic to be officially selected as ‘consensus in’ item, it required a median score of ≥ 8, with ≥ 75% of the ratings being in the highest tertile. Conversely, an indicator was excluded (‘consensus out’) if its median score was ≤ 2, with ≥ 75% of the ratings being in the lowest tertile. Indicators reaching these thresholds were not presented during subsequent rounds. If, after three rounds, consensus was not achieved, a final meeting would be organised to consolidate the perspectives of all stakeholders, finalise the core quality indicator set, and agree on specific definitions and times of measurement.

In the second and third rounds, participants who completed the previous round were invited to reassess the items, supported by a summary of the previous round's results. This analysis highlighted their personal vote, the voting results from their stakeholder group and the other stakeholder group, including the median score, score distribution and anonymized comments from other participants. During these rounds, participants were also asked to narrow down their preferences by prioritising the previously selected indicators, choosing the five to ten they believed were most critical for integration into the clinical audit. The presentation order of these indicators was based on the prioritisation results from the previous round. To maintain the integrity of this prioritisation process, responses that selected more than 15 indicators were excluded from the analysis to prevent an overestimation of item importance.

2.5. Data Collection, Management and Confidentiality

Data was collected through Welphi, an online Delphi questionnaire management software [24]. This platform efficiently distributed the questionnaires via email and automatically processed the scores submitted by participants, providing them with the summary on their own scores and those of their stakeholder group. The scores from the other stakeholder group were manually computed separately and incorporated into the system before the start of the next round. Each round was given a minimum duration of 4 weeks to maximise participation. To ensure a higher response rate, participants received reminders via email to submit their responses if they hadn't done so within 2 weeks, with a personalised email serving as the final reminder. We aimed at securing a response rate exceeding 70% across all three surveys [25]. Electronic informed consent was obtained from all participants at the beginning of the first survey round.

2.6. Data Analysis and Publication

Questionnaires that were not fully completed were excluded from the analysis; however, the input from these respondents from previous rounds was kept. We visually assessed the impact of dropout rates on the overall scoring and conducted a comparative analysis of median scores by round and by item, looking at those who finished and those who did not complete the specific Delphi round(s). The processing of data was carried out in Excel, while the analysis was performed using R Studio [26]. The presentation of our results adhered to the guidelines set in Boulkedid's systematic review [18].

3. Results

3.1. Item List

The previously described comprehensive approach resulted in an extensive long list comprising 37 patient characteristics and 114 quality indicators: 2 structure indicators, 44 process indicators and 68 outcome indicators. To simplify readability of the questionnaire, this inventory was organised by clinical themes, such as primary surgery, management of postoperative complications and follow‐up care. In the preceding Delphi round ‘zero’, approximately half of the indicators were selected for further evaluation in the subsequent three Delphi rounds. The complete inventory of items, along with their respective sources, is provided in the Supporting Information. This includes the excluded process and outcome indicators (respectively S1 and S2), followed by the results of the three Delphi rounds concerning baseline characteristics (S3), structure indicators (S4), process indicators (S5) and outcome indicators (S6).

3.2. Participants

A total of one hundred and twenty‐nine HPs pre‐registered for the study and were subsequently invited to take part in the questionnaire rounds. Of these, 96 HPs from 59 hospitals across 17 European countries completed all three rounds of questionnaires. The median number of years of experience within the HSCR field of this final group was 15 years, with an interquartile range (IQR) of 9– 23 years. On the PRs' side, 14 individuals signed up for participation, with 8 from 5 different countries finishing all three questionnaires. The median number of years of experience in HSCR care for these PRs was 14 years, with an IQR from 7 to 18 years. Descriptions of the panels for both stakeholder groups are detailed in Tables 1 and 2.

TABLE 1.

Panel description: Health care professionals who completed all three rounds.

		Number of health care professionals (n = 96)
Years of experience (median, IQR)		15 (9, 23)
Profession	Paediatric surgeon	88 (92%)
	Nurse practitioner/nurse specialist	3 (3%)
	Paediatric gastroenterologist	2 (2%)
	Paediatric urologist	1 (1%)
	Neonatologist	1 (1%)
	Researcher	1 (1%)
Country	Austria	5 (5%)
	Belgium	9 (9%)
	Bulgaria	1 (1%)
	Czech Republic	1 (1%)
	Denmark	6 (6%)
	Finland	2 (2%)
	France	6 (6%)
	Germany	9 (9%)
	Italy	22 (22%)
	Lithuania	2 (2%)
	Norway	1 (1%)
	Poland	2 (2%)
	Spain	5 (5%)
	The Netherlands	13 (13%)
	Sweden	6 (6%)
	Turkey	3 (3%)
	UK	3 (3%)

Note: Due to rounding errors, some categories may not add up to 100%.

TABLE 2.

Panel description: Patient representatives who completed all three rounds.

		Number of patient representatives (n = 8)
Years of experience (median, IQR)		14 (7, 18)
Attached to a patient organisation		7 (88%)
Country	Austria	1 (13%)
	Denmark	1 (13%)
	France	2 (25%)
	Germany	3 (38%)
	The Netherlands	1 (13%)

Note: Due to rounding errors, some categories may not add up to 100%.

3.3. The Three‐Round Delphi Questionnaires

From September 2023 to January 2024, three questionnaire rounds were conducted among preregistered HPs and PRs. The questionnaire was accessible in three rounds for, respectively five, five and seven weeks. The length of these periods was flexible based on the response rate and the number of additional comments following reminders. After the first round, 30 new indicators (comprising 1 structure, 12 process and 17 outcome indicators) and one new patient characteristic were suggested by participants, all considered unique and thus integrated into later rounds. According to established selection criteria, no items were voted ‘consensus out’ across the three rounds. The scores, categorised by round and stakeholder group, can be found in the Supporting Information (S3–S6). In total, 39 indicators (S7) along with 12 patient, treatment and care process characteristics (Table 3) reached a ‘consensus in’ agreement from both stakeholder groups. PRs exclusively selected 13 indicators, while HPs exclusively selected one characteristic and six indicators. An outline of the study's methodology, including response rates, in‐ and exclusion criteria for item selection, and the addition of new items, is depicted in Figure 1.

TABLE 3.

Characteristics included in the final core indicator set.

Characteristic	Definition	Percentage of papers (2015–2021) reporting the characteristic
Gender	Male/Female/Undetermined/Fetus (unknown)	88%
Family history of HSCR	Other family member (siblings, parents or offspring) born with HSCR	14%
Age at diagnosis	In days	17%
Age at surgery	In days	66%
Extent of aganglionosis	Histological level of transitional zone (rectosigmoid/long segment/total colonic aganglionosis)	51%
Surgical approach	Laparotomy (open surgery) or laparoscopy (minimally invasive surgery) or transanal procedure	20%
Type of definitive repair	Transanal endorectal pull through (Soave [7] like) or transanal endorectal pull through (Swenson [5] like) or Duhamel [6] or none, definitive enterostomy or other	57%
Any other (associated) congenital anomaly	Is there (at least one) another associated anomaly, not including HSCR	29%
Intestinal malformation	Is an additional intestinal malformation (not including Hirschsprung's disease) present? (e.g., anorectal malformation, duodenal atresia, malrotation	10%
Chromosomal/genetic abnormalities	Is there a chromosomal or genetic anomaly present (e.g., RET mutation, MEN2A syndrome, Haddad syndrome, Shah‐Waardenburg syndrome, Trisomy 18, Trisomy 21, Mowat‐Wilson)	40%
Age at presenting symptoms	In days	8%
Presenting symptoms	Abdominal distension, delay in evacuation/passage of meconium, vomiting, enterocolitis, intestinal perforation, poor feeding/malnutrition/failure to thrive	28%

3.3.1. Subgroup Analysis of Health Care Professionals

Over 90% (n = 88) of the participating HPs were surgeons, while the number of participants from other specialties ranged from 1 to 3, depending on the specific profession (Table 1). When comparing individual item scores between these subgroups, some significant differences emerged with non‐surgeons generally assigning lower median scores compared to surgeons. For example, surgeons gave significantly higher scores than non‐surgeons for indicators such as (1) Percentage of patients with confirmed residual aganglionosis after definitive surgical repair (median non‐surgeons 7, median surgeons 9, p = 0.0024), (2) Percentage of patients who died within 30 days post‐surgery (median non‐surgeons 7, median surgeons 9, p = 0.0133), and (3) Percentage of patients requiring reoperation due to cuff issues (median non‐surgeons 5.5, median surgeons 8, p = 0.0157). An overview of the complete subgroup analysis results is provided in the Supporting Information (S8).

3.3.2. Attrition Analysis

No significant difference was observed in scoring distribution when comparing the scores of participants who participated only in the first round to those who engaged in the first two rounds. This pattern persisted across comparisons between participants who completed just the first round and those who completed all three rounds, as well as between participants who completed the first two rounds versus those who completed all three rounds (S9).

3.4. Indicator Prioritisation, Consensus Meeting and Finalising the Core Indicator Set

A total of 38 HPs (40%) and 5 PRs (63%) correctly prioritised up to 15 indicators in the final Delphi round, and their results were exclusively considered for the analysis. Six out of the top 10 indicators overlapped between the two stakeholder groups, although ranked in a different order. Four indicators per group were unique to that group and did not overlap with the other. Detailed rankings for both stakeholder groups from rounds two and three are listed in the Supporting Information (S10 and S11).

A virtual consensus meeting was organised for all participants who completed the questionnaire series, achieving a 70% (n = 73) participation rate through both panel groups, including 38% (n = 3) of PRs and 73% (n = 70) of HPs. During this session the selected characteristics and the top 10 prioritised quality indicators per each group were presented. The universally chosen six indicators were automatically included in the final core set, with their definitions and times of measurement refined at the consensus meeting.

Discussion then focused on the four indicators chosen exclusively by each stakeholder group, debating their inclusion, definitions, and measurement times. After all participants had the chance to comment, the initially non‐selecting group was asked if there were any objections to including the indicator. Ultimately, six discussed indicators received unanimous support for inclusion in the final core indicator set. Two indicators (Percentage of patients that underwent surgical treatment for Hirschsprung's disease who subsequently experienced soiling; Percentage of patients that underwent definitive surgical repair for Hirschsprung's disease in which the anal canal was preserved) initially selected by the PRs were challenged by the HP group, with the main reasoning being absence of evidence‐based definition and a difficult objective measurement of both indicators. It was then decided to mention the two indicators in the final set but advise against their use in practice until precise definitions and practical measurement guidance have been set in empirical manner. The comprehensive list of the finalised set, including both the unanimously selected and the contested indicators, is detailed in Table 4.

TABLE 4.

Quality indicators included in the core quality indicator set.

Health care professionals prioritising indicator	Patient representatives prioritising indicator	Indicator	Proposed definition and timing of measurement
50%	63%	Percentage of HSCR patients that underwent surgical treatment, that was treated by a multidisciplinary team	1. Type of indicator: process 2. Definition: A multidisciplinary team should at least include the following: two paediatric surgeons, paediatric gastroenterology, pathological, radiological and anaesthetic expertise, paediatric and neonatal intensive care, specialised nursing and psychosocial support at all times 3. Timing of measurement: 2 years after surgery 4. Identified in ERNICA guidelines [4]
87%	50%	Number of HSCR patients that underwent definitive surgical repair	1. Type of indicator: structure 2. Definition: How many patients have received any type of definitive surgery aimed at bowel continuity repair? 3. Timing of measurement: every year 4. Identified in ERNICA guidelines [4]
82%	38%	Percentage of HSCR patients that underwent definitive surgical repair, in which a reoperation was required because of a HSCR related complication	1. Type of indicator: outcome 2. Definition: Was any type of reoperation performed related to Hirschsprung's or complications related to Hirschsprung's and its primary surgical treatment? 3. Timing of measurement: 1 year and 5 years 4. Identified as a studied parameter in 43% of published papers between 2015 and 2021 and NETS1HD core outcome set [9]
68%	38%	Percentage of HSCR patients that underwent definitive surgical repair who experienced faecal incontinence after the age of 4 years	1. Type of indicator: outcome 2. Definition: According to Rome IV criteria (‘Recurrent uncontrolled passage of faecal material in an individual with a developmental age of at least 4 years’) [27] 3. Timing of measurement: 5 years of age 4. Identified as a studied parameter in 54% of published papers between 2015 and 2021 and NETS1HD core outcome set [9]
55%	38%	Percentage of patients with a confirmed diagnosis of HSCR receiving follow‐up at least every 6 or 12 months after the first year of life	1. Type of indicator: process 2. Definition: Follow‐up is defined as any contact offered to the patient by the expert center, including but not limited to clinical appointments, telephone interviews, screening questionnaires. 3. Timing of measurement: at 2 years of age, then every year of life until transition to adult care 4. Identified in ERNICA guidelines [4]
58%	38%	Percentage of HSCR patients that underwent definitive surgical repair, in which intraoperative complications occurred	1. Type of indicator: process 2. Definition: Were intraoperative complications encountered? (e.g., anesthesiology related complications, iatrogenic surgical complications, hemodynamic complications) 3. Timing of measurement: 30 days after surgery 4. Identified in ERNICA guidelines [4]
79%	13%	Percentage of HSCR patients that underwent definitive surgical repair in which residual aganglionosis was confirmed a	1. Type of indicator: outcome 2. Definition: Residual aganglionosis objectivized by histological proof, including transition zone pull‐through 3. Timing of measurement: 6 months after surgery 4. Identified as a studied parameter in 12% of published papers between 2015 and 2021
66%	13%	Percentage of HSCR patients that underwent definitive surgical repair, in which intraoperative biopsies were performed a	1. Type of indicator: process 2. Definition: Biopsies performed during continuity repair or at time of creating a stoma 3. Timing of measurement: 30 days after surgery 4. Identified as a studied parameter in 9% of published papers between 2015 and 2021
47%	13%	Percentage of patients with a confirmed diagnosis of HSCR receiving follow‐up regularly and based on the needs of the patient during the first year of life a	1. Type of indicator: process 2. Definition: Follow‐up is defined as any contact offered to the patient by the expert center, including but not limited to clinical appointments, telephone interviews, screening questionnaires. 3. Timing of measurement: 1 year of life 4. Identified in ERNICA guidelines [4]
42%	13%	Percentage of HSCR patients that underwent surgical treatment in which postoperative complications occurred within 30 days a	1. Type of indicator: outcome 2. Definition: Postoperative complications such as wound infections, anastomotic complications, anal excoriations, postoperative HAEC, according to the Clavien‐Madadi classification [28] 3. Timing of measurement: 30 days after surgery 4. Identified as a studied parameter in 13% of published papers between 2015 and 2021
11%	50%	Percentage of patients with a confirmed diagnosis of HSCR who received information about the availability of patient support organisations and current guidelines a	1. Type of indicator: process 2. Definition: Was the family informed about the availability of patient support organisations and guidelines? 3. Timing of measurement: 1 year 4. Identified in ERNICA guidelines [4]
8%	38%	Percentage of HSCR patients that underwent surgical treatment who subsequently experienced soiling a , b	1. Type of indicator: outcome 2. Definition: According to Krickenbeck [29] 3. Timing of measurement: 1 year and 5 years after surgery 4. Identified as a studied parameter in 21% of published papers between 2015 and 2021
21%	38%	Percentage of HSCR patients that underwent definitive surgical repair in which the anal canal was preserved a , b	1. Type of indicator: process 2. Definition: Was the functionality of the anal canal preserved and maintained? 3. Timing of measurement: 30 days after surgery 4. Identified in ERNICA guidelines [4]
18%	38%	Percentage of HSCR patients that underwent surgical treatment and as an adult was transitioned from paediatric care to an adult physician with expertise in HSCR a	1. Type of indicator: process 2. Definition: Adult physician such as a general practitioner, surgeon, gastroenterologist, or any informed specialist aware of the specificities of the care of adults operated for Hirschsprung's disease 3. Timing of measurement: 18 years of age 4. Identified in ERNICA guidelines [4]

^a This indicator was only prioritised (top 10) by one of both stakeholder groups but was accepted by the other stakeholder group in a final vote.

^b This indicator was recognised for its value but was not included in the final core quality indicator set.

4. Discussion

This study developed the first international core quality indicator set for HSCR care through a Delphi consensus method, incorporating the opinions of experienced health care professionals of multiple disciplines and PRs across Europe. This finalised ERNICA Hirschsprung Core Quality Indicator Set, consisting of two structure indicators, six process indicators, six outcome indicators and 12 patient‐, treatment‐ and care process characteristics, will be implemented in the ERNICA European Pediatric Surgical Audit (EPSA) [13]. This audit is a quality monitoring tool to measure and evaluate variation in HSCR care and its outcomes, enabling comparison of these findings between participating medical centers, regions, or countries. Insight gained from these variations should help identify best practices and inform improvement actions to enhance the quality of HSCR care.

In the initial Delphi round, from 20 patient and treatment characteristics and 57 quality indicators proposed, 9 characteristics and 24 indicators were voted as ‘consensus in’ by both stakeholder groups. No items were voted as ‘consensus out’ and overall median scores were high, as the lowest median score of a characteristic or indicator was ‘Neither agree nor disagree (5)’. This demonstrates universal agreement on the significance of each item, further endorsing the steering group's expertise in selecting the concise list of indicators. Despite HPs, particularly surgeons, expressing more criticism and selecting a fewer number of indicators compared to PRs, a notable convergence of opinions emerged across the study phases, leading to a robust and mutually accepted set of quality indicators. The prioritisation process aimed to identify a core set of up to 15 items. A higher number of indicators could lead to a high administrative burden for patient registration and, consequently, a less representative set of results. Therefore, focusing on a maximum of 15 indicators allowed for a more precise definition of each, enhancing the completeness and accuracy of the registered data.

This is a versatile and comprehensive set of indicators, and measures crucial outcomes like intra‐ and early postoperative complications and incontinence, aligning closely with predominant focuses in HSCR research [22]. We thus want to highlight a significant affinity between clinical quality of care and actual research interests. Furthermore, this European core set builds upon and extends past quality indicators selected by Dutch paediatric surgery experts, embracing a broader, international, consensus‐driven approach that benefits from including a geographical diverse panel, made possible by this digital Delphi methodology [24].

Clinical audits in rare conditions like HSCR are less common, but crucial for improving rare disease care through evidence and consensus rather than relying solely on high‐level evidence, which has shown to be scarce [13, 23]. The implementation of this core indicator set represents a significant step toward supporting clinical auditing as a means to enhance care quality in HSCR and could serve as a model for other rare conditions.

Our study complements the NETS1^HD study by focusing on quality indicators for the entire care pathway, rather than a core outcome set. Both approaches are complementary, with our study aiming to impact clinical practice through continuous quality improvement, while NETS1^HD focuses on standardising research outcomes [9]. Together, the quality indicators developed here and the outcomes from the NETS1^HD study will enable clinicians to benchmark their performance against other centers. These indicators will also be crucial for new implementation strategies, such as those being developed by ERNICA's implementation teams, which focus on guideline implementation [30]. Specifically, the EPSA quality indicators will play a key role in assessing current practices, identifying knowledge gaps, and developing high‐quality guidelines for HSCR care.

The collaboration of diverse European stakeholders, including PRs, is a major strength of this study, ensuring a high‐quality indicator set development process. The participation of over 100 experts from various specialties strengthens the reproducibility and robustness of the findings. However, we acknowledge several limitations. The high proportion of paediatric surgeons among respondents, the European focus, and the overrepresentation of certain countries may affect the generalizability of our findings. Additionally, the significant differences between the scores of surgeons and other HPs reflect potential biases. Surgeons may have prioritised surgical outcomes, while other professionals focused on diverse aspects of patient care. This disparity could also reflect a bias toward the surgical aspects of care, given the predominance of surgeons in the panel. This may, in part, reflect common clinical practice, as paediatric surgeons are primarily responsible for the treatment and follow‐up of HSCR patients. The differing levels of direct patient interaction and clinical experience with these patients likely influence the assessments and priorities of the involved stakeholders.

Another limitation is the exclusion of responses from participants who prioritised more than 15 indicators in Round 3, resulting in only 40% of HPs answering correctly. This exclusion leaves some uncertainty about whether the excluded responses would have aligned with those included in the final analysis. Nevertheless, the unanimous prioritisation of 12 quality indicators—supplemented by two additional indicators recognised for their value but not included in the final set—highlights key areas for future clinical audits and potential improvements in HSCR care [12, 31].

As this core indicator set moves toward implementation in the EPSA/ERNICA registry for HSCR, upcoming assessments will validate the indicators' association with clinical outcomes and guidelines development [32], aiming at identifying best practices and launching targeted improvement initiatives. This represents a significant step forward in improving HSCR care across Europe and could set a benchmark for rare conditions care globally.

5. Conclusion

This study successfully utilised the Delphi consensus method to develop a specialised core quality indicator set tailored for HSCR care, finally including two structure indicators, six process indicators, six outcome indicators and 12 patient‐, treatment‐ and care process characteristics. This comprehensive selection, supported by HPs and PRs, underlines the collective emphasis on critical aspects of HSCR care, aiming to direct research initiatives and improvement of quality of care toward these identified priorities. The integration of this core indicator set and the NETS1^HD core outcome set into the EPSA/ERNICA registry marks an important step toward enhancing the assessment and benchmarking of HSCR care quality across all Europe. Through the subsequent validation and application of these indicators, we aim not only at identifying exemplary practices but also the initiation of targeted initiatives to elevate the standard of HSCR care. This study underscores a significant step toward optimising care for individuals affected by HSCR, encouraging a future where evidence‐based improvements will continuously strengthen clinical practice of the care for patients with rare congenital anomalies.

Author Contributions

Daniel Rossi: conceptualization, writing – original draft, methodology, formal analysis, project administration, writing – review and editing, data curation, software, visualization. Anna Löf Granström: conceptualization, writing – review and editing, supervision, resources. Mikko Pakarinen: supervision, resources, writing – review and editing. Kristin Bjørnland: supervision, resources, writing – review and editing. Ivo de Blaauw: supervision, resources, writing – review and editing. Mark Ellebæk: writing – review and editing, supervision, resources. Francesco Fascetti Leon: writing – review and editing, supervision, resources. Dirk‐Jan Gloudemans: supervision, resources. Alessio Pini Prato: supervision, resources. Udo Rolle: supervision, resources. Nicole Schwarzer: supervision, resources. Merit Tabbers: supervision, resources, writing – review and editing. Alejandra Vilanova: supervision, resources. Rene Wijnen: supervision, resources, writing – review and editing, conceptualization, methodology. Cornelius E. J. Sloots: conceptualization, methodology, writing – review and editing, supervision, resources. Tomas Wester: resources, supervision, writing – review and editing, methodology, conceptualization.

Conflicts of Interest

The European Commission funded the EPSA|ERNICA Registry in the 3rd Health Program, HP‐PJ‐2019.

Supporting information

Data S1.