The Challenge of Calculating EUSOMA Quality Indicators with Administrative Data: A Pilot Study in Belgium

Abstract

Introduction

The study aimed to assess the feasibility of calculating the European Society of Breast Cancer Specialists (EUSOMA) quality indicators (QIs) using Belgian Cancer Registry data coupled to administrative health data and to provide national results.

Methods

Women diagnosed with ductal carcinoma in situ (DCIS) or invasive breast cancer (IBC) in 2014–2018 were selected from the cancer registry. Fourteen EUSOMA QIs were chosen to assess the quality of care.

Results

Overall, 46,035 patients with IBC and 3,973 patients with DCIS were included. Most QIs had to be rephrased so that they could be calculated with the available data. None of the selected QIs on systemic treatment could be calculated due to a lack of reliable receptor status information. For some Qis, there is ample room for improvement in Belgian clinical practice: cTNM stage reporting, multidisciplinary team meetings, sentinel lymph node biopsy only in IBC with clinically negative lymph nodes. The result was 1–5% lower than the target for mammography and breast ultrasound, pTNM reporting, start of treatment, single breast surgery in DCIS, and no axillary clearance in DCIS. For histological or cytological assessment, receptor status assessment, start of radiotherapy, and single breast surgery in IBC, the results were at or above the target.

Conclusion

Several EUSOMA QIs can be calculated with routinely collected data, while for several important aspects of care additional data collection is indicated so that its quality can be assessed. The validity of the obtained results depends on the reporting accuracy.

Introduction

In the European Union (EU-27) countries, breast cancer (BC) is the most frequently occurring cancer. In 2022, BC accounted among women for 29.4% of all new cancers and 16.7% of all cancer deaths [1, 2]. Increasing incidence trends in the EU-27 have been attributed to multiple factors, such as reproductive factors, increasing prevalence of obesity, and physical inactivity, but also to increased screening intensity [3]. With an age-standardised incidence of 184.7 per 100,000 women, Belgium had in 2022 the second highest incidence estimate of invasive breast cancer (IBC) among the EU-27 countries [1]. The EU-27 average was 147.6 per 100,000 women [1, 2].

To bolster up the delivery of the highest quality of care, evidence-based standards can be set through the development of guidelines and clinical pathways [4–7]. However, this is not sufficient [8]. This was illustrated in a systematic review, which highlighted that healthcare providers’ adherence to BC clinical guidelines is low in Europe [9]. The adherence for the whole care process ranged from 54 to 69%. More specifically, the adherence for breast-conserving surgery, for instance, ranged between 35 and 95%, while that for mastectomy varied between 68 and 81% [9]. Setting standards by itself does not lead to change [8]. For that purpose, standards need to be monitored (e.g., through audit and feedback, certification) and necessary actions must be taken to implement change (e.g., public reporting) [8].

Based on the recommendations adopted in clinical practice guidelines, quality indicators (QIs) can be developed to aid the monitoring of care. They can be used to document the quality of care, to support regulation, quality improvement, and patients’ choice, as well as for benchmarking. The European Society of Breast Cancer Specialists (EUSOMA) developed thirty-four QIs which cover the whole BC care continuum [10]. The EUSOMA QIs are an essential element in the voluntary European certification process of BC centres [10].

Collecting the data for the calculation of these QIs is demanding and labour intensive. Alternatively, routinely collected data may be used, an approach that does not necessitate additional efforts for data registration and transfer. Since this approach does not depend on hospitals’ willingness to participate, nationwide results can be obtained and selection bias avoided. Therefore, in this study we evaluated whether it is feasible to calculate the EUSOMA QIs by using cancer registry data coupled with administrative healthcare data, in order to assess the quality of BC care offered in Belgian hospitals.

Methods

Study Database

The main data source for this study was the nationwide Belgian Cancer Registry (BCR) database. In Belgium, cancer registration is mandatory for hospitals with an oncological care program [11] and laboratories for anatomic pathology [12]. More than 98% of all cancer cases are included in the BCR database [13]. The BCR database was coupled with administrative healthcare data, providing information on diagnostic and therapeutic procedures as well as pharmaceuticals reimbursed by the national health insurance.

Study Cohort

Data from female patients diagnosed with either ductal carcinoma in situ (DCIS) or IBC diagnosed in 2014–2018 were selected. In order to maximally ensure that recorded procedures were indeed performed to diagnose or treat the breast tumour under study and not another malignancy, patients diagnosed between 2004 (i.e., inception of the cancer registry in Belgium) and 2018 with multiple invasive (non-breast) tumours and/or with multiple breast tumours (invasive or in situ) were not included in the study cohort. More precisely, bilateral breast tumours were excluded, but multifocal breast tumours were not. In situ tumours with an ICD-O-3 morphology other than DCIS and invasive breast tumours with an ICD-O-3 morphology corresponding with sarcoma or Paget’s disease were also excluded. Male patients were excluded because the incidence numbers are too small to assess the quality of care which is dispersed over about hundred Belgian hospitals (and even more hospital campuses).

The incidence date was defined as the date of first microscopic (cytological/histological) confirmation of malignancy. If this information was not available, the incidence date was determined in decreasing priority by (1) first hospital admission for cancer, (2) first consultation because of malignancy, (3) initial clinical or technical diagnosis, (4) start of treatment for cancer, and (5) death.

Quality Indicators

From the mandatory EUSOMA QIs, fourteen were selected based on data availability and clinical relevance (online suppl. Table S1; for all online suppl. material, see https://doi.org/10.1159/000543676). Most EUSOMA QIs had to be rephrased so that they could be calculated with the available data (Table 1). For instance, due to the lack of pertinent data one or more elements described in some QI had to be left out (e.g., no data on grading, on the proliferation index) or had to be adapted (e.g., “date of first diagnostic examination” rephrased as “incidence date”). For others, clinical experts were convinced that it was better to focus on patients younger than 70 years old (e.g., QI 13).

Table 1.

Selected EUSOMA QIs

Description of the QI	EUSOMA target	Obtained result (95% CI)
Diagnosis and staging
QI 1: Proportion of women with BC who preoperatively underwent mammography, physical examination, and US of both breasts and axillae	>95%
Rephrased: Proportion of women with IBC who received mammography and breast sonography before any oncological treatment		92.5% (92.3–92.8)
QI 3b: Proportion of women with BC (invasive or in situ) who had a preoperative histologically or cytologically confirmed malignant diagnosis	90%
Rephrased: Proportion of women with IBC with histological or cytological assessment before any oncological treatment		98.0% (97.9–98.1)
QI 4a: Proportion of invasive cancer cases for which the following prognostic/predictive parameters have been recorded: histological type (according to WHO Classification of Tumours of the Breast), grading (according to WHO and EU Guidelines: Elston and Ellis modified Bloom and Richardson-Grading system Elston, CW et al. 1991), ER, PgR*, HER2/neu, proliferation index (Ki67)*	98%
QI 4b: For patients receiving primary surgery, characterisation may be performed on the surgical specimen only. In addition to the above parameters, the following parameters must be recorded after surgery: pathological stage (pT and pN, or ypT and ypN in case of PST), size in mm for the invasive component, peritumoral vascular invasion (L, V), distance to nearest radial margin	>98%
Rephrased: Proportion of women with IBC in whom HER2 status and/or ER and/or PR status were assessed before any systemic treatment		99.4% (99.4–99.5)
Rephrased: Proportion of women with BC for whom a valid cTNM stage is reported to the BCR		DCIS: 66.6% (65.1–68.1)
		IBC: 88.8% (88.5–89.1)
Rephrased: Proportion of women with BC who had surgery, for whom a valid (y)pTNM stage is reported to the BCR		DCIS: 93.4% (92.5–94.1)
		IBC: 96.3% (96.1–96.4)
Waiting time
QI 5: Time interval of ≤6 weeks, from the date of first diagnostic examination within the breast centre to the date of surgery or start of other treatment	90%
Rephrased: Proportion of women with IBC for whom the time interval between the incidence date and the date of first treatment was ≤6 weeks		87.3% (86.9–87.6)
Multidisciplinary discussion
QI 8: Proportion of cancer patients to be discussed pre- and postoperatively by a multidisciplinary team	99%
Rephrased: Proportion of women with BC discussed at the multidisciplinary team meeting from 1 month before to 2 months after incidence date		DCIS: 83.6% (82.4–84.7)
		IBC: 90.1% (89.8–90.4)
Surgery
QI 9a: Proportion of patients (invasive cancer only) who received a single (breast) operation for the primary tumour (excluding reconstruction)	90%
Rephrased: Proportion of patients with IBC (M0) who received a single (breast) operation for the primary tumour (excluding reconstruction)		93.0% (92.7–93.2)
QI 9b: Proportion of patients (DCIS only) who received just one operation (excluding reconstruction)	90%
Rephrased: Proportion of women with DCIS who received just one operation (excluding reconstruction)		85.8% (84.6–86.9)
Radiotherapy
QI 10a: Proportion of patients with IBC (M0) who received postoperative RT after surgical resection of the primary tumour and appropriate axillary staging/surgery in the framework of BCT	95%
Rephrased: Proportion of women <70 years old with IBC (M0) who started radiotherapy within 9 months after BCS		97.5% (97.3–97.8)
Avoidance of overtreatment
QI 11a: Proportion of patients with invasive cancer and clinically negative axilla who underwent SLNB only (excluding patients who received PST)	95%
Rephrased: Proportion of patients with IBC and clinically negative axilla who undergo SLNB only (excluding patients who received neoadjuvant systemic treatment)		76.6% (76.1–77.1)
QI 11e: Proportion of patients with DCIS only who do not undergo axillary clearance	99%
Rephrased: Proportion of women with DCIS who do not undergo ALND as first axillary surgery		96.9% (96.3–97.4)
Systemic treatment
QI 13a: Proportion of patients with ER (T >1 cm or Node+)-invasive carcinoma who received adjuvant chemotherapy	95%
Rephrased: Proportion of women <70 years old with ER-negative (T >1 cm or Node+) M0 IBC who received (neo)adjuvant chemotherapy		NM
QI 13b: Proportion of patients with HER2-positive (IHC 3+ or IHC-positive FISH-positive) invasive carcinoma (T >1 cm or N+) treated with chemotherapy who received adjuvant trastuzumab	95%
Rephrased: Proportion of women <70 years old with HER2-positive (IHC 3+ or ISH positive) M0 invasive carcinoma (T >1 cm or Node+) treated with adjuvant chemotherapy who also received adjuvant anti-HER2 therapy		NM
QI 13c: Proportion of patients with HER2-positive invasive carcinoma treated with neoadjuvant chemotherapy who received neoadjuvant trastuzumab	95%
Rephrased: Proportion of women <70 years old with HER2-positive M0 IBC treated with neoadjuvant chemotherapy who also received neoadjuvant anti-HER2 therapy		NM

DCIS, ductal carcinoma in situ; IBC, invasive breast cancer; EUSOMA, European Society of Breast Cancer Specialists; NM, not measurable; BCS, breast-conserving surgery; cTNM, clinical tumour node metastasis; pTNM, pathological TNM; RT, radiation therapy; SLNB, sentinel lymph node biopsy; ALND, axillary lymph node dissection; ISH, in situ hybridisation.

The number of the QIs refers to Biganzoli et al. [10].

The fourteen EUSOMA indicators were “translated” into fifteen QIs that assessed diagnosis and staging (N = 5), waiting time (N = 1), multidisciplinary discussion (N = 1), surgery (N = 2), radiotherapy (N = 1), avoidance of overtreatment (N = 2), and systemic treatment (N = 3). For each selected QI, the numerator and denominator were defined specifying the inclusion and exclusion criteria, relevant codes for diagnostic and therapeutic procedures were searched in the fee schedule, and limitations of the database were listed (online suppl. Table S2).

The codes of the Belgian fee schedule are not directly linked with the indication (pathology) for which they are used and often the description is not specific enough to identify this indication with certainty. Therefore, only procedures performed within predefined time frames were selected for the study.

Validation in Two Steps

In order to assess the concordance between the data and methods to define diagnostic and therapeutic procedures in the study database on the one hand and the information available in the hospitals (e.g., medical files and financial data, considered as “gold standard”) on the other, a validation study was performed in ten Belgian hospitals. It led to a further optimisation of the methodology.

In a second step, all Belgian hospitals received their individual process indicator results. They were asked to validate these results and the number of patients they were assigned.

Statistical Analyses

Statistical analyses were performed with SAS 9.4 (SAS Institute, Cary, NC, USA). Descriptive statistics include means and standard deviations (SDs), percentages, and 95% confidence intervals (CIs) as well as risk ratios and 95% CI.

Results

The study cohort comprised 46,035 patients diagnosed with IBC and 3,973 patients diagnosed with DCIS (Table 2). The mean age was 62.6 years (SD ± 14.3) in patients with IBC and 58.6 years (SD ± 11.6) in patients with DCIS.

Table 2.

Patient and tumour characteristics of the study cohort, diagnosed between 2014 and 2018

	IBC (N = 46,035)	DCIS (N = 3,973)
Age
<40 years	2,261 (4.9%)	164 (4.1%)
40–49 years	6,798 (14.8%)	716 (18.0%)
50–59 years	10,580 (23.0%)	1,238 (31.2%)
60–69 years	11,611 (25.2%)	1,194 (30.1%)
70–79 years	8,294 (18.0%)	481 (12.1%)
80+ years	6,491 (14.1%)	180 (4.5%)
Comorbidities, n
0	24,330 (52.9%)	2,442 (61.5%)
1	16,362 (35.5%)	1,207 (30.4%)
2	4,938 (10.7%)	301 (7.6%)
3	405 (0.9%)	23 (0.6%)
Combined stagea,c
(y)0b	1,065 (2.4%)	3,774 (100.0%)
Yis	337 (0.7%)
(y)IA	19,104 (42.5%)
(y)IB	1,022 (2.3%)
(y)IIA	10,470 (23.3%)
(y)IIB	5,037 (11.2%)
(y)IIIA	2,914 (6.5%)
(y)IIIB	893 (2.0%)
(y)IIIC	1,123 (2.5%)
(y)IV	3,009 (6.7%)
Unknown	1,061 (2.3%)	199 (5.0%)
Grade
I	6,455 (14.0%)	530 (13.3%)
II	20,954 (45.5%)	887 (22.3%)
III	15,259 (33.1%)	1,234 (31.1%)
Unknownd	3,367 (7.3%)	1,322 (33.3%)

IBC, invasive breast cancer; DCIS, ductal carcinoma in situ; NAT, neoadjuvant therapy.

^aPercentages for stages 0–IV were calculated excluding the unknown category.

^bIn correspondence with TNM 7th and 8th edition, cTis cN0 cM0 tumours are categorised as cStage 0. For IBC, these tumours were clinically assessed as in situ but appeared to be invasive after resection.

^cPatients might have had NAT, resulting in a ypStage in these cases; the combined stage is a summary of the information included in the clinical stage and the pathological stage and is defined as follows: a known pathological stage takes priority over a known clinical stage, except when the presence of metastasis is specified in the clinical stage.

^dThe high proportion of grade unknown is due to the incomplete information BCR received from the hospitals and/or laboratories for anatomic pathology.

Overall, 92.5% (95% CI: 92.3–92.8) of patients with IBC had a mammography and breast ultrasound (US) before the start of any oncological treatment (Table 1), which is somewhat below the EUSOMA target of 95% [10]. Testing for oestrogen receptor (ER) and/or progesterone receptor (PR) and/or human epidermal growth factor receptor 2 (HER2) status as well as histological or cytological assessment were performed in nearly all patients before start of treatment. Yet, for only 66.6% (95% CI: 65.1–68.1) of women diagnosed with DCIS and 88.8% (95% CI: 88.5–89.1) of women diagnosed with IBC, valid clinical tumour node metastasis staging information was reported to the BCR. Results for pathological TNM stage reporting were better (DCIS: 93.4%, 95% CI: 92.5–94.1, and IBC: 96.3%, 95% CI: 96.1–96.4).

For 83.6% (95% CI: 82.4–84.7) of patients with DCIS and 90.1% (95% CI: 89.8–90.4) of patients with IBC, at least one MDT was invoiced from 1 month before to 2 months after incidence date, while EUSOMA sets the target at 99%. In 87.3% (95% CI: 86.9–87.6) of patients diagnosed with IBC, the first treatment was initiated within 6 weeks (42 days) after diagnosis, which is just below the EUSOMA target set at 90%. For 1 in 4 patients, treatment started within 15 days after incidence date, while a quarter of patients waited 33 days or longer to start their oncological treatment. Similarly, 97.5% (95% CI: 97.3–97.8) of women <70 years with IBC started their radiotherapy within 9 months after breast-conserving surgery; for this QI, the EUSOMA target of 95% was attained.

The proportion of patients with IBC and DCIS who received a single breast surgery for the primary tumour was 93.0% (95% CI: 92.7–93.2) and 85.8% (95% CI: 84.6–86.9), respectively, the first being above and the second being below the target (90%) set by EUSOMA. Also, 96.9% (95% CI: 96.3–97.4) of patients diagnosed with DCIS who had surgery did not receive axillary lymph node dissection (ALND) as first axillary surgery. Overall, 76.6% (95% CI: 76.1–77.1) of women with IBC and clinically negative lymph nodes who underwent surgery without systemic treatment in the neoadjuvant setting had sentinel lymph node biopsy only. Among all patients who fulfilled the criteria for this indicator, 21.8% (95% CI: 21.3–22.3) underwent ALND and for another 1.6% (95% CI: 1.4–1.7) no lymph node procedure was reported. None of the three selected QIs on systemic treatment could be calculated because receptor status information, necessary to calculate these QIs, was not available.

Discussion

In this study we illustrate that several EUSOMA QIs can be calculated with administrative, hence retrospectively collected, data, while for others additional data collection is indicated. From the selected QIs on systemic treatment, for instance, none could be calculated. They all require information on ER, PR, and/or HER2 receptor status, which was problematic. Obviously, receptor status information available in the BCR database, obtained via machine learning from pathology reports, does not correspond reliably with the individual medical files used for medical decision-making. Collaborative efforts between BCR, hospitals, and laboratories for anatomic pathology are needed, so that BCR has faultless and comprehensive receptor status information, as imposed by law since 2007 [14].

Similarly, clinical information on resection margins, the number of resected lymph nodes, and whether they were positive could not be retrieved from the available databases; hence, the dedicated EUSOMA QIs could not be calculated. This information is transferred to BCR in pathology reports, which are still not structured or standardised, hence not readily available for research purposes. Moreover, apparently not all complete pathology reports are transferred to BCR.

The lack of information prompted us also to adapt some QIs. For QI4, for instance, “histological type,” “grading,” “proliferation index (Ki67),” “size in mm for the invasive component,” “peritumoral vascular invasion,” and “distance to nearest radial margin” had to be taken out of the QI as these data are not available in the BCR database (Table 1).

Other challenges arose from the vague description of certain invoicing codes, which are in fact primarily intended for administrative purposes. Ideally, one should be able to specify whether an ER, PR, and/or HER2 test have been performed. However, during the study period (2014–2018) there was neither a dedicated invoicing code for HER2 IHC testing, nor a code for ER and PR testing separately. In addition, exploration of the data revealed that the invoicing codes for the combined testing of ER and PR status were not billed. Instead, codes for more general immunohistochemical testing were used to bill ER-PR-HER2 IHC testing collectively. It was thus impossible to disentangle whether ER as well as PR as well as HER2 IHC testing was performed.

The invoicing rules were also a factor. They stipulate, for example, that only one mammography and 1 US per patient per day can be billed. Hence, if a US of the breast is combined with a US another organ, only one is identifiable in the administrative data. Likewise, if a breast puncture is performed under US or radiographic guidance, the dedicated codes cannot be combined with a US or mammography of the breast. Therefore, a limited list of non-breast-specific US codes was included in the calculation of QI1.

In addition, the codes of the fee schedule do not provide any information on the laterality of procedures. This limitation interfered in particular with the interpretation of multiple surgical interventions identified for the same patient (QI9 a and b). From the available data, it was not apparent whether they were performed in the same or in the contralateral breast (e.g., preventive surgery). In the present study, they were considered being performed in the same breast.

For several indicators, the results were at or even above the EUSOMA targets: histological or cytological assessment, receptor status assessment, start of radiotherapy, single (breast) operation in IBC. For others, the results indicated that BC care in Belgium does not meet the targets yet. Clinical tumour node metastasis stage information, for instance, was transferred to BCR for only 66.6% and 88.8% of patients with DCIS and IBC, respectively, while the EUSOMA target is 98%. This result is particularly puzzling knowing that cancer stage reporting is mandatory [11]. Efforts to increase the proportion of patients who have mammography and breast US before the start of treatment are also indicated. Even so, still too many patients with DCIS have to undergo a second breast surgery (14.2%) or ALND (3.1%).

Another important observation that calls for action was that in only 76.6% of women with IBC and clinically negative lymph nodes, the first axillary intervention was confined to sentinel lymph node biopsy, while the EUSOMA target is set at 95%. While the results suggest that too many women have to undergo axillary clearance, which may result in significant additional morbidity and impact the quality of life, clinicians in the field called for a cautious interpretation. This low proportion may (in part) be attributed to incorrect coding practices (“optimisation”), as some healthcare professionals consider the fee corresponding to the correct codes inappropriate. The magnitude of these flaws cannot be assessed. Similar issues were observed in France [15, 16].

Lastly, as with all databases, their correctness and completeness depend heavily on those (responsible for) collecting data. If the codes of the fee schedule are not used in a correct way, this may result in an over- or underestimation of some procedures. Also, if healthcare professionals/hospitals do not invoice diagnostic or therapeutic procedures, they will not be available in the database. However, since the remuneration of Belgian healthcare professionals is mainly based on a fee-for-service system, it can be assumed that the number of non-invoiced procedures is small. Still, administrative errors (e.g., certain fields not completed, procedures billed to the wrong patient) cannot be ruled out. Distinguishing administrative flaws from true suboptimal care – without audits on the field – is impossible.

Yet, working with administrative data does carry not only challenges and limitations, but also important strengths. Firstly, the use of a nationwide cancer registry, linked to administrative healthcare data, offered the advantage that all Belgian hospitals and more than 98% of all cancer cases in Belgium were included [13]. We were not dependent on the centres’ willingness to collaborate, and selection bias is therefore very unlikely. Secondly, no additional data registration by the hospitals or professionals was required. An additional strength of this study is the validation of the methodology in two steps.

Conclusion

The present study revealed that various EUSOMA QIs can be calculated using routinely available data, while for several important aspects of care the quality could not be assessed due to missing data. Collaborative efforts between hospitals, laboratories for anatomic pathology, and BCR are needed to tackle these data issues. This will allow an accurate and comprehensive assessment of the quality of BC care.

Acknowledgments

The authors would like to acknowledge the contribution of all authors of the KCE report “Survival and Quality of care offered in Belgian hospitals with and without recognition for breast cancer” [17], which formed the basis for this manuscript. The authors are also grateful to the hospitals who participated in the validation study.

Statement of Ethics

An ethics statement was not required for this study type as it is based exclusively on data extracted from the BCR database.

Conflict of Interest Statement

The authors have no conflicts of interest to declare.

Funding Sources

The research presented in this manuscript was supported by the government-approved work programme of the Belgian Health Care Knowledge Centre (KCE).

Author Contributions

R.L. wrote the first draft of the manuscript; J.V. and C.D.G. analysed the data; all authors contributed to the conception and design of the study, the interpretation of the data, and revision of the manuscript, and read and approved the submitted version.

Funding Statement

The research presented in this manuscript was supported by the government-approved work programme of the Belgian Health Care Knowledge Centre (KCE).

Data Availability Statement

The pseudonymised data can be provided within the secured environment of the Belgian Cancer Registry upon approval of a reasonable request and the application of apt GDPR regulations.