Benchmarking and audit of breast units improves quality of care

Abstract

Quality Indicators (QIs) are measures of health care quality that make use of readily available hospital inpatient administrative data. Assessment quality of care can be performed on different levels: national, regional, on a hospital basis or on an individual basis. It can be a mandatory or voluntary system. In all cases development of an adequate database for data extraction, and feedback of the findings is of paramount importance. In the present paper we performed a Medline search on “QIs and breast cancer” and “benchmarking and breast cancer care”, and we have added some data from personal experience. The current data clearly show that the use of QIs for breast cancer care, regular internal and external audit of performance of breast units, and benchmarking are effective to improve quality of care. Adherence to guidelines improves markedly (particularly regarding adjuvant treatment) and there are data emerging showing that this results in a better outcome. As quality assurance benefits patients, it will be a challenge for the medical and hospital community to develop affordable quality control systems, which are not leading to excessive workload.

Introduction

High-quality services are essential to optimize treatment results of women with breast cancer. They can be achieved by accurate training, specialization, volume levels and a multidisciplinary approach, involving many different subspecialists, nursing staff and supporting staff members (Perry et al., 2008). Due to the increasing complexity of the breast teams it is of paramount importance to develop structured care in order to avoid a chaotic and non-consistent management of patients and ameliorate adherence to guidelines.

It has been reported that in Europe there are wide differences in treatment offered to patients with breast cancer in terms of mastectomy and radiotherapy rates and the use of adjuvant chemotherapy and hormone therapy, which result in considerable survival differences (Del Turco et al., 2010). A recent Italian survey showed adherence to the breast cancer guidelines of the Italian Association for Medical Oncology was only seen in 71% of 355 cases examined in 35 representative Italian oncological units (Barni et al., 2011). The Florida Initiative for Quality Cancer Care (a consortium of three academic and eight community hospitals in Florida) is a physician and practice-based quality improvement project that was conceived to study the barriers of delivering high quality cancer care in Florida (Gray et al., 2011). Of the 34 quality indicators (QIs) for breast cancer care they evaluated, seven for medical oncology and four for surgical oncology fell below the 85% level of adherence. A national process survey in the USA providing benchmark data showed that a crucial element in the treatment of women with breast cancer, namely determination of the estrogen- and progesterone receptor status was performed in only 83% and 81% of the patients (Owen et al., 2009). Adherence to quality measures was less than 85% for 18 of the 36 defined breast cancer quality measures (Malin et al 2006). These observations highlight a gap between optimal and actual care, that is, between what evidence has identified as recommended care and what patients actually receive (Asch et al., 2006). There is a world-wide need for tools to improve quality of care in the daily clinical practice.

QIs are measures of health care quality that make use of readily available hospital inpatient administrative data. They can be used to highlight potential quality concerns, identify areas that need further study and investigation, and track changes over time. We have recently shown that the use of QIs for breast cancer care, combined with regular internal and external audits of performance, are effective to improve adherence to guidelines in a breast cancer unit (van Dam et al., 2013). Assessing quality of care can be performed on different levels: national, regional, on a hospital basis or on an individual basis. It can be a mandatory or voluntary system. In all cases development of an adequate database for data extraction is of major importance. In the present paper we performed a Medline search on “QIs and breast cancer” and “benchmarking and breast cancer care”, and we have added some data from personal experience.

Assessing process and outcome parameters by means of population based registries

Various groups have called for a national system to monitor the quality of breast cancer care extracting data from existing cancer registries for the entire population. This allows scrutinizing breast cancer care in an entire region or country without exception of certain breast clinics. By analyzing performance on a national basis, one can identify more global and system-level elements that can be addressed to achieve more widespread quality improvement efforts (Chen et al., 2011).

In order to do this type of analysis it is mandatory that the registries contain a minimum number of parameters which can be used as QIs, the data should be reliable and the data sets complete for the vast majority of patients. Malin et al. (2002) studied the validity of such information in the California Cancer Registry by comparing the individual records of 304 women with a new diagnosis of breast cancer with the registry data. The accuracy of registry data was higher for hospital based services (eg. sensitivity = 95% for mastectomy, 95% for lumpectomy, 96% for lymph node dissection) than for ambulatory services. They concluded that registries could provide the infrastructure for collecting data on the quality of cancer care, although the cancer registry data may not be valid for all care settings.

Caldarella et al. (2012) confirmed that it is feasible to evaluate quality of cancer care using cancer registry population based data and major computerized information systems. They tested a set of breast cancer care QIs on the Tuscan Cancer Registry which collects tumor cases diagnosed in all residents in Tuscany. The estimation of the selected indicators confirmed a good homogeneity among areas, and globally a good intraregional performance.

In a similar survey McCarthy et al. (2008) collected data sets relating to treatment of four common cancers (breast, colorectal, lung and prostate) from the data bases of 188 National Health Service hospital trusts in the UK. Data sets were estimated mostly 80% complete. Process indicators were evaluated and patient outcome measures included the National Survey of Patients Satisfaction for Cancer, and cancer survival drown from cancer registration. They showed that while this datamining system of evaluation of quality of care was not yet used in practice, it is possible to implement this in performance assessment of cancer networks.

Recently a French group (Ferrua et al., 2012) has identified 8 QIs to evaluate the timelines of breast care. They feel that 3 of them are ready for nationwide implementation in France (time to surgery, time to postoperative multidisciplinary team meeting, conformity to decision of multidisciplinary team meeting).

The team of the Belgian Health Care Knowledge Centre developed a set of indicators to monitor the quality of breast cancer care in Belgium (Stordeur et al., 2012). QIs were identified from a systematic literature search and the 2010 Belgian evidence based clinical practice guideline. The selection process involved an expert panel evaluating reliability, relevance, interpretability and actionability of each indicator. The QIs were tested using the Belgian Cancer Registry data linked with claims data for all women registered with breast cancer in Belgium between 2001 and 2006 (N = 50.039). This selection process led to a final set of 32 indicators. Of these, 12 were measurable using the available data, while one indicator was measurable using proxy information. The authors showed that by data mining, linking the National Cancer Registry to claim data, these 13 indicators could be used as a multidisciplinary set of QIs for breast cancer. In a second study on the same data set they selected the women treated with invasive breast cancer between 2004 and 2006 (Vrijens et al., 2012). A total of 25.178 women had their treatment in 118 hospitals. These were classified according to their annual volume of treated patients: < 50 (very low), 50-99 (low), 100-149 (medium) and > 150 (high). Six of the eleven process indicators showed higher rates in high-volume hospitals: discussion of the case in a multidisciplinary team meeting, cytological and/or histological assessment before surgery, use of neo-adjuvant chemotherapy, breast conserving surgery rate, adjuvant radiotherapy after breast conserving surgery, and follow-up mammography. Higher volume was also associated with improved survival. The 5-year observed survival rates respectively were 74.9%, 78.8%, 79.8% and 83.9% for patients treated in very-low, low-, medium- and high-volume hospitals. After case-mix adjustment, patients treated in very-low or low-volume hospitals had a hazard ratio for death of 1.26 (95% CI: 1.12-1.42) and 1.15 (95% CI 1.01-1.30) respectively compared to high volume hospitals. The authors concluded that the survival benefits reported in high volume hospitals suggest a better application of recommended processes of care, justifying the centralization of breast cancer care in such hospitals. It is clear that this type of national audits may lead to adaptation of policies. A result of the above report was the political decision to implement the prospective evaluation of 17 QIs in the recognized breast clinics in Belgium, which up to now should treat at least 100 new patients a year. These QIs will be centrally evaluated on a continuous basis by the government and breast clinics should perform according to a minimum standard (based on these 17 QIs) in order to keep their certification.

Quality assurance through voluntary benchmarking and certification

During the last decade prospective voluntary benchmarking procedures of breast units have been established. Breast clinics have to keep a prospective database of all breast cancer patients they treat and this is at least annually audited by an independent external organization. This can be done electronically on the data in the database or by a regular in vivo site visit of the unit. Breast clinics meeting a minimum standard of quality of breast cancer care get a certification for a certain amount of years. This accreditation has to be renewed on a regular basis. A major advantage of this system is that the quality of the data entered in the database is superior to data which are retrieved by dataprocessing of national registries. The down side is that only units who are willing to comply to the certification process are looked at.

In collaboration with the Deutsches Onkologie Centrum a prospective interventional multicentre study was started in 2003 in Germany in order to establish a supraregional collaborative network of breast centers to provide proof of concept for centralized collection and independent analysis of relevant quality assurance data, and to define suitable QIs for benchmarking the quality of breast cancer care (Brucker et al., 2008). Data collection depended on voluntary participation of breast centers in a commercial benchmarking procedure conducted by an independent external institute. Patient data were anonymised, encrypted and submitted for analysis twice a year. Nine guideline-based clinical parameters designed to attainment of predefined quality targets were defined as rate-based QIs. The DKG/DGS dual certification process in its present form was established in July 2003. Essentially it combines compliance with the Requirements of Breast Centres (Fachliche Anforderungen fur Brustzentren developed by DGS, DKG) and implementation and maintenance of a certified quality maintenance system. During the period 2003 to 2010 the number of certified breast centers increased form 59 to 210. In 2010 about 90% of the new breast cancer cases in Germany, currently estimated at about 57970 per year, were diagnosed and treated in a certified breast centre (Wallwiener et al., 2012). The initial set of nine QIs had increased to 18 QIs as surrogate indicators of long-term outcome quality. The 2003-2010 period saw marked increases for the following QIs: preoperative histological diagnosis from 58 to 96%, guideline-concordant endocrine therapy in hormone receptor-positive patients from 27 to 97%; guideline accordant adjuvant and neoadjuvant chemotherapy (no age limit) from 32 to 78%; radiotherapy after breast conserving surgery from 20 to 87%; and radiotherapy after mastectomy from 8 to 74%. There is no doubt that the German voluntary program for external benchmarking has produced remarkable results. It is highly likely that the improved quality of breast cancer care will result in better outcomes. The collection of longitudinal follow-up data on the entire population of patients treated in certified breast centers is ongoing to look at the effects of the treatment related interventions on survival of patients. Beckmann et al. (2011) looked at a sample of 3940 of these patients treated for invasive breast cancer in Middle Franconia (Germany). Patients undergoing treatment in certified breast centers were younger, had lower disease stages and lower grading. Independent of the effects of these variables, patients treated in a certified breast centers had a better overall survival in the adjusted Cox model (hazard ratio 0.70; 95% CI: 0.52-0.93.

The European Society of Breast Cancer Specialists – EUSOMA – has started a voluntary certification process to assess the clinical performance in dedicated European units (Perry et al., 2008; Greco et al., 2006; Del Turco et al., 2010). So far, 32 breast units in Europe have been recognized to comply with the requirements requested by EUSOMA and other European Union guidelines on the basis of information collected by a questionnaire and by a site visit carried out by an independent team of breast cancer experts. A set of QIs was defined by experts from different disciplines based on a literature review. For each of them they reported the definition, minimum and target standard, motivation for selection and level of evidence. Overall 17 main QIs have been identified, respectively 7 on diagnosis, 4 on surgery and loco-regional treatment, 2 on systemic treatment and 4 on staging, counseling, follow-up and rehabilitation (Del Turco et al., 2010). EUSOMA has selected 10 basic indicators (Table 2) to be used for certification purposes. These clearly defined quality parameters, continuous internal audit and external social control by means of a site visit are of paramount importance to optimize adherence to evidence based guidelines and treatment results. Specialized breast units should further comply to the following requirements: Breast surgery should be performed by a specialist in breast surgery. There should be specialized radiotherapists, medical oncologists, pathologists, radiologists and breast nurses to take care of breast cancer patients. The unit should treat at least 150 new cases with breast cancer a year should have specialized clinics, genetics, psychological and social support available (Perry et al., 2008). Looking at the performance of individual units (see below), there is no doubt that this system improves the quality of breast cancer care. Currently an analysis is ongoing on the data of the entire population of women treated in EUSOMA certified centers, consisting of more then 64000 patients, to assess the impact of this system of audit and certification on the performance of treatment of women with breast cancer.

In the United States the National Accreditation Program for Breast Centers (NAPBC) is in the process of being developed. Accreditation is granted only to those centers that voluntarily committed to provide the best care in the diagnosis and treatment of breast cancer and are able to comply with NAPBC standards. Each center must undergo a rigorous evaluation and review of its performance and compliance with NAPBC standards. The most efficient methods of data collection are currently being defined (Winchester, 2011)

Audit of breast cancer care in individual hospitals

From an institutional standpoint analysis of QIs identifies areas of good and lower quality in the continuum of breast cancer care. Indicators with < 85% performance indentify specific clinical processes in most need for quality improvement. Hospitals in the Midwestern part of the Netherlands carried out a clinical audit to monitor the quality of breast cancer care during the years 2002-2008 (Verbeek et al., 2011). Nine QIs were evaluated over the years. In 2004 and 2005 the hospitals also carried out an intervention project aimed at amelioration in quality over time. At the end of the project all nine indicators showed significant improvement compared to the start of the project. Discussion of treatment before and after surgery took place more often (respectively 83 % versus 53% and 96 versus 70%.The National Guideline for maximum waiting times was met more often for the outpatient clinic (74% versus 61%), time to diagnosis (92% versus 82%), and surgical treatment (52% versus 34%). More sentinel lymph node procedures were performed successfully (92% versus 69%), and for more patients more than 10 lymph nodes were evaluated in case of axillary lymph node dissection (85% versus 58%). More patients had definitive surgical treatment consisting of one surgical intervention (87% versus 75%).

Rizzo et al. (2011) reported the compliance with the three National Quality Forum guidelines before (2005-2006) and after (2008) implementations in 2007 in a metropolitan public hospital in Atlanta. Patients receiving radiation therapy increased from 76% tot 96%. Patients receiving or considered for adjuvant chemotherapy or hormonal therapy increased from 74% to 94% and from 84% to 90% respectively (Iyengar et al., 2010).

Weber et al. (2012) assessed the adherence to Breast Cancer Care Quality Indicators (BCCQI) before and after the introduction of a patient navigation program helping to guide patients through the complex multidisciplinary cancer system, in a retrospective cohort of 134 consecutive patients treated between 1st January 2006 and 31th December 2006 and 234 consecutive patients between 1st January 2008 and 31th December 2009. Overall, compliance with BCCQI improved from 74.1 to 95.5% (p < 0.0001). The impact of disease-free and overall survival remains to be determined. Similar observations were reported by Chen et al. (2011) assessing QIs in a cohort of 105 patients with newly diagnosed stage I-III breast cancer patients in a Los Angeles public hospital.

Onerheim et al. (2008) showed that surveillance of the quality of surgical pathology reports of patients undergoing segmental resections of breast cancer was associated with significant improvement in the quality of the reports. Fifty one Quebec hospitals participated in a voluntary project in 1999 and 50 in 2003. Overall, conformity improved from 85% in 1999 for the first evaluation, to 92.5% in 2003 for the second evaluation (p < 0.001). Six of the 7 indicators showed an improvement between the first and second evaluations. Conformity was weakest for recording the distance of the resection margins (68%) and presence of lymphovascular invasion (61%) in 1999.

We recently performed an analysis of evolution of the EUSOMA quality indicators in the breast unit of Sint Augustinus Hospital for the period 2002-2010 (van Dam et al., 2013). Multiple QIs were collected prospectively, and feed back and discussion of data with the breast team was organized at least annually. If necessary corrections in policy were made based on the internal audit data. Process indicators are given in Table 1 for 2002 and 2011. Evolution of the major mandatory quality indicators defined by EUSOMA (www.eusomadb.org/indicators.htm) is shown in Table 2. It can be seen that most of these indicators are significantly better in 2011 compared to 2003. EUSOMA criteria were met for all indicators in 2011. Four years progression free survival was significantly (p < 0.05) better in the cohort of patients treated in 2006-2008 compared to 2004-2005 and 2002-2003. Similar results were obtained if our data were stratified for T1 tumors only and T2-T4 tumors (van Dam et al., 2013). Although our survival results can partly be explained by an evolution in the case mix, with considerably more patients with small tumors and negative lymph nodes in more recent years, better adherence to guidelines is likely to be beneficial for the outcome of the patients. In 2011 more the 97% of patients had state of the art adjuvant radiotherapeutic, antihormonal or cytostatic treatment when indicated according to Sankt Gallen guidelines compared to 98%, 85% and 72% respectively in 2003.

Table 1.

Process indicators in the period 2002-2011 in the clinical pathway “operable breast cancer” of the Sint Augustinus Hospital. Data were collected in the first semester of each year.

Indicator	2002	2011	p-level
Average hospital stay/patient (days)	7.0	4.1	p < 0.01
Breast conserving surgery (%)	43%	58%	p < 0.01
Preoperative guide wire (%)	14%	27%	p < 0.01
Sentinel node biopsy (%)	0%	49%	p < 0.001
Preoperative staging tests missing (%)	53%	8%	p < 0.01
Proportion second surgery (%)	25%	10%	p < 0.005
Clear margins at last surgery (%)	95%	99.5%	p = 0.016

Table 2.

Evolution of quality indicators as formulated by EUSOMA prospectively evaluated between 2003 and 2011 (data for first semester breast clinic Sint Augustinus Hospital, Antwerp).

Outcome measure	2003	2011	p-level	Minimum Standard
Positive preoperative cyto/histological diagnosis	59.7%	88.3%	0.0001	80%
Operated invasive carcinoma for which histological type, grading, ER/PR status, stage and size were recorded	94.1%	96.4%	NS	90%
Operated non invasive carcinoma for which size, histological type and grading are recorded	72%	100%	NS	90%
More then 9 lymph nodes removed when axillary lymph node dissection performed (sampling excluded)	85.6%	86.4%	NS	85%
Postoperative radiotherapy in non-metastatic invasive carcinoma treated with breast conserving surgery	98%	97.3%	NS	90%
Breast conserving surgery in invasive carcinoma with total size up to 30 mm (including DCIS component)	62%	88%	0.001	70%
Breast conserving treatment for in situ carcinoma up to 20 mm diameter	43.8%	88%	0.0016	70%
Ductal carcinoma in situ without axillary dissection	85.7%	97.1%	NS	80%
Hormonotherapy in endocrine sensitive invasive carcinoma	84.8%	93.4%	0.0002	80%
Adjuvant chemotherapy in ER negative (pT1c+ or N+) invasive carcinoma	72%	97.1%	0.0280	80%

N: number; ER: estrogen receptor; PR: progesterone receptor.

A recent Taiwanese study shows that when breast cancer patients are diagnosed and treated in complete accordance with widely accepted standards of care, they survive longer and have better outcomes (Cheng et al., 2009). This prospective study followed 1378 newly diagnosed breast cancer patients from 1995-2001 in a single cancer hospital, tracking 10 indicators of care quality and a assessing the progression of disease up to June 2007. Adherence to all 10 QIS by 100% of patients was associated with better overall (HR 0.46, 95% confidence interval 0.33-0.63) and progression-free survival (HR 0.51; 95% confidence interval: 0.39-0.67). Adherence to either the four treatment indicators, or the six diagnostic indicators by 100% of patients was also associated with a significant improvement of survival.

Audit of breast cancer care on an individual basis

Nowadays it is perfectively feasible to give individual physicians feedback on their treatment results. Most national and hospital databases collect data on the specialists who have participated in the treatment of breast cancer patients. Although one has to be very careful to audit individual performance of hospital physicians, as differences in case mix may bias the conclusions, it may be useful to do this type of analysis to identify possible points of amelioration in breast care. If feedback can be given on these findings in discrete and a non-hostile way it can help doctors to adapt certain habits and to reflect more on their clinical activities. An internal audit in the Sint Augustinus hospital in 2007 showed for example that patients treated by the core breast team compared to patients treated by other collaborating physicians had a preoperative core biopsy rate of 70% versus 62%, a mastectomy rate of 25% vs 42%, free margins of 100% versus 97%, use of sentinel node biopsy in T1-T2 of 82% versus 35%. Feedback of these findings and discussion of individual cases in the multidisciplinary meetings improved performance of both groups. For example, preoperative histological diagnosis was nearly 90% for all patients treated in Sint Augustinus in 2011. Individual discussion of cases with surgeons having positive margins with invasive carcinoma at definitive surgery has nearly eradicated this problem in our hospital. It is clear that a certain feeling of social control improves adherence to guidelines.

Conclusion

The above review shows that quality control by means of internal and external audit and benchmarking leads to better breast cancer care. Adherence to guidelines improves markedly and there are data emerging which show that this results in a better outcome. As this field of quality control is relatively new there is certainly room for more research on the development of more sophisticated tools to measure quality of care. Current QIs are sometimes unbalanced, as many indicators are encouraging more appropriate care but few indicators discourage inappropriate care. Particularly in older patients life-expectancy calculations may be useful to limit unintended harm. If the estimated life expectancy and the lag-time-benefit are similar, then the net benefits are small or uncertain (Lee et al., 2011). A crucial element in quality control is the establishment and maintenance of an accurate database. This has a considerable economical cost which only indirectly benefits the patients. However, the advantages are measurable and important. Resources will have to be found to pay for man (woman) power and computer systems to perform quality management. It will be a challenge for the medical and hospital community to develop quality control systems which are not leading to an excessive (data input) workflow for clinicians and which will not be perceived as an unpleasant “big brother is watching you”. However, as the speedometer in a car is crucial to drive safely, quality measurement is of paramount importance to care safely.