Quality indicators of clinical cancer care (QC₃) in colorectal cancer

Abstract

Objectives

Assessing the quality of cancer care (QoCC) has become increasingly important to providers, regulators and purchasers of care worldwide. The aim of this study was to develop evidence-based quality indicators (QIs) for colorectal cancer (CRC) to be applied in a population-based setting.

Design

A comprehensive evidence-based literature search was performed to identify the initial list of QIs, which were then selected and developed using a two-step-modified Delphi process involving two multidisciplinary expert panels with expertise in CRC care, quality of care and epidemiology.

Setting

The QIs of the clinical cancer care (QC₃) population-based project, which involves all the public and private hospitals and clinics present on the territory of Canton Ticino (South Switzerland).

Participants

Ticino Cancer Registry, The Colorectal Cancer Working Group (CRC-WG) and the external academic Advisory Board (AB).

Main outcome measures

Set of QIs which encompass the whole diagnostic-treatment process of CRC.

Results

Of the 149 QIs that emerged from 181 sources of literature, 104 were selected during the in-person meeting of CRC-WG. During the Delphi process, CRC-WG shortened the list to 89 QI. AB finally validated 27 QIs according to the phase of care: diagnosis (N=6), pathology (N=3), treatment (N=16) and outcome (N=2).

Conclusions

Using the validated Delphi methodology, including a literature review of the evidence and integration of expert opinions from local clinicians and international experts, we were able to develop a list of QIs to assess QoCC for CRC. This will hopefully guarantee feasibility of data retrieval, as well as acceptance and translation of QIs into the daily clinical practice to improve QoCC. Moreover, evidence-based selected QIs allow one to assess immediate changes and improvements in the diagnostic-therapeutic process that could be translated into a short-term benefit for patients with a possible gain both in overall and disease-free survival.

Article summary.

Article focus

• Quality of Cancer Care (QoCC) studies on specific quality indicators (QIs) developed worldwide since the late 1990s showed both a continuous improvement of oncological care provided by the clinical structures involved and an increased availability of specialised care in the considered areas.

• This study aims to define evidence-based QIs for colorectal cancer (CRC) care, in order to favour a feasible evaluation of the oncological diagnostic-therapeutic process from a population-based cancer registration and data collection point of view.

Key messages

• QIs should be defined, developed and tested with scientific evidence-based rigour in a careful and transparent manner, taking into account their degree of relevancy, validity, reliability and feasibility.

• The selected CRC QIs can be applied in a population-based setting, implying the inclusion of the elderly, considering age as an extremely important determinant of treatment.

Strengths and limitations of this study

• To develop CRC QIs, we used a formal iterative process, the RAND/UCLA Appropriateness Methodology widely diffused and validated within other QoCC research. The selected QIs are representative of the main steps of the diagnostic-therapeutic process.

• Owing to the evidence that research studies demonstrated that single-discipline panels select different indicators than do multidisciplinary panels and to maximise the applicability of CRC QI, we constituted two panels of experts, a local Working Group and an external national/international academic Advisory Board (AB), who could offer a multidisciplinary perspective on practice and also guarantee that the selected QIs and their results would be comparable with national and international data.

• Possible limitations of the current work are the following:

  • The level of evidence found in the literature. This situation is common to many aspects of healthcare, and it was the reason that the expert panel methodology was developed—specifically, to identify the processes that are most likely to be valid measures of quality when the highest level of evidence is not available.
  • The literature selection could have missed some relevant articles. However, members of the Working Group were likely to be very familiar with the literature, and had the opportunity to suggest other indicators based on their experience and literature search; in this way, we believe to have integrated the best research evidence with clinical expertise.
  • The feasibility of measuring indicators in terms of data collection and calculation. However, both the Working Group and AB were concerned about the feasibility, validity and reliability of clinical data collection, necessary for the calculation of each single indicator at the population-based level. In fact, in order to warrant an accurate measurement, those indicators reaching more than 70% of the agreement, confirming their scientific and clinical value, but evaluated at least by one of the experts as not feasible and difficult to be collected at the population-based level, were definitely excluded. In this way, we have overcome the feasibility limit.

Introduction

Research on quality of cancer care (QoCC) performed during the last decade has demonstrated that the increase in knowledge on treatments with proven efficacy does not directly translate into optimal delivery of such treatments to patients. Moreover, accumulating evidence suggests that underuse and overuse of care may occur for patients with cancer.^{1 2} In addition to survival analysis, to evaluate and compare quality of care at the population-based level, the assessment of QoCC has become increasingly important to providers, regulators and purchasers of care, owing to the growing demand for services, rising costs, constrained resources and evidence of variation in clinical practice.³

QoCC studies and structured programmes on specific quality indicators (QIs) have been developed worldwide since the late 1990s, showing both a continuous improvement of oncological care provided by the clinical structures involved and an increased availability of specialised care in the considered areas. Most of these studies have been implemented at the regional level on a territory with uniform legislative, health and geographical characteristics, increasing the likelihood of recruitment of involved clinicians.^{1 4–7}

Until now, in Switzerland, no population-based study on QoCC with a prospective design has been implemented. In addition to the yearly renewed international guidelines for each type of cancer, there is still a need to evaluate the real conditions of care in the community. Population-based Cancer Registry data are therefore essential to describe and reflect the real world and routine care as well as to provide regular feedback to healthcare workers and decisionmakers about the management of a disease in daily practice and those treatments that are routinely prescribed and/or effective in all patient groups.⁸ Moreover, Cancer Registries represent an independent observatory, thus assuring a fair evaluation service and avoiding any conflicts of interest.

We therefore implemented the QIs of the clinical cancer care (QC₃) project, focusing on QoCC about the diagnosis-treatment process in the colon-rectum, prostate, uterus, ovary and lung cancers in the territory of Canton Ticino (South Switzerland).

Colorectal cancer (CRC) is an important health issue worldwide. It is the most common malignancy in Europe (excluding non-melanoma skin cancers) and the second most common in terms of cancer-related mortality.⁹ In Switzerland, CRC is the second and third most frequent tumour in women and men, respectively. About 4000 CRC cases are diagnosed annually, corresponding to a European age-standardised incidence rate equal to 49.4 and 30.6 cases/100 000 inhabitants in men and women, respectively, and representing 11% of all tumours.^10–12 CRC is the third leading cancer cause of death in Switzerland, with approximately 1600 deaths/year, corresponding to a European age-standardised mortality rate equal to 18.5 and 10.6 cases/100 000 inhabitants in men and women, respectively. With a 5-year survival probability equal to 60%, Switzerland is the country with the most favourable prognosis in Europe.¹³ A recent Swiss report with follow-up to 2009 showed an additional 5-year survival increase to 62%.¹¹

The aims of the QC₃ project were as follows: (1) to define and confirm evidence-based QoCC indicators for the tumour localisations cited above, in order to favour a feasible evaluation of the oncological diagnostic-therapeutic process from a population-based cancer registration and data collection point of view; (2) to define and implement at the regional level standards of care for each QoCC measure, in terms of minimum and target requirements. In the present report, we will describe the initial part of the QC₃ project, meaning the process followed to identify the panel of specific QoCC indicators for CRC, as well as the list of QoCC indicators identified and approved both by a dedicated Working Group of local healthcare providers and by an external independent Advisory Board (AB), in a perspective of data collection feasibility by a population-based cancer registry.

Material and methods

The QC₃ project is a prospective, descriptive study on QoCC implemented in a population-based setting; it is performed by the Ticino Cancer Registry over a 3-year time period (2011–2013) on the territory of Canton Ticino (South Switzerland). In this paper, we focus on the initial part of the project: the identification of the CRC QIs which will be used to evaluate QoCC about CRC in our region.

QIs for CRC were developed involving a local expert panel, named the QC₃ Colorectal Cancer Working Group (CRC-WG). Elected members, selected on the basis of their expertise and on their daily clinical involvement in CRC care, were contacted to have their interest confirmed in being involved. The final QC₃ CRC-WG included two pathologists, four gastroenterologists, two oncologists, three surgeons, two radiologists, two radiation oncologists and one nuclear medicine specialist, making up a total of 16 panellists, all working in the public hospitals or in the private hospitals and clinics of Canton Ticino.

Published studies and references were identified through a comprehensive search on PubMed/MEDLINE, using initially specific strings/expressions, such as the following: “quality of care OR quality indicators AND colorectal cancer”, “diagnosis OR diagnostic AND quality indicators AND colorectal cancer”, “pathology OR pathological AND quality indicators AND colorectal cancer”, “surgery OR surgical AND quality indicators AND colorectal cancer”, “radiation oncology OR radiotherapy AND quality indicators AND colorectal cancer”, “chemotherapy AND quality indicators AND colorectal cancer”, “surveillance OR follow-up OR outcome AND quality indicators AND colorectal cancer”, “preoperative care OR perioperative care OR intraoperative care OR postoperative care AND colorectal cancer”, “population-based AND quality indicators AND colorectal cancer”. For each of the identified candidate QIs, we performed a systematic literature review to identify the highest level of evidence supporting the validity of that QI for articles published from 1990 onwards. The reference list of included articles were also examined to identify any additional article that had not been identified in the MEDLINE search. We included all the peer-reviewed articles, excluding case reports, letters, abstracts or editorials. If evidence at the highest level were limited or absent, then lower levels of evidence were evaluated. For example, if data were not available from randomised controlled trials, cohort or case-control studies, case series and expert opinion or clinical guidelines were reviewed. A selection of already approved QIs provided by the American Society of Clinical Oncology (ASCO), the National Comprehensive Cancer Network (NCCN), the National Initiative on Cancer Care Quality (NICCQ), the Quality Oncology Practice Initiative (QOPI) and the Florida Initiative for Quality Cancer Care (FIQCC) were included in the evaluation list, with the aim to transfer them from the clinical setting to the population-based setting.^{1 2 4 7 14–20}

The initial QIs list emerged from 181 sources of literature, and it was proposed to the CRC-WG in the context of an in-person meeting held at the very beginning of the process. The list was then left to the QC₃ CRC-WG's evaluation for a period of 2 weeks. The participants were asked to provide a whole opinion with written comments about those QIs considered pertinent for the assessment of CRC care quality, to suggest additional QIs not already included in the list and to delete those QIs considered to be not suitable. In order to make the selection and evaluation easier, the QIs were subdivided into chapters recalling the Donabedian's²¹ and the National Initiative for Cancer Care Quality schemes: diagnosis and staging, pathology, treatment, follow-up, outcome.²

Delphi round 1

The QIs selection was performed by using a two-step-modified Delphi process.²² The initial list of QIs, reanalysed by the QC₃ CRC-WG, was formatted as a questionnaire, where for each indicator was specified the numerator, the denominator and the sources of evidence from which it was extracted. The questionnaire was distributed by regular mail to the QC₃ CRC-WG, so to maintain it anonymously, along with a stamped, addressed return envelope and an attached letter with the deadline date of 2 weeks from the receipt and instructions for voting. Respondents were asked to rate each QI adopting the RAND Appropriateness Methodology (scales 1–9, 1=extremely inappropriate; 9=extremely appropriate), according to the selection criteria of relevance, scientific soundness (validity, reliability, comparability) and feasibility (precise definition and specification, data feasibility, reliability of data collection).^23–25 Each QI was judged as validated if it reached a strong consensus for acceptance (≥70% of the QC₃ CRC-WG rated the QI with a vote ≥7), discarded if it reached a strong consensus for exclusion (≥70% of the QC₃ CRC-WG rated the QI with a vote ≤3) and as a standby if there was an unclear consensus (4≤ votes ≤6), which implies an eventual in-person meeting.

Delphi round 2

The Delphi Round 2 questionnaire was performed with the same modalities as in the first round and enclosed the frequency distribution of the round 1 votes, allowing the panellists to eventually alter their responses, in the light of colleagues’ assessments.²³

AB review

The list of selected QIs derived from the two Delphi rounds was then submitted to an independent external national/international academic multidisciplinary AB, in order to get an additional evaluation on the suitability of QIs as ‘quality’ indexes according to the criteria shown in the previous paragraph. The intent was to achieve at least one health professional for each specialty. The AB included one pathologist, one gastroenterologist, two oncologists, two surgeons, one radiologist, one radiation oncologist, one nuclear medicine specialist and one epidemiologist, making up a total of 10 experts in CRC care (see Acknowledgements); all the panellists were involved daily in CRC care and they had been contacted with the same modalities of the QC₃ CRC-WG. The selected QIs as well as the corresponding literature sources were distributed to the AB as an electronic form where their opinion about QIs were expressed both as megatrends (ie, response yes/no to the suitability of each QI) and as eventual additional comments.²⁶ We considered every single QI as finally approved by the AB if it achieved ≥70% of the agreement (ie, ≥70% of respondents should have answered ‘yes’). Besides the vote (‘yes’ vs ‘no’), the panellists had the chance to comment on a single QI from a population-based cancer registration and data collection point of view. Therefore, those QIs reaching more than 70% of the agreement, confirming their scientific and clinical value, but evaluated by at least one of the experts ‘not completely feasible and difficult to be collected at the population-based level’, were definitely excluded.

Results

The QI selection process began in January 2011 and ended in December 2011.

Participation of CRC-WG members throughout the process was high: 15 (100%) participated in the in-person meeting, 12 (80%) completed both the Delphi rounds 1 and 2. The Delphi round 1 questionnaire respondent time was in the range of 18–60 days, while for the round 2, the delay time was in the range of 8–55 days; these delays and the time for recruitment of the AB influence the long time spent on this part of the project.

Figure 1 summarises the entire process used to select QI for CRC care. The literature search produced 181 citations dealing with CRC QoCC, also including the already validated QIs provided by ASCO, NCCN, NICCQ, QOPI and FIQCC.^{1 2 4 7 14–20} From this search, we initially selected a total of 149 QIs, which were proposed to the CRC-WG in the context of the initial in-person meeting. The following discussion and revision reduced the list to 104 QIs before the modified Delphi process started; these QIs were divided into the following areas: diagnosis and staging, pathology, treatment, follow-up and outcome. After the whole Delphi process, the list was shortened to 89 QIs, distributed as follows: diagnosis and staging (N=16), pathology (N=20), treatment (N=38), follow-up (N=10) and outcome (N=5). The QIs finally underwent the AB's evaluation; this last step, according to the procedure described in the methods, shortened the final list to 27 QIs: diagnosis (N=6), pathology (N=3); treatment (N=16), follow-up (N=0) and outcome (N=2). Table 1 reports detailed information for each QI: (1) QI description; (2) criteria for patients’ inclusion in the numerator and denominator; (3) list of the necessary medical documentation that should be collected by the Cancer Registry to extract the needed and relevant information to build the specific QI, such as the report of the endoscopy, the pathology report of the biopsy and/or surgical resection, the preoperative radiological reports (eg, CT and MRI), the surgery report, the tumour board documentation, the oncological report, the radiotherapy report and database/documentation of the regional Office of Population Registry Rosters for the assessment of patients’ vital status (for outcome QIs); (4) QI rationale and (5) related references.

Figure 1.

Process used to select quality indicators for colorectal cancer care. AB, Advisory Board; QC₃ CRC-WG, QC₃ Colorectal Cancer Working Group; QI, Quality Indicators.

Table 1.

Quality indicators of colorectal cancer care according to diagnostic-therapeutic process (diagnosis, pathology, treatment—surgery, chemotherapy and radiotherapy—and outcome) and tumour site

Clinical domain	Site	Quality indicator	Numerator	Denominator	Medical documentation	Rationale	Reference
Diagnosis (n=6)	C&R	Proportion of patients with colorectal cancer and diagnosis based on symptoms vs screening vs accidental finding	Number of patients with colorectal cancer whose diagnosis is based on symptoms, defined as appearance or persistence of clinical events and signs, such as rectal bleeding, occult blood in stool, weight loss with no apparent cause, general abdominal discomfort, bowel obstruction, change in bowel habits, constant tiredness, anaemia	Number of patients with colorectal cancer	Request form of endoscopic examination, endoscopy and surgical pathology reports, reports/discharge letters coming from all hospital units/departments (ie, surgery, medicine, radiation oncology, medical oncology)	Assessment of the patient's take charge	18 34–38
			Number of patients with colorectal cancer whose diagnosis is based on screening, defined as regular examination, such as faecal occult blood test (FOBT) or colonoscopy in asymptomatic patients
			Number of patients with colorectal cancer whose diagnosis is an accidental finding following examinations or therapies for other diseases (eg, hospital admission for other causes…)
	C&R	Proportion of patients with colorectal cancer, evaluated by preoperative colonoscopy	Number of patients with colorectal cancer who have been evaluated by a preoperative colonoscopy	Number of patients with colorectal cancer undergoing surgery	Endoscopy report, request form of pathology examination, pathology report of endoscopy	Planning of further diagnostic procedures and treatments, comprehensiveness of diagnostic and staging evaluation	7 16 18 40–41
	R	Proportion of patients with rectal cancer and description of the tumour localisation (distance ab ano) in the endoscopic/pathological documentation	Number of patients with rectal cancer who have the description of the tumour localisation, in terms of distance ab ano, in the endoscopic/pathological documentation	Number of patients with rectal cancer undergoing endoscopy	Endoscopy report, request form of pathology examination, pathology report of endoscopy	Planning of further diagnostic procedures and treatments, comprehensiveness of diagnostic and staging evaluation	1 19 56 57
	C&R	Proportion of patients with colorectal cancer and requests for an initial CT and/or an MRI examination completed by clinical information according to the ACR guidelines	Number of patients with colorectal cancer for which the request of an initial CT and/or an MRI examination is completed by clinical information according to the ACR guidelines	Number of patients with colorectal cancer undergoing initial CT and/or MRI examination	Radiology (CT and/or MRI examination) report	Providing the necessary information for a comprehensive radiological examination, assessment of the quality of the flux of clinical information	7 58
	R	Proportion of patients with low rectal* cancer undergoing pelvic MRI of staging	Number of patients with low rectal* cancer who have undergone a pelvic MRI of staging	Number of patients with low rectal* cancer	Radiology (MRI examination) report, discharge letters coming from all hospital units/departments (ie, surgery, medicine, medical oncology, radiation oncology)	Planning of further diagnostic procedures and treatments, comprehensiveness of diagnostic and staging evaluation	19 59–61
	R	Proportion of patients with rectal cancer and a preoperative MRI reporting the description of the radial margin status (mm)	Number of patients with rectal cancer who have undergone a preoperative MRI reporting the description of the radial margin status (mm)	Number of patients with rectal cancer undergoing preoperative MRI	Radiology (MRI examination) report	Planning of further diagnostic procedures and treatments, comprehensiveness of diagnostic and staging evaluation	62
Pathology (n=3)	R	Proportion of patients with rectal cancer for which the request for the pathological examination includes the information of neo-adjuvant RT±ChT	Number of patients with rectal cancer for which the request for the pathological examination includes the information of neo-adjuvant RT±ChT	Number of patients with rectal cancer undergoing neo-adjuvant RT±ChT and surgery†	Request form of pathology examination, surgical pathology report	Providing the necessary information for a comprehensive pathological examination, assessment of the quality of the flux of clinical information	Proposed by the QC3 CRC-WG
	C&R	Proportion of patients with colorectal cancer and a sufficient number of tumour samples (≥3)	Number of patients with colorectal cancer for which 3 or more tumour samples were processed for the pathological analysis	Number of patients with colorectal cancer undergoing surgery†	Surgical pathology report	Comprehensiveness of pathology examination	Proposed by the QC3 CRC-WG
	C&R	Proportion of patients with colorectal cancer and a surgical pathology report including the following characteristics:▸ Surgical intervention description ▸ Sample length ▸ Tumour localisation according to WHO ▸ Tumour size ▸ Histological type according to WHO ▸ Histological grade ▸ Resection margins ▸ Lymph-vascular invasion ▸ Perineural invasion ▸ Tumour deposits (discontinuous extramural extension) ▸ Pathological staging (AJCC pTNM) ▸ Number of retrieved lymph nodes ▸ Treatment effect ▸ Macroscopic integrity of the mesorectum (for the rectum only)  (this quality indicator should be provided for each characteristic)	Number of patients with colorectal cancer whose pathological report includes the following characteristics: ▸ Surgical intervention description ▸ Sample length ▸ Tumour localisation according to WHO ▸ Tumour size ▸ Histological type according to WHO ▸ Histological grade ▸ Resection margins ▸ Lymph-vascular invasion ▸ Perineural invasion ▸ Tumour deposits (discontinuous extramural extension) ▸ Pathological staging (AJCC pTNM) ▸ Number of retrieved lymph nodes ▸ Treatment effect ▸ Macroscopic integrity of the mesorectum (for the rectum only)	Number of patients with colorectal cancer undergoing surgery†	Surgical pathology report	Comprehensiveness and standardisation of surgical pathology report, comprehensiveness of staging evaluation, planning of further treatments	18 19 63 64
Treatment (n=16)	C&R	Proportion of patients with colorectal cancer who have been operated in emergency‡	Number of patients with colorectal cancer who have been operated in emergency‡	Number of patients with colorectal cancer undergoing surgery†	Radiology and surgery report/discharge letter, surgical pathology report	Assessment of the patient's take charge	65–67
	C&R	Proportion of patients with colorectal cancer and dead within 30 days and 6 months from the surgery (postoperative mortality)	Number of patients with colorectal cancer and dead within 30 days from the surgery† Number of patients with colorectal cancer and dead within 6 months from the surgery	Number of patients with colorectal cancer undergoing surgery†	Surgery report/discharge letter, surgical pathology report, access to regional Office of Population Registry Rosters for the assessment of patients’ vital status	Assessment of the quality of surgical procedure	68–71
	C&R	Proportion of patients with colorectal cancer and postoperative multidisciplinary discussion	Number of patients with colorectal cancer for which there has been a multidisciplinary discussion after surgery†	Number of patients with colorectal cancer undergoing surgery†	Surgery, oncology, radiation oncology reports/discharge letters, multidisciplinary discussion documentation	Planning of further diagnostic procedures and treatments	72 73
	R	Proportion of patients with malignant rectal polyp (pT1) and complete endoscopic polypectomy	Number of patients with malignant rectal polyp (pT1) who have undergone a complete endoscopic polypectomy	Number of patients with malignant rectal polyp (pT1)	Endoscopy report, endoscopic pathology reports	Assessment of the quality of surgical procedure	Proposed by the QC3 CRC-WG
	R	Proportion of patients with low rectal* cancer who have undergone a surgical intervention with sphincter preservation	Number of patients with low rectal* cancer who have undergone a surgical intervention with sphincter preservation	Number of patients with low rectal* cancer undergoing surgery†	Surgical pathology report, surgery report/discharge letter	Assessment of the quality of surgical procedure	7 74–76
	R	Proportion of patients with rectal cancer undergoing TEM with R0 resection	Number of patients with rectal cancer who have undergone TEM with R0 resection	Number of patients with rectal cancer undergoing TEM	Surgical pathology report, surgery report/discharge letter	Assessment of the quality of surgical procedure	77–79
	C&R	Proportion of patients with colorectal cancer and a number of resected lymph nodes ≥12	Number of patients with colorectal cancer with a number of resected lymph nodes ≥12	Number of patients with colorectal cancer undergoing surgery†, but no neo-adjuvant therapy	Surgical pathology report, surgery report/discharge letter	Assessment of the quality of surgical procedure and pathology examination	7 14 16 40 41 80–85
	C&R	Proportion of patients with colorectal cancer operated on with free margins	Number of patients with colon cancer who have undergone surgery and have free margins	Number of patients with colorectal cancer undergoing surgery†	Surgical pathology report, surgery report/discharge letter	Assessment of the quality of surgical procedure	7 86 87
	C&R	Proportion of patients with colorectal cancer and AJCC TNM clinical stage I (from T2N0M0) to III (any T, N1M0) undergoing a surgical resection with anastomosis	Number of patients with colorectal cancer and AJCC TNM clinical stage I (from T2N0M0) to III (any T, N1M0) who have undergone a surgical resection with anastomosis	Number of patients with colorectal cancer and AJCC TNM stage I (from T2N0M0) to III	Radiology report, surgical pathology report,surgery report/discharge letter	Assessment of the quality of surgical procedure	40 41 86 87
	C	Proportion of patients with colon cancer and AJCC TNM stage II (T3N0M0, T4N0M0) high-risk (presence of at least one of the following factors: LN<12, G3, lymph-vascular or perineural invasion, tumour obstruction, tumour perforation, pT4) or III undergoing adjuvant ChT	Number of patients with colon cancer and AJCC TNM stage II (T3N0M0, T4N0M0) high-risk (presence of at least one of the following factors: LN<12, G3, lymph-vascular or perineural invasion, tumour obstruction, tumour perforation, pT4) or III, who have undergone adjuvant ChT	Number of patients with colon cancer and AJCC TNM stage II high-risk or III, undergoing surgery†	Radiology report, surgical pathology report, surgery, oncology reports/discharge letters	Assessment of the quality of oncological treatment	16 18 40–41 88–91
	C	Proportion of patients with colon cancer AJCC TNM stage II high-risk or stage III undergoing adjuvant ChT within 8 weeks from surgical resection	Number of patients with colon cancer and AJCC TNM stage II high-risk or III, who have undergone adjuvant ChT within 8 weeks from surgical resection	Number of patients with colon cancer and AJCC TNM stage II high-risk or III undergoing surgery† and adjuvant ChT	Radiology report, surgical pathology report, surgery, oncology reports/discharge letters	Assessment of the quality of oncological treatment	18 92
	C&R	Proportion of patients with colorectal cancer and histology of the primary tumour or metastases obtained before the beginning of ChT	Number of patients with colorectal cancer and histology of the primary tumour or metastases obtained before the beginning of ChT	Number of patients with colorectal cancer undergoing primary ChT	Radiology and pathology reports, oncology report/discharge letter	Assessment of the quality of oncological treatment	40 41
	C&R	Proportion of patients with colorectal cancer and unresectable metastases undergoing first-line ChT or bio-ChT	Number of patients with colorectal cancer and unresectable metastases who have undergone a first-line ChT or bio-ChT	Number of patients with colorectal cancer and unresectable metastases	Radiology and pathology reports, oncology report/discharge letter	Assessment of the quality of oncological treatment	93–96
	C&R	Proportion of patients with colorectal cancer and hepatic metastases primarily unresectable turned into resectable metastases after neo-adjuvant ChT	Number of patients with colorectal cancer and hepatic metastases primarily unresectable turned into resectable metastases after neo-adjuvant ChT	Number of patients with colorectal cancer and unresectable hepatic metastases undergoing neo-adjuvant ChT	Radiology report, oncology report/discharge letter	Assessment of the quality of oncological treatment	96
	R	Proportion of patients with locally advanced rectal cancer (T3-4 and/or any T, N+ and M0) undergoing neo-adjuvant RT±ChT	Proportion of patients with locally advanced rectal cancer (T3-4 and/or any T, N+ and M0) who have undergone neo-adjuvant RT±ChT	Number of patients with locally advanced rectal cancer undergoing surgery†	Endoscopic pathology report, radiology report, radiation oncology and oncology reports/discharge letters	Assessment of the quality of oncological and radio-oncological treatment	97 98
	R	Proportion of patients with rectal cancer and undergoing neo-adjuvant RT±ChT operated within 6–8 weeks after the end of neo-adjuvant RT±ChT	Number of patients with rectal cancer who have undergone neo-adjuvant RT±ChT and were operated within 6–8 weeks after the end of neo-adjuvant RT±ChT	Number of patients with rectal cancer undergoing neo-adjuvant RT±ChT followed by surgery†	Endoscopic pathology report, radiology report, radiation oncology and oncology reports/discharge letters, surgical pathology report	Assessment of the quality of oncological and radio-oncological treatment	18 98
Outcome (n=2)	C&R	Analysis of overall survival at 1, 3, 5 and 10 years from diagnosis	Number of patients with colorectal cancer who survive at 1, 3, 5 and 10 years from diagnosis	Number of patients with colorectal cancer	Access to regional Office of Population Registry Rosters for the assessment of patients vital status	Assessment of overall survival	7 99
	C&R	Analysis of disease-free survival at 1, 3, 5 and 10 years from the curative treatment	Number of patients with colorectal cancer who are disease-free at 1, 3, 5 and 10 years from the curative treatment	Number of patients with colorectal cancer curatively treated	Reports/discharge letters coming from all hospital units/department (ie, surgery, medicine, oncology, radio-oncology)	Assessment of disease-free survival	7 99

*Low rectum: 4–7.5 cm from the dentate line.¹⁰⁰

†Surgery excludes endoscopic resection and colostomy.

‡Emergency: within 24 h from the onset of symptoms.

ACR, American College of Radiology; AJCC, American Joint Committee on Cancer; C, colon; C&R, colon-rectum; ChT, chemotherapy; CT, computed tomography; MRI, magnetic resonance imaging; R, rectum; RT, radiotherapy; TEM, transanal endoscopic microsurgery; WHO, World Health Organization.

Discussion

In the preliminary phase of the QC₃ project shown in this paper, we developed a panel of evidence-based CRC QIs which are suitable for implementation in a population-based setting.

To develop the QC₃ QIs, we used a formal iterative process, the RAND/UCLA Appropriateness Methodology widely diffused and validated within other QoCC research.^{23 24} Owing to the evidence that research studies demonstrated that single-discipline panels select different indicators than do multidisciplinary panels and to maximise the applicability of QC₃ CRC QIs, we constituted a working group which could offer a multidisciplinary perspective on practice, including specialists, professionals, clinicians and researchers coming from both public and private hospitals.^27–33 Moreover, we have used a further validation step enrolling an independent national/international academic AB. This choice was due to the aim of measuring QoCC within a Swiss region, with a point of view on the population-based data collection and evaluation, and of obtaining results which will be comparable with national and international data. We believe that the expertise and multidisciplinary representativeness of the QC3 CRC-WG and of the AB will surely increase the quality, acceptance and translation of QIs into daily clinical practice.

The selected QIs are representative of the main steps of the diagnostic-therapeutic process. The diagnosis QIs reflect the importance of a preoperative evaluation and staging, reliable evaluation of the tumour localisation and local invasion and particularly for the rectal cancers, of a feasible and effective surgery. The first indicator of the ‘diagnosis’ group is important to understand what happens in a territory where there is no organised screening programme for CRCs, but only an opportunistic screening strategy. Actually, the tumour is detected because the physician submits the patients older than 50 years old to a fecal occult blood test (FOBT) or colonoscopy control or if a patient, being aware of the possible risk, asks his family doctor to undergo screening examinations. These are interesting data to be evaluated, also in the hypothesis of a CRC screening programme implementation. We therefore believe that a higher proportion of patients diagnosed through screening (FOBT or colonoscopy in asymptomatic patients) would represent a higher diagnostic quality, since the therapeutic approach and, consequently, the patients’ outcome (in terms of recurrence and survival) would be more favourable, as reported in the literature.^{20 34–39} The pathology QIs reflect the importance of good communication between clinicians and pathologists in terms of the patient's clinical history and consequent evaluation of the effectiveness of a neo-adjuvant therapy; moreover, there is a need for standardisation of the pathological report following international guidelines (eg, take at least three samples of the tumour during macroscopy), not leaving any items unexplained or implicit. In particular, the third QI reported in table 1 (pathology section) refers to the surgical pathology report, which derives from the surgical curative intervention and should be as complete as possible to be useful for the future decision about the patient's treatment. Our intent is to calculate it not only for all the listed items considered together, but also for each item analysed individually: for example, the proportion of patients with CRC and a definitive pathological report including the surgical intervention description; the proportion of patients with CRC and a definitive pathological report including the tumour size; the proportion of patients with CRC and a definitive pathological report including the resection margins; the proportion of patients with CRC and a definitive pathological report including the pathological staging (AJCC pTNM); etc. The treatment QIs cover the general issues of surgery, such as emergency, postoperative mortality and a multidisciplinary discussion of the clinical case; furthermore, they focus on the debate of the retrieved lymph nodes, on the timing between radiotherapy and surgery, on the adjuvant chemotherapy and on the attitude towards the metastatic patients. The two main items of the outcome chapter refer to the overall and disease-free survival. Although it is necessary to wait for a certain follow-up period (ie, 1, 3, 5–10 years from the date of diagnosis for the calculation of overall survival, and from the date of curative treatment for the calculation of disease-free survival), they will represent the overall resume of the diagnostic and treatment quality of CRC patients. Our intent will be to analyse the overall and disease-free survival according to some of the proposed QIs (such as QI concerning the pathological characteristics of the tumours, QI of the adjuvant chemotherapy in patients with colon cancer and AJCC TNM stage II high-risk or III, QI of colorectal patients operated on with free margins, QI of locally advanced rectal cancer patients undergoing neo-adjuvant radio±chemotherapy, etc). We will finally compare our results with other regional and national reality, favouring the interpretation of each single QI. Concerning the QIs about follow-up, AB did not finally include any of them. Indeed, although the follow-up procedures are suggested by several international guidelines, they are based on levels II–III evidence and controversies remain regarding the selection of optimal strategies for following up patients after potentially curative CRC surgery.^40–43

The first limitation of the current work is the level of evidence found in the literature. For some QIs, strong evidence of their validity was not available from randomised controlled trials. However, this situation is common to many aspects of healthcare, and it was the very reason that the expert panel methodology was developed—specifically, to identify the processes that are most likely to be valid measures of quality when the highest level of evidence is not available.^{19 23 44} Second, we may have missed some studies during the literature search, and consequently, some QIs may not have been proposed to the QC₃ CRC-WG since the beginning of the QIs revision process. However, this limitation should have been overcome by the fact that the members of the QC₃ CRC-WG were likely to be very familiar with the literature, and had the opportunity to suggest other QIs based on their experience and literature search.^{7 27 28 45} Thus, we integrated the best research evidence with clinical expertise, as reported by Sackett et al⁴⁶ A further limit could be the feasibility of measuring QIs in terms of data collection and calculation, which is immediately the next step. Actually, the QIs selected by both the QC₃ CRC-WG and the AB represent an ideal set of criteria to measure the quality of CRC care; at the same time, they both were concerned about the feasibility, validity and reliability of clinical data collection, necessary for the calculation of each single QI at the population-based level. This is the reason why most of the identified QC₃ QIs are common to many QoCC studies.^{1 2 4 7 14–20} Besides the traditional Delphi process, the panellists had the chance to comment on the single QI from a population-based cancer registration and data collection point of view. Therefore, in order to warrant an accurate measurement, those QIs reaching more than 70% of the agreement, confirming their scientific and clinical value, but evaluated at least by one of the experts as not feasible and difficult to be collected at the population-based level, were definitely excluded. In addition, we performed a retrospective preliminary pilot collection on the detailed and necessary incidence data of CRC which occurred in 2011, realising that the measurement of most QIs is feasible, whereas for some selected QIs the retrieval of variables would need additional efforts; some preliminary results were presented in national and international conferences and congresses, receiving positive feedback in both the clinical and epidemiological settings.^47–50 Only the definitive results will give us the proportion of missing information, whose magnitude will be assessed.

The selected QC₃ CRC QIs will be applied in a population-based setting, where age is an extremely important determinant of treatment. The elderly are rarely included in the randomised clinical trials with the consequence of a possible ‘underuse of treatment’.^{25 51 52} At a broad European level, national audit registries in surgical oncology have led to improvements with a great impact and they offered the possibility, as for our project, to perform research on patients that are usually excluded from clinical trials such as the elderly and comorbid patients.^{53 54} Evidence suggests that the relative benefits of treatment for the elderly are similar to those seen for cancer patients in general, though decision-making for treatment becomes more complex as life expectancy, coexisting illnesses and functional status all need to be considered.^{25 51 52} Applying these QIs, and if all these items will be satisfied, we can affirm having a real good quality process of CRC care for the whole population. The foreseeable future in quality evaluation and improvement of healthcare will quite likely involve a more frequent use of QIs by regulatory and accrediting agencies, stakeholders, clinicians, individual hospitals and healthcare providers, as well as patients. This underlines the fact that QIs should be defined, developed and tested with scientific evidence-based rigour in a careful and transparent manner, taking into account their degree of relevancy, validity, reliability and feasibility.^{30 32} Although QIs have been defined in several different ways, all authors agreed that the final aim was the improvement of patients’ outcome.^{31 33 55}

The systematic trend analysis of QIs allows the assessment of immediate changes and improvements in the diagnostic-therapeutic process, which could be translated into a short-term benefit for patients, without waiting for survival analysis typically needing some years to produce, because of the patients’ follow-up. Furthermore, this system of evaluation and autoevaluation could favour the surveillance and monitoring of the comprehensive level of oncological care in the region, the clinical performance homogeneity, the possible weakness of the clinical network and finally the corrective interventions to be adopted to improve QoCC.

With this study, we hope to increase the awareness of the value of QIs in healthcare so as to encourage more uniform practices and improve provider documentation of medical care in our region; moreover, we hope that the standardisation of QIs among different regions will help to define the threshold of a minimal standard of care.