Quality indicators in lung cancer: a review and analysis

Introduction

Optimal management of patients diagnosed with lung cancer is rapidly evolving with updated evidence, and is often complex and involves multimodality treatment that requires a coordinated approach. Internationally, numerous clinical practice guidelines (CPGs) have been developed in order to provide a framework for evidence-based best practice care to guide clinician decision-making.¹ How CPGs and other standard of care are implemented into daily practice needs to be measured to be able to identify areas for quality improvement and address barriers to care, to ensure the delivery of high-quality care.

Quality indicators (QIs) are used to monitor and evaluate various aspects of the quality of healthcare services received by patients in daily practice. They are defined as ‘measurable elements of practice performance for which there is evidence of consensus that they can be used to assess the quality of care’.² Evaluating the quality of care received in ‘real-world’ clinical practice is crucial for optimising health outcomes for patients with lung cancer. QIs provide a means to measure the receipt of best practice care as determined by evidence and expert consensus.

Determining the usefulness of QIs and how they should be used depends on what is intended to be achieved. There are a wide variety of lung cancer QIs that have been developed and are in use. QIs may be used as a measurement tool to document standards, identify variations in care between patient groups or over time, guide performance improvement (including informing policy making), as well as promote transparency and accountability.^{3 4} There are differences in the methods employed to develop QIs, how data are collected and the ways they are used by healthcare providers and patients. The aim of this study was to review and analyse current QIs used in all aspects of lung cancer management.

Materials and methods

A literature review was conducted using the search terms ‘quality indicators’ and ‘lung neoplasms’ limited to English from the time period of 2001 to 2019 using the Medline database. Deploying these search terms in other databases yielded a large number of non-specific publications and so the decision was made to restrict the search to Medline. In addition, grey literature was also searched using a web search of government and relevant health organisation websites. References, abstracts and articles were managed using EndNote software.

Full-text review by a single oncology clinician reviewer was performed to include only articles that fulfilled inclusion criteria of original research that developed or applied QIs related to the care of adult patients with lung cancer. Data were collected for each individual indicator including the description, numerator, denominator, type of indicator, treatment modality, frequency, characteristics, data source authors used for measuring QIs, measured results, benchmarking, use in composite scores, detection of differences between variables, link to outcomes, assessment or practice testing and adjustments for confounding factors. The type of indicator was classified according to the Donabedian model of structure, process or outcome measures. Structure measures reflect the attributes of the whole service, process measures reflect what happens to the patient during care and outcome measures what the effects or end result of care provided to the patient.²

These data were analysed and synthesised using previously published characteristics for ideal QIs including method of development or selection process of indicator, measurability and potential to discriminate or detect differences (table 1).^{3 5–8} An analysis of QIs classified an indicator as meeting all characteristics in a minimum set of desirable characteristics for QIs or not. The minimum set included (1) evidence-based or developed by RAND-modified Delphi process⁵; (2) feasible or measurable (assessed by documented measurement with the QI); (3) were shown to be able to discriminate/detect variation in care. The capacity to discriminate was assessed as fulfilled if studies documented the QI had been used to detect statistically significant variations in care. This included, but was not limited to, patient characteristics such as age, treatment characteristics such as differences between facilities and changes detected over specified time periods. Validity and reliability, sensitivity and specificity, and relevance depend on the population being studied and type of data collected, so could not be assessed in our study.

Table 1.

Characteristics of ideal quality indicators*

Characteristics	Explanation
Well defined	Explicitly defined inclusion and exclusion criteria
Specific	Few false positives
Sensitive	Few false negatives
Valid	Robust selection process or development
Reliable	Minimal interobserver or intraobserver variability
Able to discriminate	Ability to detect variation or change for comparisons
Based on evidence and clinically relevant	An acceptable identifiable event for user
Feasible or measurable	Can be measured with data that are available

*From Mainz and Campbell et al.^{3 5}

Results

Search results

A literature Medline search resulted in 75 abstracts. The full-text screening resulted in the exclusion of 26 articles that did not meet the inclusion criteria and 1 duplicate. Eight additional publications were identified through grey literature searching, and a total of 56 articles were included (figure 1).^9–64

Figure 1.

PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) diagram for selection of studies for review.

Study characteristics

All of the studies included as part of the review either developed QIs, evaluated QIs or measured QIs in the management of patients with lung cancer. Varying techniques were used for developing or electing indicators between publications and within publications for individual indicators. These included evidence-based, literature review, consensus expert review and a structured review approach, such as a RAND-modified Delphi process.⁵ The most robust method of developing indicators is by a structured panel review, such as the RAND-modified Delphi method. During this process, panel members independently rate indicators, traditionally on clinical face validity, and provide feedback over a number of rounds to provide an assessment of the indicators’ utility.⁶ Thirty-five studies reported using indicators for assessing quality of care in a patient population with varying data sources used. The included studies, study type, number of QIs published and data source for measurement of QIs are listed in online supplemental appendix A.

Quality indicators

We found a total of 304 unique QIs, of these indicators 42 (13.8%) were structural measures, 235 (77.3%) were process measures and 27 (8.9%) were outcome measures. The types of QIs are depicted in table 2 and are divided into the most relevant components of lung cancer management along the continuum of care, including prevention, screening, diagnosis, staging, pretreatment assessment, treatment and follow-up. The most frequently reported indicators were related to surgery (n=71, 23.4%), symptom assessment and management (n=43, 14.1%), and diagnosis and staging (n=40, 13.2%). There were fewer indicators related to systemic therapy (n=33, 10.9%), radiotherapy (n=18, 5.9%), combined treatments (n=10, 3.3%), supportive care (n=25, 8.2%) or palliative care (n=8, 2.6%). The remaining indicators measured screening or early detection (n=3, 1.0%), general pretreatment assessment (n=3, 1.0%), preoperative assessment (n=22, 7.2%), non-specific treatment (n=11, 3.6%), general outcomes (n=8, 2.6%), prevention (n=1, 0.3%) and follow-up (n=8, 2.6%).

Table 2.

Types of quality indicators for lung cancer

Management continuum of care in lung cancer		All QIs assessed (n, %)
Prevention	Prevention	1 (0.3)
Screening	Screening or early detection	3 (1.0)
Diagnosis	Diagnosis	21 (6.9)
Staging	Staging	18 (5.9)
Pretreatment assessment	General pretreatment assessment	4 (1.3)
	Preoperative assessment	22 (7.2)
Treatment	Surgery	71 (23.4)
	Systemic therapy	33 (10.9)
	Radiotherapy	18 (5.9)
	Combined treatment	10 (3.3)
	Non-specific treatment	11 (3.6)
	Symptom assessment	25 (8.2)
	Symptom management	18 (5.9)
	General outcomes	8 (2.6)
	Supportive care	25 (8.2)
	Palliative care	8 (2.6)
Follow-up	Follow-up	8 (2.6)
	Total	304 (100)

QIs, quality indicators.

Assessment of indicators

Those indicators that were measured were reported to be feasible indicators. Data sources that were used to measure indicators included administrative data, clinical registry data, medical records, prospectively collected clinical data, patient reported or questionnaires. These data were both retrospectively and prospectively collected. Of these indicators, 106 (34.9%) were also able to detect differences or discriminate between factors such as facilities, time periods, patient, disease or treatment characteristics. Examples of patient, disease or treatment characteristics included stage of disease, availability of multidisciplinary team, comorbidities, facility volume, treating clinician, patient residence location, marital status and gender.

Only 73 (24.0%) of the 304 QIs met the minimum criteria set for characteristics of an ideal QI. The QIs that met the minimum criteria can be found in online supplemental appendix B. Their characteristics are shown in table 3. These included 12 (16.4%) related to diagnosis and staging, 4 (5.5%) to pretreatment assessment, 13 (17.8%) to surgery, 12 (16.4%) to systemic treatment, 9 (12.3%) to radiotherapy or chemoradiotherapy treatment, 3 (4.1%) to general treatment, 3 (4.1%) to symptom assessment, 3 (4.1%) to general outcomes, 11 (15.1%) to supportive care and 1 to palliative care (1.4%).

Table 3.

Assessment of lung cancer QIs that met the minimum criteria

Management along the continuum of care		Met minimum criteria for QI*
		All QIs (n)	Process QIs (n)	Structural QIs (n)	Outcome measures (n)
Prevention	Prevention	0	0	0	0
Screening	Screening or early detection	0	0	0	0
Diagnosis	Diagnosis	5	4	1	0
Staging	Staging	7	7	0	0
Pretreatment assessment	General pretreatment assessment	2	2	0	0
	Preoperative assessment	2	2	0	0
Treatment	Surgery	13	4	1	8
	Systemic therapy	12	11	1	0
	Radiotherapy	6	5	1	0
	Combined treatment	3	3	0	0
	Non-specific treatment	3	2	1	0
	Symptom assessment	3	3	0	0
	Symptom management	0	0	0	0
	General outcomes	5	0	0	5
	Supportive care	11	11	0	0
	Palliative care	1	1	0	0
Follow-up	Follow-up	0	0	0	0
	Total	73	55	5	13

*Includes (1) evidence-based or developed by RAND-modified Delphi process, (2) feasible, (3) detect variation in care.

QIs, quality indicators.

Discussion

A wide range of QIs have been developed and used in lung cancer. Most of these relate to surgery, which is only applicable to a small proportion of all patients with lung cancer. Only 10%–28% of all patients with lung cancer are managed with surgery in the USA and Europe, while utilisation studies show that optimally 61%–74% of all patients with lung cancer should be receiving radiotherapy and 73% receiving chemotherapy.^65–67 In addition, half of all patients with lung cancer present with incurable metastatic disease where palliative care is an important component of management. Yet, 87 of the identified QIs related to preoperative assessment or surgery compared with 30, 17 and 8 QIs specifically for systemic therapy, radiotherapy and palliative care, respectively. This is a disproportionate representation compared with the actual utilisation of treatment modalities in lung cancer. QIs should be relevant to the population and more work is needed in developing and implementing QIs in non-surgical therapies. Technical aspects of surgical management are examined in detail, while QIs for the technical aspects of radiotherapy are lacking and this is known to impact on lung cancer outcomes.⁶⁸ When considering the continuum of cancer care, there is also a gap where there are few QIs related to end-of-life and palliative care compared with diagnosis, staging and treatment.

The majority of QIs are related to process outcomes and appropriateness of care such as adhering to CPGs. Numerous indicators have also been developed in order to measure access to care, timeliness of care and delivery of coordinated or multidisciplinary care. Those indicators related to the technical aspects or safety and complications are largely surgical based. Modern radiotherapy clinical trials in lung cancer have shown that high quality in the technical treatment delivery of radiotherapy leads to lower rates of severe toxicity.⁶⁹ There is an apparent gap in measuring these domains in the delivery of both radiotherapy and systemic therapy. Both of these fields are rapidly evolving, and in particular new treatment standards in systemic therapy for lung cancer have been introduced in recent years. In this review, there were no QIs related to the use of immunotherapy, as this has only become standard practice recently. As treatment evolves with updated research and evidence, so do QIs need to be continually reassessed and implemented to reflect current clinical practice. There are also QIs that have been included which have been superseded by new evidence, investigations or procedures.

We identified 73 robust QIs that fulfilled characteristics of ideal QIs, that is, evidence-based, feasible and discriminating well. Of these, those that fulfilled the minimum ideal set of characteristics most were related to diagnosis and screening, treatment and supportive care. There were no or few indicators related to prevention, screening, pretreatment assessment or follow-up that met these criteria. Overall, although there are many published QIs related to lung cancer, only a relatively few number can be categorised as adhering to ideal characteristics of QIs (24%). Future development of QIs in lung cancer should focus on fulfilling ideal characteristics of QIs to ensure more useful measurement of care.

Previously developed QIs should be evaluated prior to being used in a real-world population that is to be measured. The selected indicators may fulfil the ideal characteristics but may be difficult or resource intensive to measure in real-world settings. These should be assessed with a practice test in the target population being evaluated with the available database, records and resources. For indicators to be used successfully to improve quality of care in a patient population, they should not only be measurable but also detect variation, have the potential to improve and be applicable to a meaningful proportion of the target population.⁴⁹ For example, in a clinical setting where the number of patients in the numerator is small, the QI is unlikely to detect variation in that population.

We found relatively few QIs that address patient-centred outcomes, such as assessment of quality-of-life aspects of care. Patient-reported outcomes and patient-reported experience measures have emerged as particularly important components of patient-centred care in cancer management.⁷⁰ These can identify and refocus care on otherwise unmet issues or patient needs that are impacting their care. In the cancer setting, they have shown to improve aspects in quality of care including health-related quality of life, treatment outcomes and patient satisfaction.⁵⁸ Future efforts should continue to focus on this important aspect of care.

This review is limited due to the search being confined to a single database, exclusion of studies not published in English and having a single reviewer screening and assessing the publications. A more rigorous systematic review was not performed due to time constraints. Additionally, as QIs change over time, some of the indicators that may have met ideal characteristics when published may no longer be relevant to contemporary practice. For this reason, our aim is to provide an overview of the types and characteristics of QIs in lung cancer and identify current gaps for future development, rather than endorse a set of useable indicators. The QIs we have published may also become obsolete with time and changes in management. Which QIs, when and how they should used also depends on the purpose of measurement and the target population, and is beyond the scope of this review. To further develop a more comprehensive set of QIs, we would suggest the QIs reported undergo a structured expert panel review process for the specific purpose that is intended. Our future work will focus on radiotherapy-related QIs to be developed with this method.

Conclusions

We found a large number of published QIs in lung cancer but they focused on relatively few areas not reflective of patterns of contemporary practice. We identified gaps in lung cancer QIs especially for systemic therapies, radiotherapy, palliative care and patient-reported outcomes. In order to comprehensively assess the care of patients with lung cancer, future efforts should focus on developing readily measurable QIs in these areas where there are limited QIs and also where current QIs do not comply with ideal characteristics.

Ethics statements

Patient consent for publication

Not required.