Critical pathways for continuous quality improvement: a multicentric analysis on the management of patients with lung cancer in Italian best performing hospitals

Summary

Introduction

Critical pathways (CPs) are effective change management tools used to improve quality in healthcare nationally implemented in Italy in 2015. This study aims to map the country’s state-of-the-art regarding the adoption of CPs and to verify the existence of factors that determine the success of their implementation and the relative entity of their impact, by analysing the management of Lung Cancer (LC) as a case-study.

Methods

Our methodology followed the SQUIRE guidelines for quality improvement reporting (2015). Starting from the 2017 ranking table published by the National Outcome Program, we selected and included in our sample all Italian hospitals who, according to Ministerial Decree n. 70/2015, met national quality threshold for LC treatment. To investigate regional-level and hospital-level factors believed to be responsible for the successful implementation of a CP, a Google Modules questionnaire was constructed and sent to the selected facilities; subsequently, a web-based research was carried out for missing data. Associations between variables were tested in STATA by means of correlation tests and a linear regression model.

Results

41 hospitals matched our inclusion criteria. Of these, 68% defined an internal Lung Cancer Critical Pathway (LCCP). Our results confirmed the presence of critical success factors that favour the correct implementation of a LCCP.

Conclusions

Notwithstanding the availability of CPs, their adoption in routine clinical practice still lacks consistency, suggesting the necessity to resort to digital solutions, to increment the level of regional commitment and workforce commitment and to reinforce quality standards monitoring.

Introduction

The Italian National Health Service (NHS) was set up in 1978 and founded on the principle of universal coverage and organized into national, regional and local layers of control. Particularly, in recent years, it has undergone a process of strong decentralization that has attributed a greater deal of power to regions [1].

In a country characterized by growing economic constraints, aggravated by the struggle of satisfying complex needs of an ageing population, with increasing comorbidities and chronicity, it is ever more important to focus on the six dimensions of quality (Safety, Effectiveness, Patient Centredness, Timeliness, Efficiency and Equity, [2]) when providing services within the NHS. Among the preferred tools for continuous quality improvement [3], institutions are resorting to Critical Pathways (CPs), defined by the European Pathway Association as: “Complex intervention(s) and methodology for the mutual decision making and organization of predictable care processes for a well-defined group of patients during a well-defined period”. Also known in literature as clinical pathways, care maps or integrated care pathways [4], these clinical management tool were first mentioned in 1985 by Zander et al. (New England Medical Center) [5, 6] as a methodology to balance costs and quality of delivered services; subsequently, their use was spread all over the world in the early 90s when they were recognized the capacity of assuring continuity of care [7, 8].

In Italy, Ministerial Decree (MD) 70/2015 [9] nationally set patient management through Critical Pathways (CPs) as a national requisite for hospital accreditation to the NHS and, subsequently, reinforced their accountability role through Law 24/2017 (Gelli Law) [10].

Our research question stems from the doubt that, despite the efforts of Italian policymakers to issue laws that enhance their implementation at national, regional and local levels, the dissemination and actual implementation of CPs may still be inconsistent and fragmented in the country.

Several authors [11-15] have investigated the effects of CPs on care outcomes (i.e. length of stay, appropriateness of setting, infections and readmission rates), demonstrating improvements especially for acute conditions requiring hospitalization, however there is evidence that specific factors are responsible for the successful implementation of a CP, meaning that its sole existence does not, by itself, assure an effective improvement of patient management. Rotter’s 2010 Cochrane review [16] pointed out the following determinants:

• the presence of multidisciplinary teams;

• the resort to evidence based tools;

• the resort to ICT in support of CPs;

• the establishment of audit and feedback mechanisms;

• the conduction of Gap analyses.

Thus, if there is a lack of these supplementary critical success factors, the adoption of CPs may be inhibited from providing their full potential of benefits.

With the aim of detecting the extent to which CPs have been adopted in Italy and, consequently, identify those organizational requirements that have made such CPs successful or those still requiring improvements and additional efforts, we have analysed the state-of-the-art of the application of CPs in the treatment of LC, which is known as Italy’s Big Killer therefore chosen among all pathologies due to the significant burden it has on the country. It is, in fact, the 4^th most prevalent type of cancer in Italy and is responsible for 19% of all deaths due to cancer, making it the first cause of oncological death in the country [17].

Methods

Our methodology followed the 2015 SQUIRE guidelines for quality improvement reporting (Tab. I) [18].

Tab. I.

SQUIRE 2.0 checklist, September 15^th, 2015.

Text section and item name	Section or item description	Page number
1. Title	Indicate that the manuscript concerns an initiative to improve healthcare (broadly defined to include the quality, safety, effectiveness, patient-centeredness, timeliness, cost, efficiency, and equity of healthcare)	1
2. Abstract	Provide adequate information to aid in searching and indexing Summarize all key information from various sections of the text using the abstract format of the intended publication or a structured summary such as: background, local problem, methods, interventions, results, conclusions	2
Introduction	Why did you start?
3. Problem Description	Nature and significance of the local problem	3
4. Available knowledge	Summary of what is currently known about the problem, including relevant previous studies	3
5. Rationale	Informal or formal frameworks, models, concepts, and/or theories used to explain the problem, any reasons or assumptions that were used to develop the intervention(s), and reasons why the intervention(s) was expected to work	3
6. Specific aims	Purpose of the project and of this report	3
Methods	What did you do?
7. Context	Contextual elements considered important at the outset of introducing the intervention(s)	3
8. Intervention(s)	Description of the intervention(s) in sufficient detail that others could reproduce it Specifics of the team involved in the work	N/A – We aim to analyse the state of multiple interventions. Research methods are reproducible and detailed in page 6 The team is composed of the members of the Clinical Pathway and Outcome Evaluation Unit in FPG-IRCCS
9. Study of the Intervention(s)	Approach chosen for assessing the impact of the intervention(s) Approach used to establish whether the observed outcomes were due to the intervention(s).	9 9
10. Measures	Measures chosen for studying processes and outcomes of the intervention(s), including rationale for choosing them, their operational definitions, and their validity and reliability Description of the approach to the ongoing assessment of contextual elements that contributed to the success, failure, efficiency, and cost Methods employed for assessing completeness and accuracy of data	6 9 9
11. Analysis	Qualitative and quantitative methods used to draw inferences from the data Methods for understanding variation within the data, including the effects of time as a variable	9 N/A
12. Ethical considerations	Ethical aspects of implementing and studying the intervention(s) and how they were addressed, including, but not limited to, formal ethics review and potential conflict(s) of interest	16
Results	What did you find?
13. Results	Initial steps of the intervention(s) and their evolution over time (e.g., time-line diagram, flow chart, or table), including modifications made to the intervention during the project Details of the process measures and outcome Contextual elements that interacted with the intervention(s) Observed associations between outcomes, interventions, and relevant contextual elements Unintended consequences such as unexpected benefits, problems, failures, or costs associated with the intervention(s). Details about missing data	N/A 10 10 10 N/A Web-based research allowed us to have no missing data
Discussion	What does it mean?
14. Summary	Key findings, including relevance to the rationale and specific aims Particular strengths of the project	14 14
15. Interpretation	Nature of the association between the intervention(s) and the outcomes Comparison of results with findings from other publications Impact of the project on people and systems Reasons for any differences between observed and anticipated outcomes, including the influence of context Costs and strategic trade-offs, including opportunity costs	14 14 14 14 N/A
16. Limitations	Limits to the generalizability of the work Factors that might have limited internal validity such as confounding, bias, or imprecision in the design, methods, measurement, or analysis Efforts made to minimize and adjust for limitations	15 15 15
Conclusions
17. Other information	Usefulness of the work Sustainability Potential for spread to other contexts Implications for practice and for further study in the field Suggested next steps	15 15 15 15 15
18. Funding	Sources of funding that supported this work. Role, if any, of the funding organization in the design, implementation, interpretation, and reporting	16

Initially, we consulted the 2017 league tables released by the National Outcome Program (Piano Nazionale Esiti, PNE) which lists all public and private accredited hospitals in Italy that perform surgery for malignant LC. We then applied the national quality standard set by MD n. 70/2015 [14] (according to which the minimum standard for quality is the performance of 100 Lung Cancer surgical interventions per year) as inclusion criteria to select those facilities to submit to our analysis.

Starting from Rotter’s 5 requisites [16], an electronic questionnaire was formulated and validated by means of the Delphi process [19, 20] conducted as follows:

1. authors scanned available international literature to capture factors believed to be determinant for the successful implementation of a CP [16];

2. face-to-face meetings aligning authors, agreeing upon a set of items to guide the development of the questionnaire;

3. a draft questionnaire was created based on Step 1 and 2;

4. two rounds of an on-line survey were completed to reach anonymous consensus; the first Delphi round sought to extract relevant domains to assess the presence of critical success factors for the implementation of a CP. Subsequently, the second round aimed at screening relevant items within the domains extracted from the results of Round 1;

5. all authors participated in a final group discussion to validate the proposed set of items and domains.

Two relevant domains were included: one regional and one at hospital level. Table II shows items included for each of these domains, resulting in the inclusion of 19 questions.

Tab. II.

Questionnaire – Google Modules.

Dimension: Regional level characteristics
General information	In which Region is your hospital located?
Information related to the presence of a Clinical Pathway and in particular a Lung Cancer CP	Is there in your Region a law about the implementation of Clinical Pathways?
	Is there a regional office dedicated to clinical governance issues?
	Is there a specific CP dedicated to Lung Cancer?
	Is there an oncological network for patients with Lung Cancer?
	Are there any citizen associations that take part in the development of CPs?
	Is your hospital taking part in the regional monitoring system?
Dimension: Hospital level characteristics
General characteristics	Type of facility (i.e.: IRCCS, RTH, ASL, etc.)
	Ownership (public or private)
	ISO or JCI Certification
	Use of Electronic Health Record
Information related to the presence of Clinical Pathways and, in particular, to Lung Cancer CP	Does your organizational structure have a specific hospital unit dedicated to the development of CPs?
	Has a specific LCCP been implemented?
	Have specific indicators been settled for this CP?
	Specify what the nature of the defined indicators is, which are the standards used for their calculation and to what level of assistance they are referred
	Is there a Tumor Board?
	Specify by which kind of Specialists the Tumor Board is composed
	Is there a monitoring system?
	Is there an auditing system?

The formulated electronic survey was sent out through Google Modules to offices or units of the included facilities deemed to be responsible for the Clinical Governance and, hence, the management and the implementation of Clinical Pathways. Following, from June 2019 to September 2019, a web-based research on the included facilities was carried out to further investigate the regional-level and hospital-level variables (Tab. III) for which the response rate resulting from the electronic questionnaire was low, to improve the reliability of our findings.

Tab. III.

List of investigated variables.

Regional variables	Hospital variables
Presence of regional law on CPs Presence of regional functions with responsibilities on CPs Presence of regional cancer network Presence of regional CP on lung cancer Presence of citizens’ associations Belonging to interregional or regional monitoring systems	Volume of Surgical Interventions for LC Mortality at 30 days from hospitalization for LC Number of hospital beds Number of Oncological Beds Ownership Presence of specific units for CPs Presence of internal CP for Lung Cancer Presence of indicators Presence of the Tumor Board Presence of the Audit and monitoring systems Electronic Health Records JCI accreditation/ISO 9001 certification

Data was gathered in a comprehensive Table of Contents assembled by two authors in Microsoft Excel and divided into regional level variables and hospital level variables.

It was then analysed and tested in STATA (version 14). The association among variables was then tested through a Pearson correlation (ρ) and a Tetrachoric correlation (rter) [21].

Finally, a linear regression model was set up to test the impact of the covariates on the dependent variable (i.e.: the implementation of an Internal Lung Cancer Critical Pathway (LCCP)).

The linear regression model is the following:

In this Equation, our dependent variable Y_i constitutes the presence of an Internal LCCP, β₀ is the constant term, β₁ represents the coefficients given by the test, X_i stands for the studied covariates and ε_i is the error term.

Results

Among 194 institutions listed on the PNE league table, 41 healthcare facilities met our inclusion criteria (at least 100 surgical interventions for LC per year [14]) and were utilized for our study. Of these, 14% responded to the Google Modules questionnaire and remaining data was obtained through the complementary web-based research.

Results have been analysed and presented according to the two main domains (regional level and hospital level).

DESCRIPTIVE RESULTS

The first analysis investigated general characteristics of the facilities, in order to map the context of interest. Comprehensively, 36% of the analysed facilities are Research Teaching Hospitals (RTHs), 30% are independent public hospital trusts (AOs), 22% are Research Hospitals (IRCCSs), 17% are private practices and the remaining 12% are hospitals administered by their Local Health Units (LHUs).

Overall, the facilities are distributed in 14 Italian regions (Fig. 1), therefore the descriptive statistics were observed grouping regions into three main geographical areas of Italy as follows:

Fig. 1.

Facilities distribution in Italy.

1. North: Lombardy, Veneto, Liguria, Piedmont, Emilia-Romagna and Friuli Venezia Giulia;

2. Centre: Lazio, Tuscany, Marche and Umbria;

3. South: Campania, Puglia, Abruzzo, Sicily.

   1. Regional level

   2. For what concerns regional level observations, results are synthesised in Table II and show that 71% of the facilities are covered by a Regional Law on CP with equal proportions in the North and in the South (both 29%). Half of the regions (50%), have structured a regional LCCP and are mostly located in the North (29%), followed by the South (14%) and lastly, the Centre (7%).

   3. We also observed that all regions with a specific law on CPs are equipped with dedicated regional office units and cancer networks, however only in 43% of regions are citizen associations present and monitoring systems adopted.

   4. Hospital level.

   5. At hospital level, our analysis showed that 68% of the facilities have implemented an internal LCCP. Almost all facilities are equipped with dedicated staff units (93%), Tumor Boards (78%) and have formulated indicators (68%). However, percentages decrease to below half when it comes to Monitoring and Auditing systems (44% and 39%, respectively), presence of Electronic Health Records (22%) and quality certifications (JCI 15%, ISO 20%), intended as those certifications or accreditations that prove that a facility has reached a standard level of quality which is known and recognized by international and national bodies.

STATISTICAL RESULTS

We statistically analysed whether these variables are correlated and whether they have an impact on the successful implementation of an internal LCCP.

CORRELATION

To identify associations between variables, we resorted to the Tetrachoric correlation test for dichotomous variables and to Pearson’s correlation test for all the others, as shown in Figure 2.

Fig. 2.

Pearson’s and Tetrachoric correlation test results.

1. Regional level

   At regional level, we can underline first of all that in Regions that have issued a Regional Law on CPs it is more likely to find office units dedicated to the management of care maps, cancer networks and Regional LCCP.

   Secondly, the presence of a Regional LCCP increases the likelihood of finding citizens’ associations involved in the creation of the CP as well as dedicated regional office units.

   Third, hospital facilities located in Regions with their own monitoring systems are more likely to be aware of their own performance and additionally show larger volumes of surgical interventions compared to those located in regions lacking regional monitoring systems. Mostly located in the centre of Italy, they are also more likely to use electronic health records, to have a JCI accreditation.

2. Hospital level

   At hospital level, the adoption of an Internal LCCP shows correlation with geographical location, as most of the equipped facilities are located in the North, and with the presence of certain organizational factors that enhance its implementation and spread, such as CP office units, sets of specific indicators, an internal monitoring system and Tumour Boards.

   Secondly, the presence of an internal monitoring system increases the likelihood of finding a structured auditing system and, if both of these factors coexist, this also implies that the organization is equipped with a dedicated set of indicators, office staff units and multidisciplinary teams working together to improve hospital performance.

LINEAR REGRESSION

The linear regression model allowed us to identify which critical success factors enhance the probability of finding a LCCP at hospital level. The most significant results are presented in Table IV.

Tab. IV.

Estimated coefficients after linear regression model.

Variables	Coef.	t	Variables	Coef.	t
Hospital variables			Regional variables
Use of Indicators	0.1176	0.70	Regional Units	0.2063*	1.82
Tumour Board	0.7614***	3.60	Adoption of Reg CP	-0.9349***	-7.69
Monitoring System	0.0801	0.65	Regional LCCP	0.1076	-1.22
Auditing System	0.0399	0.46	Cancer Network	-0.2404	-1.15
EHR	-0.0355	-0.39	Reg. Mon. System	0.0669	-0.60
JCI accreditation	-0.2890	-1.14	Regional Law on CPs	0.1185	0.65
ISO certification	0.1715	1.72
Staff Units	0.0042	0.03
Citizens’ Associations	-0.1557	-1.35
Ownership	-0.3118	-0.26
Interventions	-0.0001	-0.35
Hospital beds	-0.0000	-0.82
Adj. Mortality Rate	-0.0757	-1.48
Oncological beds	-0.0000	-0.02
Type of facility	-0.0016	-0.05
Belonging Area	0.0839	0.86
Significance Level: *** 1%; ** 5%; * 10%.

The presence of a Tumour Board, the collocation in a region which has a Clinical governance dedicated units and the adoption, at hospital level, of a Regional LCCP have, indeed, a statistical effect on the presence of an internal LCCP, increasing the probability of finding a CP within a facility.

Although not statistically significant, it is also interesting to point out that there are institutional factors that, similarly, promote the presence of an internal integrated CP such as regional monitoring systems and Regional laws on CPs and the presence, within the Region, of a Regional LCCP. Hospital level variables such as the presence of indicators, of internal monitoring and auditing system, of dedicated staff units, the geographical area and presence of an ISO 9001 certification are all factors that seem to favour the successful implementation of an internal pathway.

Discussion

This study aimed to detect the extent to which CPs have been adopted in Italy and, consequently, identify those organizational requirements that have made such CPs successful alongside those still requiring improvements and additional efforts. To do this, we have analysed the state-of-the-art of the application of CPs in the treatment of LC, known as Italy’s Big Killer and therefore chosen among all pathologies due to the significant burden it has on the country. It is, in fact, the 4^th most prevalent type of cancer in Italy and is responsible for 19% of all deaths due to cancer, making it the first cause of oncological death in the country [17].

In literature there is evidence that implementation of a CP alone is not sufficient to assure its successful use as there is the need to set up ulterior factors that are responsible for its successful utilization. For this study, the success factors we chose to focus our attention on Rotter’s five criteria [16] that make a path really effective (multidisciplinary teams, evidence based tools, ICT solutions, audit and feedback systems and gap analyses).

The same evidence emerged from BMJ article by Fulop and Ramsay (2019) [22]. They conducted a study on the US organizations with the highest and lowest 5% risk-standardized mortality rates for acute myocardial infarction in 2017 and they discovered that for both the categories of hospitals, the presence of protocols and formalized processes alone do not report associations with high or low performances, hence the differences in mortality rates are imputable to other factors such as different organizational approaches.

Starting from these evidences found in literature, and assuming that specific critical success factors for the consolidation and effectiveness [16] of CPs are needed, we explored whether these were present in Italian best performing hospitals and our findings are indeed in line with what is suggested in literature. At both Regional Level and Hospital level there is awareness about the importance of CPs, and efforts are being made to exploit their full potential through the adoption of specific organizational factors, however some areas are still lacking attention and require additional efforts.

Overall, LCCPs are not nationally disseminated or regularly implemented in Italy as, among the selected facilities, only 68% have formalized a Hospital LCCP. Among the others, 10% adopting the Regional CP while the rest to not manage the pathology through a pathway.

Our statistical tests show the correlation between the adoption of an internal LCCPs and the presence of the five requisites suggested by Rotter (multidisciplinary teams, evidence based tools, ICT, audit and feedback systems and gap analyses) [16] that determine the successful utilization of a CP. Tumour Boards and Evidence Based Tools, both resulted as factors that enhance the success of internal CPs and are largely spread among Italian best performing facilities: Tumour Boards are present in 78% of them and all the Hospital CPs adopt and apply guidelines and best practices. Indicators, on the other hand, are formalized in 68% of facilities however structured and developed monitoring and auditing systems are rarely present (44% and 39%, respectively). Similarly, Electronic Health Records and Quality certifications were detected in 22% and 17.5% of facilities. It appears that, although the relevance of such mechanisms is recognized nationwide, they are not uniformly spread or well developed. By looking at the heterogeneity of the percentages, it appears, indeed, that only some of these requisites have been given primary importance (Tumour Boards, evidence based tools and indicators). The remaining variables (monitoring and auditing systems, Electronic Health Records and gap analyses) are strictly linked to the availability of technological solutions and ICT. Literature suggests that the adoption of technology plays an important role in the management and implementation of CPs [16]; conducting a survey among more than 40 Italian facilities, it was found that the absence of indicators, monitoring and auditing systems or Electronic Health Records in the Italian context may be imputable to the lack of digital literacy [23].

Alongside what has up to now been discussed, our research also highlighted the presence of two other variables may affect the success of a LCCP: the level of Regional commitment, especially because both the geographical area and the characteristics of the Region of belonging impact the behaviour of healthcare organizations, and the presence of quality certifications (JCI, ISO etc) which, indeed, indicate which organizations pay more attention to performance outcome and, thus, are more likely to resort to Clinical Governance tools such as CPs.

LIMITATIONS OF OUR STUDY AND IMPLICATIONS FOR FUTURE RESEARCH

Obtaining data online through questionnaires and a web-based research lead to some limitations in our study. First of all, the sent questionnaires received a response rate of 14% which is below the literature standard of 30% for emails and online surveys [24]. Collecting information for all the facilities through their websites implied that the desired information was not always available or standardized and required authors to apply non uniform methods to assess the level of accuracy and up-to-dateness. Furthermore, notwithstanding the fact that a logistic regression is the model that better fits with a binary dependent variable in this case the model showed co-linearity among the variables.

Conclusions

In conclusion, we can state that hospitals included in our analysis have almost entirely fulfilled Rotter’s Criteria list, especially when it comes to multidisciplinary teams and evidence based tools. Our analysis also allows us to identify additional criteria that could be playing a role in the successful implementation of a CP within the Italian healthcare system. In fact, collocation of the healthcare facilities in a Northern region and quality certifications seem to be factors promoting the likelihood of finding a LCCP. Depending on the geographical position, in particular, sensitivity and awareness of the Regional Governments varies thus, each hospital in the country should be spreading the urgency to control performances and establish indicators, monitoring and auditing systems through CPs.

In the light of the above, it seems that the most urgent gap to bridge is the one deriving from the lack of digital literacy. One of the most successful factors that enhance the use of CPs is full knowledge on available ICT tools, especially those designed for the gathering and measurement of performance measurement indicators and create a monitoring and auditing system. Enhancing and improving the awareness of employees about the importance of a shift to digitalization and of resorting to Big Data and ICT solutions systems, should be set as priority when selecting in which direction to address efforts, especially because most of the above-listed determinants are related to ICT solutions which imply the utilization of large amounts of data, an area of expertise that still requires substantial innovation in the country.

Secondly, Regional Governments should establish regional monitoring systems for performance measurement of their facilities, to stimulate facilities and regions to be competitive and strive to obtain best ranking on the league tables.

Lastly, the National Government should request a quality certification from each facility to be sure that they are pursuing continuous quality improvement.

To truly assist Policy formulation decision making, future investigations that build on to our findings could be similarly conducted for diseases with a comparable level of complexity (i.e. other cancers) or for other countries which have founded their NHSs on the principle of Universal Coverage.

Acknowledgments

Authors thank Fabio Lo Giudice, Alessandro Maria Lomeo, Ilaria Panzini, Marco Scarci and Marcello Tiseo for their collaboration.

Conflict of interest statement

Authors declare no conflict of interest.

Ethical considerations

No relevant ethical issues stemmed from this study.

Authors’ contributions

A.G.B. conceived the study, devised the project and supervised the research. A.G.B. and C.A. were in charge of overall direction and planning. The theoretical framework was developed by all authors combined whereas the data gathering Table of Contents was developed by A.M., F.W. and G.G. and then validated and filled in by all authors. A.P. and C.A. fabricated the sample. M.C.C. and C.A. designed the model and the computational framework and M.C.C. derived the statistical models and analysed the data. Analytical methods and results were verified by A.G.B. and all authors discussed the results, providing critical feedback and helping shape the research. M.C.C. took the lead in writing the manuscript with support from A.M. and G.S.