Lean interventions in healthcare: do they actually work? A systematic literature review

Abstract

Purpose

Lean is a widely used quality improvement methodology initially developed and used in the automotive and manufacturing industries but recently expanded to the healthcare sector. This systematic literature review seeks to independently assess the effect of Lean or Lean interventions on worker and patient satisfaction, health and process outcomes, and financial costs.

Data sources

We conducted a systematic literature review of Medline, PubMed, Cochrane Library, CINAHL, Web of Science, ABI/Inform, ERIC, EMBASE and SCOPUS.

Study selection

Peer reviewed articles were included if they examined a Lean intervention and included quantitative data. Methodological quality was assessed using validated critical appraisal checklists. Publically available data collected by the Saskatchewan Health Quality Council and the Saskatchewan Union of Nurses were also analysed and reported separately.

Data extraction

Data on design, methods, interventions and key outcomes were extracted and collated.

Results of data synthesis

Our electronic search identified 22 articles that passed methodological quality review. Among the accepted studies, 4 were exclusively concerned with health outcomes, 3 included both health and process outcomes and 15 included process outcomes. Our study found that Lean interventions have: (i) no statistically significant association with patient satisfaction and health outcomes; (ii) a negative association with financial costs and worker satisfaction and (iii) potential, yet inconsistent, benefits on process outcomes like patient flow and safety.

Conclusion

While some may strongly believe that Lean interventions lead to quality improvements in healthcare, the evidence to date simply does not support this claim. More rigorous, higher quality and better conducted scientific research is required to definitively ascertain the impact and effectiveness of Lean in healthcare settings.

Introduction

Globally, healthcare systems are at a cross roads. Many political and healthcare leaders, and in fact the public itself is calling for, if not demanding, the redesign of healthcare delivery. The concern is fuelled by ever increasing costs and high expectations, while at the same time having surprisingly low rates of patient adherence to care and high rates of adverse events [1]. In response, many jurisdictions have attempted to introduce standardized protocols like Lean.

Lean is a widely used quality improvement methodology. Lean thinking was first developed in the automotive and manufacturing industries but it has recently expanded to the healthcare sector. Lean thinking begins with identifying and ‘removing waste’ in order to ‘add value’ to the customer or patient [2]. The Lean Enterprise Institute articulates five main principles of Lean: specify value from the standpoint of the customer, identify all the steps in the value stream and eliminate steps that do not create value, make the steps flow smoothly toward the customer, let customers pull value from the next upstream activity and begin the process again until a state of perfection is reached [3].

The introduction of these principles placed ‘customer value’ and ‘removing waste’ at the centre of Lean thinking. In this manner, the process is essentially driven by ‘what customers want’ and then organizational steps are taken to define which activities are considered to be ‘value-adding’ as opposed to ‘non-value adding’. ‘Value adding’ activities are encouraged because they directly contribute to creating a product or service a customer wants. On the other hand, ‘non-value adding’ activities are considered a waste and need to be removed or avoided [4].

To date, there have been a limited number of reviews of Lean or Lean interventions in healthcare. One of the reviews started with 207 articles under consideration. However, when the authors applied their inclusion criteria of only accepting papers that were published in peer review journals and studies that had quantifiable data available, it left them with merely 19 papers (9.2%) for critical appraisal [5].

Among the papers accepted, it was noted that the vast majority of studies had methodological limitations that undermined the validity of the results. These limitations included weak study designs, lack of statistical analysis, inappropriate statistical assumptions, inappropriate analysis, failure to rule out alternative hypotheses, no adjustment for confounding, selection bias and lack of control groups. The studies also did not review long-term organizational change, long-term impact or the independent effect of Lean while controlling for other organizational or staffing changes occurring at the same time [5]. Although this review was well-conducted, it was not a systematic literature review and it did not include a quality control checklist.

In North America, there are many examples of Lean healthcare interventions but the largest Lean transformation in the world was attempted in the province of Saskatchewan, Canada [6]. The Health Quality Council (HQC) of Saskatchewan concludes on its website that Lean increases patient safety by eliminating errors, increases patient satisfaction, reduces cost and improves patient health outcomes [7].

On the surface, Lean thinking seems to be an approach that generates positive results [8]. Yet, its application in healthcare has been controversial and its effectiveness questioned. As such, the purpose of this systematic literature review is to independently assess the effect of Lean thinking and Lean interventions on worker and patient satisfaction, health and process outcomes and financial costs.

Methods

We conducted an extensive systematic literature review on the following electronic databases: Medline, PubMed, Cochrane Library, CINAHL, Web of Science, ABI/Inform, ERIC, EMBASE and SCOPUS.

Searches were carried out using the following keywords: Lean Production System, Lean enterprise, Lean manufacturing, Virginia Mason Production System, Toyota Production System, Just in time production, Kaizen, HoshinKanri, Lean method, Lean thinking, Lean intervention, Lean healthcare, Lean principles, Lean process, Muda and Healthcare.

Peer-reviewed articles

Articles had to satisfy the following inclusion criteria to be considered: published in English, publicly available, peer reviewed, examined a Lean intervention and included quantitative data. These liberal criteria allowed the inclusion of a wide variety of relevant articles in our study. However, it also served as a means to exclude news reports, blog commentary, informational/promotional pieces and general ‘feel good’ success stories that lacked the necessary quantitative data to be able to critically judge the information presented.

The identification and approval of studies was carried out in three steps. First, the authors examined titles and abstracts to remove duplicates. Second, two of the authors (C.N. and M.L.) reviewed the full-text articles for relevance with regard to the field of healthcare and conformity to the inclusion criteria. Third, methodological quality was assessed by using validated critical appraisal checklists. The diffusion of innovations in health service checklists helped the authors assess the baseline comparability of the groups in each study, the research design, outcome measures and potential sources of bias. They were originally modelled after the Cochrane Effective Practice and Organization of Care Group for interventions in service delivery and organization [9]. Studies that scored >50% on the quality checklist were accepted (i.e. satisfied 6 or more out of 11 questions for before and after studies). Any disagreement between the two authors (C.N. and M.L.) was resolved by additional review and, if required, with a tie-breaking vote by the third author (J.M).

Grey literature

As mentioned, the largest Lean healthcare transformation in the world was attempted in the province of Saskatchewan, Canada [6]. The HQC has been surveying tens of thousands of patients over the years about their experiences in Saskatchewan hospitals. For the purposes of this systematic review, February 2012 was used as the cut-off point for the evaluation of pre- and post-Lean data as it coincided with the official date of the signed provincial contract with a Lean consultant firm [10]. A 26-month period was used to collect and analyse data on a monthly basis before Lean implementation (December 2009 to January 2012) and after Lean implementation (February 2012 to March 2014). This high quality data collected by certified Lean professionals have sample sizes ranging from 17 698 to 92 127 patients with a response rate of ∼51% and it is publicly available on a web site [11]. Additionally, the largest healthcare union or association in the province, the Saskatchewan Union of Nurses (SUN), contracted an external professional polling company to randomly survey 1500 nurses about their Lean experience in 2014 [12]. All 1500 nurses contacted, participated in the telephone survey.

Results

We identified a total of 1056 peer-reviewed articles of which 164 were removed as duplicates, 768 were removed due to lack of relevance to healthcare and 76 were removed because they did not meet the inclusion criteria. Among the 48 articles that were assessed for methodological quality, 22 articles passed [13–34] and 26 articles failed the checklist review [35–60] (Fig. 1 and Table 1). The original two reviewers (C.N. and M.L.) independently assessed and agreed on 43 studies with a tie breaking vote required by the third reviewer (J.M.) on five out of the 48 studies. Once finalized, the data from the included studies was pooled and summarized and confidence intervals for rate ratios were calculated with an established software application (SPSS 22.0).

Figure 1.

Prisma flow diagram of the included studies.

Table 1.

Detailed list of eligible peer review articles from the literature search

Articles that passed methodology review
First author's last name, year of publication, country where study was done	Study design	Number of participants	Location of intervention (ex. Emergency department)	Intervention	Intervention goal	Type of outcome	Quality scores	Outcome rate ratio and 95% CI
Health outcome studies
Jha, 2012, USA [13]	Retrospective cohort	6 000 000	Hospital	Pay for performance	Reduce 30 day mortality rate	Health outcome	9/11 Pass	30 day mortality rate 0.08 (−0.30 to 0.46)
McCulloch, 2010, UK [14]	Interrupted time series	2083	Emergency surgery ward	PDCA	Reduced risk of care related harm	Health outcome	6/11 Pass	Adverse events 0.91 (0.72–1.16)
Muder, 2008, USA [15]	Pre-/post-test	215	ICU and a surgical unit	Hand hygiene, contact precautions, active surveillance (TPS)	Reduce incidence of MRSA	Health outcome	7/11 Pass	MRSA infections per 1000 patient days 2.47 (1.87–3.27)
Ellingson, 2011, USA [16]	Pre-/post- test	109	Veteran affairs hospital surgical ward	Systems and behaviour change to increase adherence to infection control precautions	Reduce in MRSA incidence rates	Health outcome	7/11 Pass	MRSA incidence rate ratio 0.99 (0.98–1.01)
Process outcome studies
Murrell, 2011, USA [17]	Pre-/post-test	64 907	Emergency department	Rapid triage and treatment	ED length of stay and physician wait time	Process outcome	7/11 Pass	Unable to compute RR Length of stay reduced from 4.2 (4.2–4.3) to 3.6 (3.6–3.7) hours Physician start time reduced from 62.2 (61.5–63.0) to 41.9 (41.5–42.4) minutes
Kelly, 2007, Australia [18]	Pre-/post-test	63 085	Emergency department	Streaming of patients from triage, reallocation of medical and nursing staff (VSM)	Reduce number of patients who leave without being seen	Process outcome	8/11 Pass	Left without being seen 0.99 (0.92–1.08)
Naik, 2012, USA [19]	Pre-/post-test	22,527	Emergency department	Identify and eliminate areas of waste	Emergency wait time	Process outcome	6/11 Pass	Unable to compute RR Wait time reduced from 4.6 (4.5–4.9) to 4.0 (3.7–4.1) hours
Simons F, 2014, Netherlands [20]	Pre-/post-test	8,009	Operating room of University medical centre	DMAIC using A3 intervention	Door movements in the operating room	Process outcome	6/11 Pass	Unable to compute RR Door movements reduced by 78% from an average of between 15 and 20 times per hour during surgery to 4 times per hour
Burkitt, 2009, USA [21]	Retrospective pre-/post	2,550	Veteran affairs surgical center	Staff training on hand hygiene, systematic culturing of all admissions, patient isolation	Increase appropriateness of perioperative antibiotics and reduction in length of stay	Process outcomes	7/11 Pass	Length of stay 0.91 (0.76–1.08)
Weaver, 2013, USA [22]	Pre-/post-test	2444	Mental health clinic	Identify and eliminate areas of waste (TPS)	Improving number who attend first appointment, reduce wait for appointment	Process outcome	9/11 Pass	Number who attended first appointment 1.0 (1.0–1.0) Wait reduced from 11 days to 8 days
LaGanga, 2011, USA [23]	Pre-/post-test	1726	Mental health center	Remove over booking	Increase capacity to admit new patients and reduce no-shows	Process outcome	7/11 Pass	No shows 1.13 (1.03–1.23)
van Vliet, 2010, Netherlands [24]	Pre-/post-test	1207	Eye hospital	Identify and eliminate areas of waste	Reduce patient visits	Process outcome	9/11 Pass	Patient visits 1.84 (1.33–2.56)
Martin, 2013, UK [25]	Pre-/post-test	500	Radiology department	Value stream analysis (VSM)	Reduce patient journey time	Process outcome	6/11 Pass	Unable to compute. No pre and post raw data—only percentage changes were given
White, 2014, Ireland [26]	Cross-sectional study	338	Hospital	Implementation of productive ward program	Improve work engagement	Process outcome	7/11 Pass	Overall work engagement score1.06 (0.96–1.18)
Ulhassan, 2014, Sweden [27]	Pre-/post-test	263	Emergency department and two cardiology wards	Identify and eliminate areas of waste (DMAIC)	Improve teamwork	Process outcome	8/11 Pass	Overall inclusion 1.02 (0.74–1.42) Overall trust 1.04 (0.79–1.38) Overall productivity 1.0 (1.0–1.0)
Collar, 2012, USA [28]	Pre-/post-test	234	Otolaryngology operating room	Identify and eliminate areas of waste (DMAIC)	Improve efficiency and workflow	Process outcome	7/11 Pass	Unable to compute due to data not being provided. Turn-over time reduced from 38.4 min to 29 min
Blackmore, 2013, USA [29]	Retrospective cohort	200	Breast clinic	Identify and eliminate areas of waste	Improve timeliness of diagnosis and reduce surgical consults	Process outcome	6/11 Pass	Reduced surgical consults 4.60 (1.82–11.62)
Simons P, 2014, Netherlands [30]	Pre-/post-test	167	Radiotherapy department	Implementation of a standard operating procedure	Improve compliance to patient safety tasks	Process outcome	8/11 Pass	Overall compliance 0.96 (0.58–1.58)
Mazzocato, 2012, Sweden [31]	Case study	156	Accident and Emergency department	Identify and eliminate areas of waste, system restructuring	Increase number of patients seen and discharged within four hours	Process outcome	10/13 Pass	Discharged within four hours 1.07 (0.92–1.26)
Health and process outcome studies
Vermeulen, 2014, Canada [32]	Pre-/post-test Only study with control group	6 845 185	Emergency department	Training and system redesign	Left without being seen, discharged within 48 h, readmitted within 72 h, died within 7 days of discharge	Process and health outcome	8/11 Pass	In comparison to control group: Left without being seen 1.05 (0.77–1.43) Discharged within 48 h 1.19 (0.72–1.98) Readmitted within 72 h of discharge 1.0 (1.0–1.0) Died within 7 days of discharge 1.03 (0.84–1.26)
Yousri, 2011, UK [33]	Pre-/post-test	608	Hospital	Identify and eliminate areas of waste	Overall mortality, 30 day mortality, door to theatre time, admission to a trauma ward	Health and process outcome	6/11 Pass	30 day mortality rate 1.71 (0.70–4.17) Door to theatre time within 24 h 1.17 (0.86–1.60) Admission to trauma bed 1.03 (0.90–1.20)
Ford, 2012, USA [34]	Pre-/post-test	219	Emergency department	Value stream analysis (VSM)	Reduce time dependant stroke care and stroke mimic	Process outcome and health outcome	7/11 Pass	Percent of patients with DNT < 60 min 1.50 (1.21–1.86) Stroke mimic 0.64 (0.26–1.58)
Articles that failed methodology review
First author's last name, year of publication, country where study was done	Study design	Number of participants	Location of intervention (ex. Emergency department)	Intervention	Intervention goal	Type of outcome	Quality scores	Major methodological drawbacks
Health outcome studies
Ulhassan, 2013, Sweden [35]	Pre-/post-test	4399	Cardiology department	Changes to work structure and process	Improve patient care	Health outcome	4/11 Fail	Intervention could not be said to be independent of other changes over time No formal statistical test was used Outcomes were not blinded
Wang, 2014, China [36]	Pre-/post-test	622	Nephrology department	Training, treatment of high risk patients, specialized outpatient clinic	Incidence of peritonitis	Health outcome	4/11 Fail	Intervention could not be said to be independent of other changes over time Primary outcome measure was not reliable Data did not cover most episodes of intervention at follow-up
Process outcome studies
Wong, 2012, USA [37]	Pre-/post-test	234 616	Cytology laboratory	New imaging system, workflow redesign	Turnaround time, productivity and screening quality	Process outcome	4/11 Fail	Intervention could not be said to be independent of other changes over time Primary outcome measure was not reliable Outcomes measures were not blinded
Lodge, 2008, UK [38]	Post-test	9297	Division of diagnostics and clinical support	Intranet based waiting list for radiology services	Reduce radiology wait times	Process outcome	3/11 Fail	Intervention could not be said to be independent of other changes over time Insufficient data points for statistical analysis No formal statistical analysis was done
Willoughby, 2010, Canada [39]	Pre-/post-test	1728	Emergency department	Visual reminders, standard process worksheets (PDCA)	Improve wait times	Process outcome	1/11 Fail	Intervention could not be said to be independent of other changes over time No formal statistical test was used Primary outcome measure was not reliable
Piggott, 2011, Canada [40]	Pre-/post-test	1666	Emergency department	Identify and eliminate areas of waste (VSM)	Time to ECG, time to see MD, time to aspirin administration	Process outcome	3/11 Fail	Intervention could not be said to be independent of other changes over time Primary outcome measure was not reliable Outcomes were not blinded
Mazzocato, 2014, Sweden [41]	Pre-/post-test	1046	Emergency department	Identify and eliminate areas of waste (VSM)	To reduce time to see MD, to increase number of patients leaving within 4 h, reduce number present at 4pm shift	Process outcome	5/11 Fail	Intervention could not be said to be independent of other changes over time Insufficient data points for statistical analysis No formal statistical analysis was done
Richardson, 2014, USA [42]	Pre-/post-test	565	Emergency department	Educational training	Decrease wasted nursing time	Process outcome	3/11 Fail	Intervention could not be said to be independent of other changes over time Primary outcome measure was not reliable Outcomes were not blinded
Wojtys, 2009, USA [43]	Pre-/post-test	454	Sport medicine practice	Identify and eliminate areas of waste (VSM)	Improve patient scheduling	Process outcome	1/11 Fail	Intervention could not be said to be independent of other changes over time No formal statistical test was used Primary outcome measure was not reliable
Niemeijer, 2012, Netherlands [44]	Pre-/post-test	445	Traumatology department	Identify and eliminate areas of waste (DMAIC)	Reduce length of stay and cost	Process outcome	1/11 Fail	Intervention could not be said to be independent of other changes over time Insufficient data points for statistical analysis No formal statistical analysis was done
Hakim, 2014, USA [45]	Pre-/post-test	361	Medical and surgical units	Identify and eliminate areas of waste (PDCA)	Improve admission medication reconciliation	Process outcome	3/11 Fail	Insufficient follow-up time Primary outcome measures not reliable Primary outcome measure was not valid
van Lent, 2009, Netherlands [46]	Pre-/post-test	255	Chemotherapy day unit	Identify and eliminate areas of waste (PDCA)	Data efficiency, patient satisfaction and staff satisfaction	Process outcome	4/11 Fail	Intervention could not be said to be independent of other changes over time No formal statistical test was used Primary outcome measure was not reliable
Bhat, 2014, India [47]	Case study	224	Outpatient health information department	Identify and eliminate areas of waste (DMAIC)	Reduce registration time	Process outcome	2/11 Fail	Intervention could not be said to be independent of other changes over time No formal statistical test was used Primary outcome measure was not reliable
Al-Araidah, 2010, Jordan [48]	Case study	217	Inpatient pharmacy	Identify and eliminate areas of waste (DMAIC)	Lead time reduction	Process outcome	4/11 Fail	Intervention could not be said to be independent of other changes over time No formal statistical test was used Primary outcome measure was not reliable
Hydes, 2012, UK [49]	Pre-/post-test	178	Hospital	Value stream analysis (VSM)	Improve efficiency and patient satisfaction	Process outcome	2/11 Fail	Insufficient data points for statistical analysis No formal statistical test was used Primary outcome measure was not reliable
Smith, 2011, USA [50]	Pre-/post-test	171	Cystic fibrosis clinic	Identify and eliminate areas of waste (DMAIC)	Decrease non-value added time	Process outcome	3/11 Fail	Intervention could not be said to be independent of other changes over time Primary outcome measure was not reliable Outcomes were not blinded
Kullar, 2010, UK [51]	Post-test	141	Cochlear implant unit	Value stream analysis (VSM)	Wait time for cochlear implantation	Process outcome	1/11 Fail	Intervention could not be said to be independent of other changes over time No formal statistical test was used Primary outcome measure was not reliable
Siddique, 2012, UK [52]	Post-test	80 (or 129)	General surgery department	One stop cholecystectomy clinic	Waiting list time, number of hospital visits and pre op admissions	Process outcome	4/11 Fail	Intervention could not be said to be independent of other changes over time Primary outcome measure was not reliable Outcomes were not blinded
Lunardini, 2014, USA [53]	Case series	38	Operating room	Value stream analysis (VSM)	To optimize instrument utilization	Process outcome	4/13 Fail	Insufficient data points for statistical analysis, outcomes were not blinded, primary outcome measure was not reliable
Yeh, 2011, Taiwan [54]	Pre-/post-test	36	Private hospital	Identify and eliminate areas of waste (DMAIC)	Improve door to balloon time (AMI revascularization), length of stay	Process outcome	3/11 Fail	Intervention could not be said to be independent of other changes over time No formal statistical test was used Primary outcome measure was not reliable
Luther, 2014, UK [55]	Pre-/post-test	20	Medical admission unit ward	Identify and eliminate areas of waste (PDCA)	Improve patient handover	Process outcome	3/11 Fail	Insufficient data points for statistical analysis No formal statistical test was used Primary outcome measure was not reliable
Shah, 2013, USA [56]	Pre-/post-test	17	Breast imaging centre	Identify and eliminate areas of waste (VSM)	Improve workflow	Process outcome	2/11 Fail	Intervention could not be said to be independent of other changes over time Insufficient data points for statistical analysis Primary outcome measure was not reliable
Gijo, 2013, India [57]	Case study	Not stated	Pathology department	Identify and eliminate areas of waste (DMAIC)	Reduce wait time	Process outcome	2/11 Fail	Intervention could not be said to be independent of other changes over time No formal statistical test was used Primary outcome measure was not reliable
Belter, 2012, USA [58]	Pre-/post-test	Not stated	Oncology outpatient	Identify and eliminate areas of waste (DMAIC)	Decrease patient wait times and improve communication	Process outcome	2/11 Fail	Insufficient data points for statistical analysis No formal statistical test was used Primary outcome measure was not reliable
Snyder, 2009, USA [59]	Pre-/post-test	Not stated	Rural healthcare organization	Training	Decrease supply time, patient wait time, documentation in EMR within 30 minutes	Process outcome	0/11 Fail	Intervention could not be said to be independent of other changes over time Insufficient data points for statistical analysis No formal statistical analysis done
Silva, 2012, USA [60]	Pre-/post-test	Not stated	Clinical engineering department	Identify and eliminate areas of waste (DMAIC)	Improve medical equipment inventory control	Process outcome	0/11 Fail	Intervention could not be said to be independent of other changes over time Primary outcome measure was not reliable Outcomes were not blinded

DMAIC: define, measure, analyse, improve, control; PDCA: plan do check act; TPS: Toyota production system; VSM: value stream mapping; DNT: door to needle time.

Rate ratio <1 is intervention resulted in negative outcome; rate ratio >1 is intervention resulted in positive outcome.

Among the 22 studies accepted, none used high quality experimental study designs (i.e. randomized controlled trials) or even lesser quality quasi-experimental study designs (i.e. prospective longitudinal cohorts). All study designs were of relatively low quality with almost all using before and after study designs without control groups. In fact, only one accepted study had a control group [26]. Among accepted studies, 4 were exclusively concerned with health outcomes, 3 included both health and process outcomes and 15 included process outcomes only (Fig. 2).

Figure 2.

Diagrammatic mapping of included studies to specific outcomes.

Health outcomes

Among the four accepted studies with health outcomes, only one found a statistically significant impact of Lean. They found a reduced relative rate of MRSA infections (RR = 2.47, 95% CI 1.87–3.27), although absolute reductions were very small [15]. The largest study by far included six million patients. This study found no impact of Lean on 30-day mortality rate post-hospital discharge (RR = 0.08, 95% CI −0.30 to 0.46) [13]. The other two studies under this category found no statistically significant impact on adverse events (RR = 0.91, 95% CI 0.72–1.16) or on MRSA incidence (RR = 0.99, 95% CI 0.98–1.01) [14,16] (Table 1).

Process outcomes

Among the 15 accepted studies that examined a vast array of process outcomes (including wait times, patient flow and workplace engagement, inclusion and productivity), only 2 found a statistically significant positive effect of Lean. The benefits included reduced patient visits (RR = 1.84, 95% CI 1.33–2.56) and reduced surgical consults (RR = 4.60, 95% CI 1.82–11.62) [24,29]. In five studies, rate ratios and confidence intervals were not computed because the authors did not include raw data (only summary data). None of the accepted studies reviewed actual financial costs (Table 1).

Health and process outcomes

Of the three articles that evaluated both health and process outcomes, only one article reported a positive effect of Lean in that it improved time dependent stroke care (RR = 1.50, 95% CI 1.21–1.86) [34]. Conversely, in a large study of over 6.8 million patients, Lean had no statistically significant impact on patients leaving without being seen (RR = 1.05, 95% CI 0.77–1.43), patients discharged within 48 h of presentation (RR = 1.19, 95% CI 0.72–1.98) or number of patients readmitted to the hospital within 72-h of discharge (RR = 1.00, 95% CI 1.00–1.00) [32] (Table 1).

The largest Lean healthcare transformation in the world – results from Saskatchewan

The HQC of Saskatchewan surveyed tens of thousands of patients discharged from hospitals pre- and post-Lean [11]. In this systematic review, the most relevant 30 outcomes are reported under the umbrella of 5 broad groupings, which include: self-reported health, hospital experience, communication, respect and patient management. Among the 30 outcomes considered, Lean had no statistically significant impact in 27 of them (Table 2). For example, 30 574 patients were surveyed on self-reported health with no observed impact from Lean (RR = 1.00, 95% CI 0.98–1.04). When measuring direct outcomes for 90 000 patients on their experience with doctors (RR = 1.01, 95% CI 1.00–1.02) and nurses (RR = 1.00, 95% CI 0.99–1.01), no effect of Lean was observed. Only three outcomes showed statistically significant positive outcomes of Lean including: staff washing or disinfecting their hands (RR = 1.179 07, 95% CI 1.05–1.10), staff checking ID bands (RR = 1.08, 95% CI 1.06–1.10) and patients given safety brochures (RR = 1.56, 95% CI 1.49–1.63). The results are found in Table 2.

Table 2.

Data collected by the Saskatchewan health quality council

Saskatchewan health quality council—pre- and post-Lean data
SHQC variables	Pre-Lean (December 2009–January 2012)			Post-Lean (February 2012–March 2014)			Total sample size (n)	Rate ratio	95% CI
	Sample size (N)	%	LCL–UCL	Sample size (n)	%	LCL–UCL
Reported health
High self-reported health	16 637	34.52	26.78–37.96	13 937	34.75	26.16–38.58	30 574	1.00	0.98–1.04
Hospital experience
Patient experience—quality of care transitions	42 435	31.48	28.45–35.43	36 000	32.80	28.09–35.78	78 435	1.02	1.00–1.03
Percentage of patients rating their hospital as 9 or 10/10	16 526	51.95	47.42–59.38	13 803	52.93	46.76–60.05	30 329	1.01	0.99–1.04
Percentage of patients reporting they would definitely recommend the hospital to family and friends	16 498	58.8	52.78–64.60	13 828	57.38	52.13–65.25	30 326	0.98	0.94–1.01
Communication
Patient experience—quality of communication with nurses	50 162	68.30	64.26–70.71	41 965	69.31	63.91–71.07	92 127	1.01	1.00–1.02
Patient experience—Quality of communication with doctors	49 826	73.78	70.36–76.47	41 593	73.93	70.01–76.81	91 419	1.00	0.99–1.01
Percentage of patients reporting they always received good communication about medicines	18 852	50.19	43.55–54.78	16 504	49.94	43.08–55.26	35 356	0.99	0.97–1.02
Percentage of patients responding nurses always listened to them carefully	16 750	63.60	56.93–68.46	14 045	64.76	56.30–69.08	30 795	1.02	1.00–1.04
Percentage of patients responding nurses always explained things clearly	16 699	63.95	57.53–69.03	13 937	64.90	56.88–69.68	30 636	1.01	1.00–1.03
Percentage of patients responding doctors always explained things clearly	16 637	67.07	61.02–72.30	13 885	66.98	60.39–72.93	30 522	1.00	0.99–1.01
Percentage of patients responding doctors always listened to them carefully	16 562	70.92	65.07–75.99	13 830	71.52	64.46–76.61	30 392	1.00	0.99–1.02
Treatment plan explained clearly	15 753	77.79	73.25–83.37	13 201	78.58	72.69–83.93	28 954	1.01	1.00–1.01
Family encouraged to participate in care plan	13 955	80.60	75.47–85.78	11 809	81.31	74.92–86.33	25 764	1.00	0.99–1.02
Percentage of patients reporting staff took their preferences into account discussing health needs	12 886	24.88	19.16–30.93	10 980	26.28	18.56–31.52	23 866	1.05	1.00–1.10
Percentage of patients reporting staff always told them what their new medicine was for	9468	64.17	54.65–70.10	8292	63.29	54.00–70.67	17 760	0.99	0.97–1.01
Percentage of patients reporting staff always talked to them about medication side effects	9413	36.09	28.32–43.58	8245	36.54	27.67–44.22	17 658	1.01	0.97–1.05
Respect
Percentage of patients responding nurses always treated them with courtesy and respect	16 800	77.28	71.41–81.50	14 056	78.26	70.85–81.87	30 856	1.00	0.99–1.01
Percentage of patients responding doctor always treated them with courtesy and respect	16 661	83.27	78.51–87.48	13 906	83.25	78.00–87.99	30 567	1.00	0.99–1.01
Staff respect culture, beliefs, values	15 753	92.23	89.18–95.68	13 221	92.43	88.83–96.03	28 974	1.00	0.99–1.01
Doctors treated patients as a partner in care	15 736	82.47	78.04–87.34	13 159	83.3	77.52–87.85	28 895	1.01	1.00–1.02
Staff treated patients as a partner in care	15 552	78.85	73.68–83.80	13 054	80.07	73.13–84.34	28 606	1.02	1.00–1.03
Doctors respect culture, beliefs, values	15 493	93.81	87.18–91.45	12 948	94.39	91.13–97.49	28 441	1.00	1.00–1.00
Patient care management
Percentage of patients responding their pain was always well managed	22 183	63.90	57.35–67.38	19 174	61.55	56.90–67.82	41 357	0.96	0.95–0.98
Percentage of patients reporting they always received help they needed when they wanted it	17 599	60.50	53.98–65.39	15 737	59.12	53.57–65.60	33 336	0.98	0.96–1.01
Unnecessarily long wait time for room	16 607	79.45	74.62–84.29	13 889	79.18	74.08–84.83	30 496	1.00	0.99–1.02
Staff washed or disinfected their hands	16 529	43.49	36.41–48.27	13 839	46.71	35.76–48.91	30 368	1.07	1.05–1.10
Discharge organization	16 432	27.71	23.05–33.91	13 753	27.88	22.45–34.50	30 185	1.00	0.97–1.10
Suffered medical error	15 976	3.70	1.26–5.75	13 352	3.77	1.10–6.00	29 328	0.98	0.87–1.10
Staff checked ID band before care	14 085	60.52	50.31–63.18	12 224	65.42	49.73–63.76	26 309	1.08	1.06–1.10
Given patient safety brochure	10 854	30.64	18.58–41.42	8980	36.63	17.85–42.16	19 834	1.56	1.49–1.63

Pre- and post-Lean periods were identical (26 months each).

In 2014, the SUN randomly surveyed 1500 nurses on their Lean experience [12]. Among nurses who had direct experience with Lean (729–173 nurses—depending on the variable), 15 outcomes were reviewed. All 15 outcomes reported a statistically significant negative effect of Lean on nurse engagement, usefulness, patient care, time for patient care, workplace issues, availability of supplies, workload, stress and patient safety (Table 3). For example, the following outcomes were reduced, nurse engagement (RR = 0.50, 95% CI 0.40–0.65), quality of patient care (RR = 0.23, 95% CI 0.17–0.31) and patient safety (RR = 0.44, 95% CI 0.37–0.53) while the nurses workload and stress levels increased (RR = 0.29, 95% CI 0.24–0.35) (Table 3).

Table 3.

Data collected by the Saskatchewan Union of Nurses

Saskatchewan Union of Nurses (SUN)—Lean Healthcare 2014 Survey
	Strongly disagree (%)	Strongly agree (%)	n	Rate ratio	95% CI
Experience with Leana
Lean activities engage frontline registered nurses	23.00	10.00	729	0.50	0.40–0.65
Ideas put forward by registered nurses are taken seriously	30.50	6.10	729	0.27	0.20–0.37
Registered nurse input is meaningfully incorporated into the Lean process	35.70	6.00	729	0.25	0.18–0.33
Registered nurses feel safe and supported in voicing criticisms and concerns about Lean initiatives	41.00	5.60	729	0.21	0.16–0.30
Lean is a useful support for the nursing process	38.30	4.00	729	0.17	0.11–0.24
Lean leads to improvements in direct patient care	38.20	5.80	729	0.23	0.17–0.31
Lean has resulted in policies and procedures that improve the workplace	29.10	5.20	729	0.23	0.17–0.33
	Declined	Improved	n	Rate ratio	95% CI
Did Lean decline, stay the same or improveb
The quality of supplies	42.20	9.90	1173	0.37	0.31–0.44
The availability of supplies	50.50	17.90	1173	0.58	0.52–0.66
The time available for direct patient care	41.40	10.40	1173	0.38	0.32–0.47
Workload and stress	49.50	7.90	1173	0.29	0.24–0.35
Patient safety	31.00	10.60	1173	0.44	0.37–0.53
The ability to meet professional standards in the nursing process	34.50	9.30	1173	0.37	0.31–0.45
Time and opportunity for clinical education and training	35.00	7.50	1173	0.33	0.27–0.41
Staff morale and engagement	58.20	7.80	1173	0.30	0.25–0.36

Note: Rate ratio <1 = negative impact of intervention; rate ratio >1 = positive impact of intervention.

^an, sample size—individuals who say they have been involved personally in a workplace Lean initiative. Likert scale was used (where 1 means ‘strongly disagree’ and 5 means ‘strongly agree’).

^bn, sample size—individuals who say their workplace has gone through a Lean improvement process (denominator equals 1500).

Discussion

The purpose of this systematic literature review was to independently assess the effect of Lean thinking or Lean interventions on worker and patient satisfaction, health and process outcomes and financial costs.

For worker satisfaction, the largest study was carried out by the SUN. With every outcome reviewed, Lean had an overall negative effect on worker satisfaction [12]. Among other accepted studies from the electronic search of peer reviewed articles, Lean was shown to have no impact on workplace engagement, inclusion and productivity [26,27]. These outcomes are surprising in that worker engagement and input are essential for Lean principles to succeed [2].

For patient satisfaction, the largest dataset available has been collected by the Saskatchewan HQC [11]. When measuring direct outcomes for patient experience with doctors and nurses, no statistically significant positive or negative effect of Lean was observed. In the 22 studies accepted from the electronic search of peer reviewed articles, none directly evaluated patient satisfaction. That is also surprising because Lean reportedly begins with identifying and ‘removing waste’ in order to ‘add value’ to the customer or patient [2]. That said, it is unclear if other variables, like reduced number of medical consultations were used as proxy outcomes for patient satisfaction and what the patient's perception is (positive or negative) as a result of receiving less visits with their physician [24,29].

Among health outcomes like mortality, no study found a statistically significant impact of Lean. As mentioned previously, the largest study included six million patients and found no impact of Lean on 30-day mortality rate post-hospital discharge [13]. This is perhaps not surprising as Lean potentially only influences healthcare delivery. It obviously has no impact on complex health outcomes like patient adherence to care, let alone the behavioural or social determinants of health [1].

With regard to safety and errors, our systematic review shows that one study found no impact on adverse events while two studies had conflicting results on the impact of Lean on MRSA incidence [14–16]. The suggested impact of Lean on variables like adverse events is interesting because hospitals everywhere have successfully implemented various safety interventions that have proved effective but are not directly related with Lean. For example, the Agency for Healthcare Research and Quality estimates that 1.3 million fewer patients were harmed in American hospitals from 2010 to 2013. These outcomes were mostly due to common sense efforts to reduce surgical site infections, adverse drug events and other preventable incidents. As such, it is unclear what, if any, was the independent effect of Lean in comparison to a multitude of other diverse initiatives to promote safety and reduce errors in healthcare [61].

Although reduced financial cost is a reported benefit of Lean, it is worthy to note that we were unable to identify a single study that had actual quantifiable data to that effect. The province of Saskatchewan appears to be the only jurisdiction with actual financial cost information. External consultant fees were originally estimated to be $40.5 million but were reduced to $35 million when the Lean contract was terminated early [62]. Additionally, $17 million per year was required for internal kaizen promotion offices or $51 million total over the first 3 years. In return, official estimates of cost savings from the Saskatchewan health regions totalled $56934.26 [63]. If the numbers reported are accurate and true, it will mean that $1511 was spent on Lean for every one dollar saved by the province.

Strengths and limitations

The key strengths of our study are that it was a systematic review of Lean interventions in healthcare, it used a quality control checklist, and included a separate examination of both peer-reviewed articles and grey literature. There are also several limitations to our study. First, there are many and quite differing definitions of Lean in healthcare. This study did not attempt to strictly define what Lean is but rather relied on the definitions used by the authors of the articles included in our systematic review. Second, the outcomes were too diverse to permit a meta-analysis. Third, the study designs under review did not incorporate the use of control groups and therefore, it is unclear if the results are actually valid or what the results would be in comparison with a control group. Finally, the pre Lean HQC data for the province of Saskatchewan includes three small pilot projects in three health regions. However, month-to-month comparisons pre- and post-Lean found no statistically significant difference from the small pilot projects.

Comparison of findings

The results of our systematic review on Lean thinking and Lean interventions in healthcare provide additional insight and support the findings of other recent systematic reviews [5,64]. For example, Vest et al. [5] concluded that Lean interventions mainly focused on process outcomes in healthcare. Similarly, a Lean review completed by Mason et al. [64] found that the studies demonstrated improved process outcomes.

However, both Vest et al. [5] and Mason et al. [64] acknowledged that when critically examined, only a few articles met the inclusion criteria for their respective reviews. While Lean was found to be successful in some process outcomes, there were several and serious concerns with the reported study findings. Specifically, they noted that the articles reviewed were fraught with systematic bias, imprecision and serious methodological limitations, which undermined the validity of the results and made measuring and interpreting the true and independent effect of Lean on process and healthcare outcomes unclear and difficult.

Conclusion

The findings of our systematic review suggest that Lean interventions have: (i) no statistically significant association with patient satisfaction and health outcomes, (ii) a negative association with financial costs and worker satisfaction and (iii) potential yet inconsistent benefits on process outcomes like patient flow (reduced patient visits, reduced surgical consults, improved time dependent care) and safety (washing hands, staff checking ID bands and giving patients safety brochures).

More rigorous, higher quality and better conducted scientific research is required to definitively ascertain the impact and effectiveness of Lean in healthcare settings.

While some may strongly believe that Lean interventions lead to quality improvements in healthcare, the evidence to date simply does not support this claim. It is far more likely that Lean is but one of many strategies that might or might not have an impact on healthcare delivery.

The reality is that there are a multitude of internal and external variables that impact complex healthcare and process outcomes and that the independent effect of a specific intervention such as Lean is potentially minimal. For now, the question remains whether continuing to heavily invest in Lean is bringing us closer to or taking us further away from a much needed, viable, long-term solution to an increasingly problematic and unsustainable healthcare delivery system.

Authors’ contributions

J.M. and M.L. contributed to the original conception and design of the study. C.N. and M.L. were responsible for the acquisition of data. M.L. was in charge of the data analysis. J.M., M.L. and C.N. contributed to the interpretation of the data and the drafting of and critical revisions to the manuscript. All authors read and approved the final manuscript.

Funding

This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.

Conflict of interest statement

None declared.