Skip to main content
. 2016 Jan 24;28(2):150–165. doi: 10.1093/intqhc/mzv123

Table 1.

Detailed list of eligible peer review articles from the literature search

Articles that passed methodology review
First author's last name, year of publication, country where study was done Study design Number of participants Location of intervention (ex. Emergency department) Intervention Intervention goal Type of outcome Quality scores Outcome rate ratio and 95% CI
Health outcome studies
Jha, 2012, USA [13] Retrospective cohort 6 000 000 Hospital Pay for performance Reduce 30 day mortality rate Health outcome 9/11 Pass 30 day mortality rate
0.08 (−0.30 to 0.46)
McCulloch, 2010, UK [14] Interrupted time series 2083 Emergency surgery ward PDCA Reduced risk of care related harm Health outcome 6/11 Pass Adverse events
0.91 (0.72–1.16)
Muder, 2008, USA [15] Pre-/post-test 215 ICU and a surgical unit Hand hygiene, contact precautions, active surveillance (TPS) Reduce incidence of MRSA Health outcome 7/11 Pass MRSA infections per 1000 patient days
2.47 (1.87–3.27)
Ellingson, 2011, USA [16] Pre-/post- test 109 Veteran affairs hospital surgical ward Systems and behaviour change to increase adherence to infection control precautions Reduce in MRSA incidence rates Health outcome 7/11 Pass MRSA incidence rate ratio
0.99 (0.98–1.01)
Process outcome studies
Murrell, 2011, USA [17] Pre-/post-test 64 907 Emergency department Rapid triage and treatment ED length of stay and physician wait time Process outcome 7/11 Pass Unable to compute RR
Length of stay reduced from 4.2 (4.2–4.3) to 3.6 (3.6–3.7) hours
Physician start time reduced from 62.2 (61.5–63.0) to 41.9 (41.5–42.4) minutes
Kelly, 2007, Australia [18] Pre-/post-test 63 085 Emergency department Streaming of patients from triage, reallocation of medical and nursing staff (VSM) Reduce number of patients who leave without being seen Process outcome 8/11 Pass Left without being seen
0.99 (0.92–1.08)
Naik, 2012, USA [19] Pre-/post-test 22,527 Emergency department Identify and eliminate areas of waste Emergency wait time Process outcome 6/11 Pass Unable to compute RR
Wait time reduced from 4.6 (4.5–4.9) to 4.0 (3.7–4.1) hours
Simons F, 2014, Netherlands [20] Pre-/post-test 8,009 Operating room of University medical centre DMAIC using A3 intervention Door movements in the operating room Process outcome 6/11 Pass Unable to compute RR
Door movements reduced by 78% from an average of between 15 and 20 times per hour during surgery to 4 times per hour
Burkitt, 2009, USA [21] Retrospective pre-/post 2,550 Veteran affairs surgical center Staff training on hand hygiene, systematic culturing of all admissions, patient isolation Increase appropriateness of perioperative antibiotics and reduction in length of stay Process outcomes 7/11 Pass Length of stay
0.91 (0.76–1.08)
Weaver, 2013, USA [22] Pre-/post-test 2444 Mental health clinic Identify and eliminate areas of waste (TPS) Improving number who attend first appointment, reduce wait for appointment Process outcome 9/11 Pass Number who attended first appointment
1.0 (1.0–1.0)
Wait reduced from 11 days to 8 days
LaGanga, 2011, USA [23] Pre-/post-test 1726 Mental health center Remove over booking Increase capacity to admit new patients and reduce no-shows Process outcome 7/11 Pass No shows
1.13 (1.03–1.23)
van Vliet, 2010, Netherlands [24] Pre-/post-test 1207 Eye hospital Identify and eliminate areas of waste Reduce patient visits Process outcome 9/11 Pass Patient visits
1.84 (1.33–2.56)
Martin, 2013, UK [25] Pre-/post-test 500 Radiology department Value stream analysis (VSM) Reduce patient journey time Process outcome 6/11 Pass Unable to compute.
No pre and post raw data—only percentage changes were given
White, 2014, Ireland [26] Cross-sectional study 338 Hospital Implementation of productive ward program Improve work engagement Process outcome 7/11 Pass Overall work engagement score1.06 (0.96–1.18)
Ulhassan, 2014, Sweden [27] Pre-/post-test 263 Emergency department and two cardiology wards Identify and eliminate areas of waste (DMAIC) Improve teamwork Process outcome 8/11 Pass Overall inclusion
1.02 (0.74–1.42)
Overall trust
1.04 (0.79–1.38)
Overall productivity
1.0 (1.0–1.0)
Collar, 2012, USA [28] Pre-/post-test 234 Otolaryngology operating room Identify and eliminate areas of waste (DMAIC) Improve efficiency and workflow Process outcome 7/11 Pass Unable to compute due to data not being provided.
Turn-over time reduced from 38.4 min to 29 min
Blackmore, 2013, USA [29] Retrospective cohort 200 Breast clinic Identify and eliminate areas of waste Improve timeliness of diagnosis and reduce surgical consults Process outcome 6/11 Pass Reduced surgical consults
4.60 (1.82–11.62)
Simons P, 2014, Netherlands [30] Pre-/post-test 167 Radiotherapy department Implementation of a standard operating procedure Improve compliance to patient safety tasks Process outcome 8/11 Pass Overall compliance
0.96 (0.58–1.58)
Mazzocato, 2012, Sweden [31] Case study 156 Accident and Emergency department Identify and eliminate areas of waste, system restructuring Increase number of patients seen and discharged within four hours Process outcome 10/13 Pass Discharged within four hours
1.07 (0.92–1.26)
Health and process outcome studies
Vermeulen, 2014, Canada [32] Pre-/post-test
Only study with control group		 6 845 185 Emergency department Training and system redesign Left without being seen, discharged within 48 h, readmitted within 72 h, died within 7 days of discharge Process and health outcome 8/11 Pass In comparison to control group:
Left without being seen
1.05 (0.77–1.43)
Discharged within 48 h
1.19 (0.72–1.98)
Readmitted within 72 h of discharge
1.0 (1.0–1.0)
Died within 7 days of discharge
1.03 (0.84–1.26)
Yousri, 2011, UK [33] Pre-/post-test 608 Hospital Identify and eliminate areas of waste Overall mortality, 30 day mortality, door to theatre time, admission to a trauma ward Health and process outcome 6/11 Pass 30 day mortality rate
1.71 (0.70–4.17)
Door to theatre time within 24 h
1.17 (0.86–1.60)
Admission to trauma bed
1.03 (0.90–1.20)
Ford, 2012, USA [34] Pre-/post-test 219 Emergency department Value stream analysis (VSM) Reduce time dependant stroke care and stroke mimic Process outcome and health outcome 7/11 Pass Percent of patients with DNT < 60 min
1.50 (1.21–1.86)
Stroke mimic
0.64 (0.26–1.58)
Articles that failed methodology review
First author's last name, year of publication, country where study was done Study design Number of participants Location of intervention (ex. Emergency department) Intervention Intervention goal Type of outcome Quality scores Major methodological drawbacks
Health outcome studies
Ulhassan, 2013, Sweden [35] Pre-/post-test 4399 Cardiology department Changes to work structure and process Improve patient care Health outcome 4/11 Fail

  • Intervention could not be said to be independent of other changes over time

  • No formal statistical test was used

  • Outcomes were not blinded
Wang, 2014, China [36] Pre-/post-test 622 Nephrology department Training, treatment of high risk patients, specialized outpatient clinic Incidence of peritonitis Health outcome 4/11 Fail

  • Intervention could not be said to be independent of other changes over time

  • Primary outcome measure was not reliable

  • Data did not cover most episodes of intervention at follow-up
Process outcome studies
Wong, 2012, USA [37] Pre-/post-test 234 616 Cytology laboratory New imaging system, workflow redesign Turnaround time, productivity and screening quality Process outcome 4/11 Fail

  • Intervention could not be said to be independent of other changes over time

  • Primary outcome measure was not reliable

  • Outcomes measures were not blinded
Lodge, 2008, UK [38] Post-test 9297 Division of diagnostics and clinical support Intranet based waiting list for radiology services Reduce radiology wait times Process outcome 3/11 Fail

  • Intervention could not be said to be independent of other changes over time

  • Insufficient data points for statistical analysis

  • No formal statistical analysis was done
Willoughby, 2010, Canada [39] Pre-/post-test 1728 Emergency department Visual reminders, standard process worksheets (PDCA) Improve wait times Process outcome 1/11 Fail

  • Intervention could not be said to be independent of other changes over time

  • No formal statistical test was used

  • Primary outcome measure was not reliable
Piggott, 2011, Canada [40] Pre-/post-test 1666 Emergency department Identify and eliminate areas of waste (VSM) Time to ECG, time to see MD, time to aspirin administration Process outcome 3/11 Fail

  • Intervention could not be said to be independent of other changes over time

  • Primary outcome measure was not reliable

  • Outcomes were not blinded
Mazzocato, 2014, Sweden [41] Pre-/post-test 1046 Emergency department Identify and eliminate areas of waste (VSM) To reduce time to see MD, to increase number of patients leaving within 4 h, reduce number present at 4pm shift Process outcome 5/11 Fail

  • Intervention could not be said to be independent of other changes over time

  • Insufficient data points for statistical analysis

  • No formal statistical analysis was done
Richardson, 2014, USA [42] Pre-/post-test 565 Emergency department Educational training Decrease wasted nursing time Process outcome 3/11 Fail

  • Intervention could not be said to be independent of other changes over time

  • Primary outcome measure was not reliable

  • Outcomes were not blinded
Wojtys, 2009, USA [43] Pre-/post-test 454 Sport medicine practice Identify and eliminate areas of waste (VSM) Improve patient scheduling Process outcome 1/11 Fail

  • Intervention could not be said to be independent of other changes over time

  • No formal statistical test was used

  • Primary outcome measure was not reliable
Niemeijer, 2012, Netherlands [44] Pre-/post-test 445 Traumatology department Identify and eliminate areas of waste (DMAIC) Reduce length of stay and cost Process outcome 1/11 Fail

  • Intervention could not be said to be independent of other changes over time

  • Insufficient data points for statistical analysis

  • No formal statistical analysis was done
Hakim, 2014, USA [45] Pre-/post-test 361 Medical and surgical units Identify and eliminate areas of waste (PDCA) Improve admission medication reconciliation Process outcome 3/11 Fail

  • Insufficient follow-up time

  • Primary outcome measures not reliable

  • Primary outcome measure was not valid
van Lent, 2009, Netherlands [46] Pre-/post-test 255 Chemotherapy day unit Identify and eliminate areas of waste (PDCA) Data efficiency, patient satisfaction and staff satisfaction Process outcome 4/11 Fail

  • Intervention could not be said to be independent of other changes over time

  • No formal statistical test was used

  • Primary outcome measure was not reliable
Bhat, 2014, India [47] Case study 224 Outpatient health information department Identify and eliminate areas of waste (DMAIC) Reduce registration time Process outcome 2/11 Fail

  • Intervention could not be said to be independent of other changes over time

  • No formal statistical test was used

  • Primary outcome measure was not reliable
Al-Araidah, 2010, Jordan [48] Case study 217 Inpatient pharmacy Identify and eliminate areas of waste (DMAIC) Lead time reduction Process outcome 4/11 Fail

  • Intervention could not be said to be independent of other changes over time

  • No formal statistical test was used

  • Primary outcome measure was not reliable
Hydes, 2012, UK [49] Pre-/post-test 178 Hospital Value stream analysis (VSM) Improve efficiency and patient satisfaction Process outcome 2/11 Fail

  • Insufficient data points for statistical analysis

  • No formal statistical test was used

  • Primary outcome measure was not reliable
Smith, 2011, USA [50] Pre-/post-test 171 Cystic fibrosis clinic Identify and eliminate areas of waste (DMAIC) Decrease non-value added time Process outcome 3/11 Fail

  • Intervention could not be said to be independent of other changes over time

  • Primary outcome measure was not reliable

  • Outcomes were not blinded
Kullar, 2010, UK [51] Post-test 141 Cochlear implant unit Value stream analysis (VSM) Wait time for cochlear implantation Process outcome 1/11 Fail

  • Intervention could not be said to be independent of other changes over time

  • No formal statistical test was used

  • Primary outcome measure was not reliable
Siddique, 2012, UK [52] Post-test 80 (or 129) General surgery department One stop cholecystectomy clinic Waiting list time, number of hospital visits and pre op admissions Process outcome 4/11 Fail

  • Intervention could not be said to be independent of other changes over time

  • Primary outcome measure was not reliable

  • Outcomes were not blinded
Lunardini, 2014, USA [53] Case series 38 Operating room Value stream analysis (VSM) To optimize instrument utilization Process outcome 4/13 Fail

  • Insufficient data points for statistical analysis, outcomes were not blinded, primary outcome measure was not reliable
Yeh, 2011, Taiwan [54] Pre-/post-test 36 Private hospital Identify and eliminate areas of waste (DMAIC) Improve door to balloon time (AMI revascularization), length of stay Process outcome 3/11 Fail

  • Intervention could not be said to be independent of other changes over time

  • No formal statistical test was used

  • Primary outcome measure was not reliable
Luther, 2014, UK [55] Pre-/post-test 20 Medical admission unit ward Identify and eliminate areas of waste (PDCA) Improve patient handover Process outcome 3/11 Fail

  • Insufficient data points for statistical analysis

  • No formal statistical test was used

  • Primary outcome measure was not reliable
Shah, 2013, USA [56] Pre-/post-test 17 Breast imaging centre Identify and eliminate areas of waste (VSM) Improve workflow Process outcome 2/11 Fail

  • Intervention could not be said to be independent of other changes over time

  • Insufficient data points for statistical analysis

  • Primary outcome measure was not reliable
Gijo, 2013, India [57] Case study Not stated Pathology department Identify and eliminate areas of waste (DMAIC) Reduce wait time Process outcome 2/11 Fail

  • Intervention could not be said to be independent of other changes over time

  • No formal statistical test was used

  • Primary outcome measure was not reliable
Belter, 2012, USA [58] Pre-/post-test Not stated Oncology outpatient Identify and eliminate areas of waste (DMAIC) Decrease patient wait times and improve communication Process outcome 2/11 Fail

  • Insufficient data points for statistical analysis

  • No formal statistical test was used

  • Primary outcome measure was not reliable
Snyder, 2009, USA [59] Pre-/post-test Not stated Rural healthcare organization Training Decrease supply time, patient wait time, documentation in EMR within 30 minutes Process outcome 0/11 Fail

  • Intervention could not be said to be independent of other changes over time

  • Insufficient data points for statistical analysis

  • No formal statistical analysis done
Silva, 2012, USA [60] Pre-/post-test Not stated Clinical engineering department Identify and eliminate areas of waste (DMAIC) Improve medical equipment inventory control Process outcome 0/11 Fail

  • Intervention could not be said to be independent of other changes over time

  • Primary outcome measure was not reliable

  • Outcomes were not blinded

DMAIC: define, measure, analyse, improve, control; PDCA: plan do check act; TPS: Toyota production system; VSM: value stream mapping; DNT: door to needle time.

Rate ratio <1 is intervention resulted in negative outcome; rate ratio >1 is intervention resulted in positive outcome.