. 2019 Sep 26;3(4):384–408. doi: 10.1016/j.mayocpiqo.2019.07.006

Table 1.

Study Characteristics Stratified by Intervention Type^a

Reference, year	Country	Study design	No. of participants	Population and setting	Type of intervention	Outcome	Follow-up	Level of evidence^b
Teamwork (N=20)
Chapman & Blash,¹⁹ 2017	United States	Cross-sectional	886	Primary care practices	Teamwork: Employing medical assistants in an innovative model of care with new roles with a focus on career advancement, training, and enhanced compensation for the new medical assistant roles	Staff satisfaction	4 y	4
Contratto et al,²⁰ 2016	United States	Pre-post intervention survey	9	Physicians Urban academic general internal medicine primary care practice	Teamwork: To evaluate the impact of using full-time clerical support to enter tests ordered by physicians, identify incomplete health maintenance measures, and preload new patient information	14-Item survey	4 mo	4
Contratto et al,²¹ 2017	United States	Quasi-experimental (single-group pre-post intervention) mixed-methods	7	Academic general internal medicine practice	Teamwork: Clerical support personnel for physician order entry	Physician satisfaction	4 mo	4
Danila et al,²² 2018	United States	Pre-post intervention survey	6	Physicians (3 rheumatologists and 3 endocrinologists) Rheumatology and endocrinology clinics	Teamwork: Use of scribes	JSS	6 wk	4
Gidwani et al,²³ 2017	United States	RCT	4	Physicians Academic family medicine clinic	Teamwork: Use of scribes to draft all relevant documentation	Physician satisfaction, measured by a 5-item instrument that included physicians’ perceptions of medical record quality and accuracy	1 y	1b
Heaton et al,²⁴ 2016	Multinational	Systematic review	NA	NA	Teamwork: Use of scribes	Physician satisfaction	NA	4
Hung et al,²⁵ 2018	United States	Pre-post intervention survey	680	Physicians 46 Primary care departments in a large ambulatory care delivery system	Teamwork/Transitions: Lean-based workflow redesigns, which included colocating physician and medical assistant dyads, delegating major responsibilities to nonphysician staff, and mandating greater coordination and communication among all care team members	MBI	3 y	4
Imdieke & Martel,²⁶ 2017	United States	Quasi-experimental, nonrandomized pre- and post-intervention study	2	Internal medicine physicians Hospital-based, outpatient primary care clinic	Teamwork: Incorporating medical scribes in an ambulatory clinic to support physician documentation in the electronic medical record	Physician satisfaction	4-6 wk	4
Koshy et al,²⁷ 2010	United States	Nonrandomized, static-group comparison study	5	Urologists, residents Urology clinic within a single academic medical center	Teamwork: Scribes to record electronic medical information throughout the patient-physician encounter	Physician acceptance and satisfaction	10 mo	4
Linzer et al,²⁸ 2015	United States	Cluster RCT	166 (135 completed the study)	Primary care physicians (family and general internists) 34 Clinics in Upper Midwest and NYC	Teamwork/Transitions: Projects to improve communication, changes in workflow, and targeted quality improvement projects	Survey tools from MEMO and PWS	12 mo, 18 mo	2b
Linzer et al,²⁹ 2017	United States	Cluster RCT	165	Primary care physicians (family and general internists) 34 Clinics in Upper Midwest and NYC	Teamwork/Transitions: Quality improvement projects to improve communication between physicians, workflow design, and chronic disease management	OWL	6 mo, 12 mo	2b
McCormick et al,³⁰ 2018	United States	Pre-post intervention survey	6	Urologists Academic urology clinic	Teamwork: Use of scribes	Work satisfaction	3 mo	4
Pierce et al,³¹ 2017	United States	Pre-post intervention survey	55	Physicians and advanced practice clinicians Academic hospital	Teamwork: 13 Team-based and organizational tactics to improve resilience, including expansion of leadership roles, faculty coaching for new hires, and value-based clinical schedule redesign	NR	3 y	4
Pozdnyakova et al,³² 2018	United States	Prospective, pre-post pilot study	6	General internal medicine faculty	Teamwork: Use of scribes to complete EHR	Workplace satisfaction; burnout	1 wk	4
Quenot et al,³³ 2012	France	Longitudinal, monocentric, before-and-after, interventional study	4	Physicians ICU	Teamwork: Intensive communication strategy regarding end-of-life practices in the ICU to alleviate stress for caregivers	MBI	Post-intervention	4
Shaw et al,³⁴ 2017	United States	Pre-post intervention survey	NR	Medical doctors NR	Teamwork: Team-based primary care redesign, “Primary Care 2.0”, with the goal of addressing the Quadruple Aim of health care (ie, the Triple Aim plus reducing workforce burnout) with the following components: (1) an expanded “care coordinator” role for medical assistants including scribing, population health management, and between-visit care management, (2) health coaching and motivational interviewing, (3) “lean” quality improvement to support a Learning Health System, (4) telehealth, (5) protected physician time for care coordination, and (6) an onsite extended interdisciplinary care team (ie, mental health, pharmacy, physical therapy)	NR	5 mo	4
Shultz & Holmstrom,³⁵ 2015	Multinational	Systematic review	NA	Emergency department, urology, or cardiology clinicians	Teamwork: Use of scribes	Clinician satisfaction	NA	4
Was & Cornaby,³⁶ 2016	United States	Pre-post intervention survey	23	Residents Large academic center	Teamwork: Common space for residents (ie, “Gas Lounge”)	NR	Post-intervention	4
West et al,³⁷ 2014	United States	RCT	74	Physicians Department of medicine at the Mayo Clinic	Teamwork: 19 Biweekly facilitated physician discussion groups incorporating elements of mindfulness, reflection, shared experience, and small group learning	JSS, Empowerment at Work Scale, Medical Outcomes Study Short-Form Health Survey, MBI, Perceived Stress Scale, Jefferson Scale of Physician Empathy	1 y	1b
Willard-Grace et al,³⁸ 2017	United States	Cross-sectional	236	Clinicians County-run primary care clinics	Teamwork: A defined model of team-based care in which the association between enhanced roles for medical assistants, registered nurses, and behavioral health professionals is defined	MBI	NR	4
Time (N=14)
Ali et al,³⁹ 2011	United States	Cluster RCT	45	Physicians with various specialties ICU	Time: Two intensivist staffing schedules were compared: continuous and interrupted (rotations every 2 wk) for 14 mo	Scales derived from the National Study of the Changing Workforce	9 mo	1b
Desai et al,⁴⁰ 2018	United States	Cluster-randomized trial	80	First-year residents 63 Internal medicine residency practices	Time: Duty hour policies of the 2011 ACGME	Overall well-being, MBI	7 mo	1b
Fassiotto & Maldonado,⁴¹ 2016	United States	Pre-post intervention survey	60	Medical school faculty NR	Time: Time-banking intervention measures unacknowledged teaching, service, and clinical activities and acknowledges them with practical rewards	NR	Post-intervention	4
Garland et al,⁴² 2012	Canada	Crossover RCT	34	Physicians ICU	Time: Shift work staffing in which there was 24-7 intensivist presence. The same pool of intensivists supplied day shift and night shift coverage. In any given week, a single intensivist was responsible for all 7-day shifts, whereas 2 different intensivists alternated the 7 night shifts	MBI (emotional exhaustion subscale)	Post-intervention	1b
Kim & Wiedermann,⁴³ 2011	United States	Prospective cohort	56	Residents Large pediatric training program	Time: 2003 ACGME work hour limits	NR	7 y	4
Landrigan et al,⁴⁴ 2008	United States	Prospective cohort	220	Residents Pediatric residency programs at hospitals	Time: 2003 ACGME work hour limits for US resident physicians. Residents can work no more than 30 consecutive hours and no more than 80 to 88 h/wk, averaged over 4 wk	MBI	1 y	4
Lucas et al,⁴⁵ 2012	United States	Cluster randomized crossover noninferiority trial	62	Physicians University-affiliated teaching hospital	Time: Assignment to random sequences of 2- and 4-wk rotations	Questionnaire includes questions from MEMO study, Perceived Stress Scale, MBI, national job burnout survey	2/4 wk	1b
Moeller & Walker,⁴⁶ 2017	United States	Pre-post intervention survey	NR	Physicians NR	Time: Practice Refresh pilot program that initially reduces and then gradually increases the time physicians spend with patients so that physicians can learn and practice skills in efficiency, teamwork, and self-care	NR	NR	4
Morrow et al,⁴⁷ 2014	United Kingdom	Cross-sectional	82	Junior doctors Deanery	Time: United Kingdom WTR applied fully to junior doctors since 2009, with a limit of 48 h/wk, averaged across a reference period of 26 wk, alongside specified minimum rest periods	NA	NA	4
Parshuram et al,⁴⁸ 2015	Canada	RCT	47	Residents University-affiliated ICUs	Time: In-house overnight schedules of 24, 16, or 12 h	MBI	2 mo	2b
Ripp et al,⁴⁹ 2015	United States	Pre-post intervention survey	128 (2011-2012 cohort); 111 (2008-2009 cohort)	Internal medicine residents Academic medical centers	Time: 2011 ACGME modified duty hours standards to limit continuous duty of first-year residents to 16 h	MBI, ESS	1 y	4
Schuh et al,⁵⁰ 2011	United States	Prospective, unblinded study	34	Neurology residents Neurology residency program	Time: 2008 Institute of Medicine work duty hour recommendations that limit shifts to 16 or 24 h with a 5-h nap, eliminate averaging of any on-call shifts, increase time off between shifts for night float and overnight call, limit consecutive night float shifts to 4, and provide 1 d off/wk/5 per mo without averaging	MBI	1 mo	4
Shea et al,⁵¹ 2014	United States	RCT	106	Graduate internal medicine interns Internal medicine service hospital	Time: A 2-h period of protected time in which interns were expected to sleep (12:30 am-5:30 am) for 4 wk	MBI	Post-intervention	1b
Tucker et al,⁵² 2010	Wales	Cross-sectional	336	Residents and interns NR	Time: Schedule design	Questionnaire developed for study	NA	4
Transitions (N=9)
Albadry et al,⁵³ 2014	Egypt	Cross-sectional	140	Residents and assistant lecturers Outpatient clinic	Transitions: Six Sigma methodology as quality improvement intervention	MBI	6 mo	4
Amis & Osicki,⁵⁴ 2018	United Kingdom	Pre-post intervention survey	13	First-year residents	Transitions: A checklist aimed to reduce the number of inappropriate prescribing tasks	Job satisfaction	3 Weekends	4
Callahan et al,⁵⁵ 2018	United States	Pre-post intervention survey	9	Fellows NR	Transitions: Bundle of evidence-based interventions to improve burnout and professional satisfaction that were designed to fit the fellowship program	ESS, quality of life	6 mo	4
Dunn et al,⁵⁶ 2007	United States	Noncontrolled prospective intervention study	22-32	Physicians Primary care group	Transitions: Data-guided interventions and systematic improvement processes that included (1) leadership valuing physician well-being equal to quality of care and financial stewardship, (2) physicians identifying factors that influenced well-being, followed by plans for improvement with accountability, and (3) measuring the well-being of physicians regularly using validated instruments	ACP/ASIM survey on physician satisfaction, MBI	6 y	4
Giannini et al,⁵⁷ 2013	Italy	Pre-post intervention survey	71	Doctors ICU	Transitions: Increase in daily visiting time to at least 8 h (policy change)	MBI, STAI	6 mo, 12 mo	4
Hung et al,²⁵ 2018	United States	Pre-post intervention survey	680	Physicians 46 Primary care departments in a large ambulatory care delivery system	Teamwork/Transitions: Lean-based workflow redesigns, which included colocating physician and medical assistant dyads, delegating major responsibilities to nonphysician staff, and mandating greater coordination and communication among all care team members	MBI	3 y	4
Lee et al,⁵⁸ 2017	United States	Pre-post intervention survey	Baseline, 18; postin-tervention, 15	Neuroradiology fellows and neuroradiologists Academic neuroradiology practice, part of a larger health care system with 6 hospitals and 80 outpatient imaging sites	Transitions: Image interpretive and non–image interpretive reading room workflows	14-Question survey, Likert scale rating 1-5	1 mo	4
Linzer et al,²⁸ 2015	United States	Cluster RCT	166 (135 completed the study)	Primary care physicians (family and general internists) 34 Clinics in Upper Midwest and NYC	Teamwork/Transitions: Projects to improve communication, changes in workflow, and targeted quality improvement projects	Survey tools from MEMO and PWS	12 mo, 18 mo	2b
Linzer et al,²⁹ 2017	United States	Cluster RCT	165	Primary care physicians (family and general internists) 34 Clinics in Upper Midwest and NYC	Teamwork/Transitions: Quality improvements projects to improve communication between physicians, workflow design, and chronic disease management	OWL	6 mo, 12 mo	2b
Technology (N=10)
Agha et al,⁵⁹ 2010	United States	NR	9	Pulmonary, rheumatology, and endocrine physicians NR	Technology: To measure the impact of EHR use on physician satisfaction	NR	NR	4
Babbott et al,⁶⁰ 2013	United States	Prospective	422	Internal medicine and family medicine physicians	Technology: Secondary analysis on data from the MEMO study in which physicians and office managers completed questionnaires about their office practice, including specific EHR features the office used	NR	NR	4
Beam et al,⁶¹ 2017	United States	Pre-post intervention survey	158	Physicians Neonatal ICU	Technology: Computerized physician order entry implementation	Job satisfaction	1 y	4
Ehrlich et al,⁶² 2016	United States	Pre-post intervention survey	25	Ophthalmologists Large academic ophthalmology department	Technology: EHR system	30-Question survey using Likert scale rating, job satisfaction	24 mo	4
Heyworth et al,⁶³ 2012	United States	Pre-post intervention survey	163	Primary care and specialty NR	Technology: To measure predictors of physician satisfaction following EHR adoption	Massachusetts eHealth Collaborative survey	Post-intervention	4
Joseph et al,⁶⁴ 2017	United States	Pre-post intervention survey	NR	Physicians NR	Technology: The impact of a brief, intensive technology deployment and training intervention that was aimed at improving individual clinician’s efficiency in using EHR	NR	NR	4
Lapointe et al,⁶⁵ 2018	United States	Pre-post intervention survey	25	Internal medicine residents 591-Bed urban hospital	Technology: EHR-based text paging system to communicate with internal medicine residents	Stress	6 mo	4
Menachemi et al,⁶⁶ 2009	United States	Cross-sectional	4203	Primary care physicians and clinical specialists Outpatient settings	Technology: To evaluate the relationship between physician IT adoption and practice satisfaction	Survey using Likert scale questions on job/practice satisfaction	NA	4
Milenkiewicz,⁶⁷ 2017	United States	Pre-post intervention survey	NR	Physicians Department of Addiction Medicine at Kaiser Permanente	Technology: To test the usability of an EHR tool to improve and standardize the documentation process	NR	Post-intervention	4
Wylie et al,⁶⁸ 2014	United States	Cross-sectional	2365	Primary care physicians Practice with more than 10 physicians	Technology: To identify how EHR use affected clinical practice	Likert-type scale questions regarding how EHR affected medical practice	NA	4

ACGME = Accreditation Council for Graduate Medical Education; ACP/ASIM = American College of Physicians/American Society of Internal Medicine; her = electronic health record; ESS = Epworth Sleepiness Scale; JSS = Physician Job Satisfaction Scale; ICU = intensive care unit; IT = information technology; MBI = Maslach Burnout Inventory; MEMO = Minimizing Error, Maximizing Outcome; NA = not available; NR = not reported; NYC = New York City; OWL = Office and Work Life measures; PWS = Physician Worklife Study; RCT = randomized controlled trial; STAI = State-Trait Anxiety Inventory; WTR = Working Time Regulations.

Oxford Centre for Evidence-based Medicine Levels of evidence¹⁸: 1b = individual RCT (with narrow confidence interval); 2b = individual cohort study (including low-quality RCT; eg, <80% follow-up); 4 = case series (and poor-quality cohort and case control studies).