Skip to main content
PMC home page
BMC Medicine logoLink to BMC Medicine
. 2012 Jun 28;10:65. doi: 10.1186/1741-7015-10-65

A systematic review of the effects of residency training on patient outcomes

Renée M van der Leeuw 1,✉, Kiki MJMH Lombarts 1,#, Onyebuchi A Arah 2,3,#, Maas Jan Heineman 4,#
PMCID: PMC3391170  PMID: 22742521

Abstract

Background

Residents are vital to the clinical workforce of today and tomorrow. Although in training to become specialists, they also provide much of the daily patient care. Residency training aims to prepare residents to provide a high quality of care. It is essential to assess the patient outcome aspects of residency training, to evaluate the effect or impact of global investments made in training programs. Therefore, we conducted a systematic review to evaluate the effects of relevant aspects of residency training on patient outcomes.

Methods

The literature was searched from December 2004 to February 2011 using MEDLINE, Cochrane, Embase and the Education Resources Information Center databases with terms related to residency training and (post) graduate medical education and patient outcomes, including mortality, morbidity, complications, length of stay and patient satisfaction. Included studies evaluated the impact of residency training on patient outcomes.

Results

Ninety-seven articles were included from 182 full-text articles of the initial 2,001 hits. All studies were of average or good quality and the majority had an observational study design. Ninety-six studies provided insight into the effect of 'the level of experience of residents' on patient outcomes during residency training. Within these studies, the start of the academic year was not without risk (five out of 19 studies), but individual progression of residents (seven studies) as well as progression through residency training (nine out of 10 studies) had a positive effect on patient outcomes. Compared with faculty, residents' care resulted mostly in similar patient outcomes when dedicated supervision and additional operation time were arranged for (34 out of 43 studies). After new, modified or improved training programs, patient outcomes remained unchanged or improved (16 out of 17 studies). Only one study focused on physicians' prior training site when assessing the quality of patient care. In this study, training programs were ranked by complication rates of their graduates, thus linking patient outcomes back to where physicians were trained.

Conclusions

The majority of studies included in this systematic review drew attention to the fact that patient care appears safe and of equal quality when delivered by residents. A minority of results pointed to some negative patient outcomes from the involvement of residents. Adequate supervision, room for extra operation time, and evaluation of and attention to the individual competence of residents throughout residency training could positively serve patient outcomes. Limited evidence is available on the effect of residency training on later practice. Both qualitative and quantitative research designs are needed to clarify which aspects of residency training best prepare doctors to deliver high quality care.

Background

It is globally understood that high quality and safe patient care can only be provided if doctors are well prepared for this task through (residency) training [1]. Worldwide, medical educationalists and clinicians involved in residency training are focused on the improvement of education through modernizations including implementing competency based learning and assessing and advancing the quality of residents' education through accreditation standards [2,3]. Ultimately, societies need to know if it matters how, where and by whom doctors were trained.

Quality of care improvement initiatives have focused on finding solutions to ensure quality and safety in health care services. Research reports underscore the effectiveness of quality improvement initiatives in bettering patient outcomes [4,5]. However, it is not yet known to what extent residency training influences patient outcomes [6]. This is surprising given that there is a shared belief that quality and performance initiatives encourage life-long learning, which starts during undergraduate medical education (UGME) and continues through residency training. Research in medical education conducted to evaluate the effects of changes to residency training typically focuses on educational outcomes and does not include patient outcomes [7,8]. Research in UGME has focused more successfully on the link between medical education and patient outcomes. For example, scores achieved by medical students on qualifying examinations before licensing can be linked to later complaints to medical regulatory authorities [9,10]. In addition, professional behavior in both medical school and residency training has been correlated with later disciplinary action by medical boards [11,12]. Given the indisputable link between training and care delivery in daily practice, we would expect to find a vast amount of research focusing on the link between residency training and patient outcomes, to investigate and explain the relationship between the various aspects of training and their impact on patient care.

In daily practice, a resident is a 'learner' while being responsible for patients as a 'provider of care'. Thus, it is pertinent to know whether care delivered by residents is of at least equal quality to that delivered by faculty, or if it introduces a risk for patient care. Once training is completed, residents are expected to be well prepared to deliver a high quality of care. It is, therefore, essential to assess aspects of residency training through patient outcomes to evaluate the direct and future effects of global investments made in training programs. We assume that patient care provided during and after residency training will benefit from residents being well-trained. To our knowledge, outcomes of patient care delivered by residents during and after residency training have not been comprehensively studied in a systematic review. For that reason, we systematically reviewed recent literature on the broad research question: 'What is the effect of (aspects of) residency training on patient outcomes?'.

Methods

Data sources and searches

The primary data sources for this review were electronic databases MEDLINE, Cochrane, Embase and the Education Resource Information Center (ERIC). Databases were searched from December 2004 until February 2011 to place our review in the context of recent modernization efforts in residency training, work-hour restrictions and implementation of competency based learning in residency training.

A preliminary search was conducted with the assistance of a senior librarian to specify our keywords and optimize the search strategy. Databases were searched using keywords for both free text and Medical Subject Heading (MeSH) terms on the subjects of residency training and patient outcomes. In addition to the electronic search, reference lists from selected articles were later searched manually to obtain any additional relevant studies. We defined residency training as the training of residents, specialist registrars or trainees to become a specialist, consultant, general practitioner, family physician or faculty. The following keywords described residency training in our search string: Education, Medical; Teaching; Training, Clinical; Residency/Resident; Internship; Consultant; Faculty, Medical; Alumni. We added general and commonly used patient care outcome measurements to our search using the keywords: Outcome, Assessment, Clinical; Quality of Care; Safety; Complications, Postoperative; Surgical Wound Infections; Patient Readmission; Reoperation; Length of Stay; Iatrogenic Disease; Mortality, Hospital; Adverse Events; Patient Satisfaction. No language restrictions were applied. The search was limited to exclude comments, editorials or letters. The complete search string can be found in Additional file 1.

Study selection

Clearly irrelevant articles were excluded based on the title and abstract by one reviewer (RML). Two independent reviewers (RML, KMJMHL) then assessed the title and abstract of all remaining articles for relevance to the study. If abstracts were unavailable, full-text articles were retrieved to assess relevance. After this selection of articles, all full-text articles were retrieved to examine compliance with the inclusion criteria. Any disagreement in the assessment of articles was resolved through discussion within the review team.

Studies were included if they explicitly related residency training to patient outcomes of residents' care. We included studies with an educational intervention or comparison and clearly defined and reported patient outcomes. Residents and faculty of all levels of experience were included if they were either participant or comparator. All types of study designs were included. Studies were excluded if the intervention was not educational in content (for example, duty hour reform) or if outcome measures were not directly related to patients (for example, knowledge or skill tests or performance indicators). Other reasons for exclusion were other types of education (such as dental, undergraduate or continuing medical education), non-research designs (for example, commentary) or participants who were not targeted as subjects of this systematic review (for example, medical students or nursing home residents).

Data extraction and quality assessment

The review team agreed upon a data extraction form, which enabled one reviewer (RML) to extract data from included studies. Data extraction was regularly discussed and checked with other members of the review team. Information about participant characteristics, applied teaching interventions, patient outcomes, additional outcome measures, effect of residency training on patient outcomes, and study design were extracted. Review Manager 5 was used to collect data [13]. The quality assessment of included studies was performed using the validated Medical Education Research Study Quality Instrument (MERSQI) [14,15]. Two independent reviewers assessed the study quality of 10 articles to reach consensus; thereafter, one reviewer (RML) could complete the MERSQI for the remaining studies. The MERSQI enabled us to assess and compare the quality of all included studies by calculating the final MERSQI score as the sum of all scores, corrected for 'not applicable' items such as the internal structure, content and relation to other variables of measurement instruments.

Data synthesis and analysis

If the data are suitable, we will perform a meta-analysis to synthesize and pool research findings using effect measures of studies with related research hypotheses. However, if the results are too heterogeneous, we will describe all study outcomes using a narrative analysis and a construction of subgroups based on primary objectives of studies to clarify study results and draw conclusions.

Results

We identified 2,001 citations, 1,934 citations by the literature search and 67 additional citations from a manual search. The broad selection of articles by title and abstract led to the retrieval of 182 potentially eligible studies. After a full-text review of the 182 studies, nine studies had to be discussed by the review team to reach consensus on inclusion or exclusion. A total of 97 studies met the inclusion criteria of this review. All studies were published in English. The selection process and subsequent categorization is summarized in Figure 1.

Figure 1.

Figure 1

Open in a new tab

Disposition of the articles found in the literature on the link between residency training and patient outcomes.

Study design and quality

Because we included all types of study design, including a variety of different interventions and participants from different disciplines at all levels of training, the degree of heterogeneity was too great for any quantitative analysis of the data. The formation of subgroups also did not allow us to perform a meta-analysis. Therefore, we descriptively report study outcomes, with detailed quantitative information on individual studies available in the tables in Additional file 2. Study quality ranged from 9 to 15.6 (mean 12.9) on the MERSQI scale ranging from 4.5 to 18.

Categorization of studies

Ninety-six studies evaluated the relation between resident participation in patient care and patient outcomes during residency training (Tables S1a to S1d and S2, Additional file 2) and one study focused on patient outcomes post-residency training, when residents practiced as independent faculty (Table S3, Additional file 2).

In the largest group of studies during residency training, two categories could be defined: describing patient outcomes as a result of either the level of experience of residents (79 studies; Tables S1a to S1d, Additional file 2) or specific training interventions (17 studies; Table S2, Additional file 2). To clarify the effects of the level of experience of residents on patient outcomes, the 79 studies were further divided into four subcategories according to the seniority of residents. First, 19 studies compared new residents commencing their training at the start of the academic year (July/August) to other months of the year (Table S1a, Additional file 2). Second, seven studies evaluated the effect of individual progress of residents on patient outcomes (Table S1b, Additional file 2). Third, 10 studies clarified the progression of residents through residency training by comparing patient outcomes of residents of different training years (Table S1c, Additional file 2). Fourth, 43 studies used patient outcomes of those cared for by faculty as a 'gold standard' to evaluate patient outcomes of residents' care (Table S1d, Additional file 2).

Nineteen studies comparing the start of the academic year to another time of year (Table S1a, Additional file 2)

The first subcategory includes 19 studies investigating the impact of inexperienced residents on patient outcomes at the start of the academic year, also referred to as the July effect [16-34]. Of the 19 studies evaluating a July effect, 14 mostly surgical studies reported no differences in patient outcomes compared with other months of the year (risk-adjusted) [16-20,22-26,31-34]. Of the five remaining studies, one study reported no difference in mortality in July, but potentially preventable complications did occur more often in July, although numbers were too small to allow for effective root cause analysis [27]. Another study reported a higher risk-adjusted mortality in July, which progressively decreased over the course of the year [21]. In a study on medication errors, the July effect could partially explain a spike of fatal medication errors [29]. One study reported a small July effect on outcomes related to cerebral shunt surgery in children and emphasized the need for supervision [28]. Another study showed a reduction in resident scores on a patient satisfaction questionnaire around July [30].

Seven studies evaluating the individual progression of residents using patient outcomes (Table S1b, Additional file 2)

In the second subcategory on the level of experience of residents, the individual progress or 'learning curve' of residents was covered by seven studies [35-41]. These studies refer to the progress individual residents make during their training or to the number of interventions they need to perform to reach a benchmark performance for specific procedures. Studies aimed to discover a plateau in the residents' learning curve in selected cases [40,41], under supervision [35,37], or with a stepwise approach [36,38,39]. Of the seven studies, six surgical studies (general surgery and obstetrics and gynecology) describe a decrease in operation time between the first and last cases performed by residents [35-38,40,41]. One emergency medicine study reported improved residents' performance on bedside ultrasonography [39].

Ten studies comparing residents of different training years (Table S1c, Additional file 2)

Ten studies in the third subcategory evaluated patient outcomes between residents of different training years [42-51]. Six studies showed that more senior residents had better patient outcomes or higher patient satisfaction scores compared with junior residents [42-44,46,47,50]. One study showed that morbidity and mortality were not adversely affected by residents working under different levels of supervision [45]. Airway management by residents was reported safe in a large multicenter study with adequate rescue options [51]. One study reported a non-significant trend between the seniority of the resident and improved patient outcome [48]. Although one prospective cohort study reported that more senior residents were involved in complications, this was likely secondary to their disproportionate roles in more difficult operations [49].

Forty-three studies using faculty as a gold standard to evaluate patient outcomes of residents' care (Table S1d, Additional file 2)

The fourth subcategory of studies related to the level of experience of residents includes 43 studies comparing faculty to residents. Thirty-eight of these studies focused on surgical disciplines (aesthetic surgery, general surgery, thoracic surgery, orthopedics, urology, ophthalmology and obstetrics and gynecology) [52-89]. The remaining five studies were conducted in radiology [90,91], family medicine [92,93] and anesthesiology [94]. Thirty-one studies reported no statistically significant differences between faculty and residents on a wide variety of outcomes including mortality, morbidity and length of stay [52,54,56-58,61,63,67-69,73-90,92-94]. Although patient outcomes were similar, 12 of these 31 studies did report a significantly longer operation time for residents [61,67,69,74,77,78,81,83,84,87-89]. Nine studies reported negative outcomes of residents' involvement in patient care [53,55,59,60,62,65,66,71,91]. Of these nine studies, five studies found significant differences between faculty and residents [53,55,59,60,91]. Residents' cases resulted in a modestly elevated blood loss during surgery without clinical consequences [53], radiographic or clinical leaks after esophagectomies without a higher take-back rate [55], a small but significantly higher take-back rate after cardiac operations [59], a discrepancy rate of 13.6% for residents' preliminary interpretations of radiology reports [91], and higher morbidity rates and length of stay for patients cared for by residents working without supervision [60]. Of the other four studies, two compared their outcomes to results in the literature [62,65], one did not formally train their residents [71] and another study reported the negative impact of residents' assistance in laparoscopic gastric bypass surgery compared with fellow or attending level assistance [66]. Three studies reported negative outcomes for faculty cases, likely caused by selection bias [64,70,72].

Sixteen studies highlight the need for supervision by showing similar patient outcomes for supervised residents [52,54,56-58,68,73,75,76,79,80,85,86,90,94] or worse patient outcomes for unsupervised residents [60,84]. Many studies compared faculty with residents in small settings with small sample sizes. However, two studies reported patient sample sizes > 5,000 patients with similar patient outcomes for both faculty and supervised residents after adjusting for case-mix [56,85].

Seventeen studies evaluating the effect of specific training interventions (Table S2, Additional file 2)

The second category, of specific training interventions during residency training, comprised 17 studies that evaluated training programs on department level [95-111]. These specific training interventions were investigated in six medical [97,102,107-109,111], five surgical [98-100,103,105] and three intensive care [95,96,104] studies, and one each in psychiatry [110], anesthesiology [106] and all specialties [101]. Three randomized controlled trials were included in this category [101,107,111]. Two studies reported on audit and feedback intervention for residents providing diabetic care, which at first showed no differences in a small study [107], but improved diabetes mellitus control was found in a larger study [111]. Another randomized controlled trial in this category investigated a 40-hour role-play and feedback skills training program over eight months for residents, resulting in significantly higher patient satisfaction scores in the intervention group [101]. Seven of the 14 observational studies reported improved patient outcomes after implementation of a new training program [96,108], supervision [104,106] or simulator training and debriefing [95,97,105]. Six studies reported no difference in patient outcome [98-100,102,103,109] although one study did conclude that adherence to guidelines was better [109] and another study found that a redesign initiative reduced trainee workload and increased time for educational activities [102]. One study reported a drop in patient satisfaction after implementation of a training program in a psychiatric facility [110]. In conclusion, the studies in the category of specific training interventions showed improved or unchanged patient outcomes in the majority of studies and a drop in patient satisfaction in one study.

One study describing post-resident patient outcomes according to where the resident trained (Table S3, Additional file 2)

One retrospective cohort study evaluated patient outcomes in relation to where the practicing physician completed their residency training [112]. In the only study on this subject, 43% of accredited obstetrics and gynecology residency programs in the USA (4,124 physicians from 107 residency programs) were evaluated. Training programs could be ranked by the maternal complication rates occurring in the patients of their graduates.

Discussion

Main findings

Residents need to provide high quality and safe patient care both during and after their residency training. Based on our review that explicitly focused on the effects of residency training on patient outcomes, we could not answer unequivocally whether residency programs can differentiate in producing 'better doctors'. Only one study related clinicians' training background to patient outcomes of the care they provide today [112]. The other 96 included studies provided more insight into the effects of residency training on patient outcomes. The start of the academic year is not without risk, but individual progression of residents as well as progression through residency training had positive effects on patient outcomes. Compared with faculty, care provided by residents resulted mostly in similar patient outcomes, when dedicated supervision and additional operation time are provided. Overall, specific training situations yield equal or improved patient outcomes, with additional educational benefits, compared with the original training situation.

Limitations

Different sources of bias inherent to systematic reviews should be addressed. First, although we did not exclude non-English publications, all included studies were published in English, thus allowing for possible language bias. Publication bias is likely but difficult to assess in this heterogeneous body of evidence. Selection bias could be a possible limitation of our study design. However, two independent reviewers selected articles for inclusion by assessing the title and abstract and a limited selection bias in our approach is underlined by the fact that only nine articles had to be discussed after full-text retrieval.

Second, the studies included in our systematic review were too heterogeneous to perform a meta-analysis. To provide the reader with additional information alongside the narrative review of our results, the tables contain quantitative information on each individual study.

Third, we classified articles into categories based on their primary objective as described in the method section. We believe that the results are more easily read and understood with this categorization. Although we systematically assessed articles before assigning them into categories, categorization is always subject to discussion. However, only seven articles additionally reported on subjects of a category other than the one they were placed in [42,50,59,71,73,91,100]. Nonetheless, the results of these studies were consistent with the conclusions we drew for the categories they were not placed in.

Explanation of results

Considering the recent modernization efforts in medical education and public attention to patient safety, we expected to find a shift towards research using patient outcomes in medical education. Although in daily practice residency training is inextricably bound up with patient care, current literature fails to relate the two explicitly. A study reported in 2001 demonstrated that in leading medical education journals only 0.7% of articles used patient outcomes [113] as a measure of performance. The anticipated change in medical education literature could not be proven given the fact that none of the articles in our review were published in solely medical education journals. This demonstrates a lack of the use of patient outcomes in residency training research. The fact that all but two studies were published in clinical journals shows that the clinical community has an interest in medical education. In most studies we reviewed, patient safety was seen as an important motivator of research into patient outcomes of residents' participation in patient care.

Given that new doctors have to learn, the practical significance of the differences observed and their acceptability to patients and those involved in residency training need explanation. From the results in this review, it is clear that the start of the academic year and residents' (individual) progression through residency training have the most potential as targets to improve patient outcomes of residents' delivered care. The start of the academic year has previously been reported to be a time for extra vigilance [114]. Although no firm conclusions about the degree of risk posed to patients can be made based on existing literature, Young et al. did conclude that mortality increases and efficiency decreases in hospitals during academic changeover [114]. Furthermore, individual progression through residency training requires the residents' level of experience (or inexperience) to be supported with adequate faculty supervision [36,51]. In studies comparing patient outcomes of residents' delivered care with patient outcomes from care provided by faculty, supervision was emphasized as an important part of residency training [84]. Furthermore, enhanced clinical supervision was found to be associated with improved patient- and education-related outcomes in a recent systematic review by Farnan et al. [115]. The intensity of faculty supervision depends also on the level of competency of residents, but determining residents' competency is a complex and multifactor process [116]. Operative exposure is essential for competency development as a certain number of operations need to be performed to reach benchmark standards [35-41]. The correlation between the residents' seniority and improved patient outcomes provides evidence for the positive effect of residency training on patient outcomes [42-48,50,51]. Finally, the overall positive patient outcomes of residents' care during residency training show that, within the complex situation of residency training, patients can be safely cared for by residents who are well supervised and given the time to learn. This could be reassuring to patients who might oppose being treated by residents.

Implications for practice and research

As the above presented results show, there is a need for adequate supervision, patient selection, operative exposure, competency assessment and additional operation time to optimize residency training and ensure good patient outcomes. Faculty are primarily responsible for residents' learning during residency training and their support is essential, besides the additional time required. Therefore, it would be interesting to determine how much dedicated supervision and additional time is needed to ensure future health care workforce. The balance between the investment in teaching (time and money) and delivering care is something that is especially relevant for teaching hospitals. Although exploring differences between teaching hospitals and non-teaching hospitals was not the focus of this review, nine retrieved but excluded articles did compare them [117-125]. Overall, the teaching hospitals in these studies appeared to show better patient outcomes compared with non-teaching hospitals, predominantly on complex surgical procedures.

Clearly, the relationship between residency training and patient outcomes requires thorough investigation by both health care services and medical education researchers. In particular, studies evaluating the effect on patient outcomes after finishing residency training are currently lacking, since there was only one such study in this review [112]. Research on the training background of practicing physicians or prospective longitudinal follow-up of residents after finishing their residencies should be conducted. Although difficult, investigating causal factors that explain the relationship between residency training and patient outcomes can help us move forward in developing residency training. Furthermore, the effects of organizational aspects of residency training, like the impact of the teaching quality of the faculty on patient outcomes are lacking. Multicenter longitudinal databases of large student and resident cohorts exist, but they lack patient outcomes for individual doctors [126]. Cook et al. comprehensively describe longitudinal research databases facilitating the study of educational outcomes, taking patient outcomes into account. Intensifying collaboration between researchers and clinicians and encouraging the diversification of research perspectives should enrich clinical, health services and medical education research fields [127]. Therefore, both qualitative and quantitative research designs are needed to clarify which aspects of residency training best prepare doctors to deliver a high quality of care.

Conclusions

The majority of studies included in this systematic review drew attention to the fact that patient care appears safe and of equal quality when delivered by residents. A minority of results pointed to some negative patient outcomes from the involvement of residents. We, therefore, conclude that adequate supervision, room for extra operation time, and evaluation of and attention to the individual competence of residents throughout residency training could positively serve patient outcomes. What is currently lacking is knowledge on how, where and by whom doctors should be trained to deliver high quality care in their careers after residency training.

Competing interests

The authors declare that they have no competing interests.

Authors' contributions

RML and KMJMHL designed the study and had full access to all of the data in the study and take responsibility for the integrity of the data, the accuracy of the data analysis, and writing the manuscript. OAA and MJH designed the study, discussed the results and critically revised the manuscript. All authors read and approved the final version of the manuscript.

Pre-publication history

The pre-publication history for this paper can be accessed here:

http://www.biomedcentral.com/1741-7015/10/65/prepub

Supplementary Material

Additional file 1

Complete search string for seaching databases for relevant studies.

Click here for file (70.3KB, PDF)
Additional file 2

All tables (S1a to S1d, S2 and S3) containing detailed information on all individual studies.

Click here for file (340KB, PDF)

Contributor Information

Renée M van der Leeuw, Email: r.m.vanderleeuw@amc.nl.

Kiki MJMH Lombarts, Email: m.j.lombarts@amc.nl.

Onyebuchi A Arah, Email: arah@ucla.edu.

Maas Jan Heineman, Email: m.j.heineman@amc.nl.

Acknowledgements

Financial disclosure: OAA was supported by a VENI career grant (# 916.96.059) awarded by the Netherlands Organization for Scientific Research.

References

  1. Mission Statement of the Association of American Medical Colleges (AAMC) http://www.acgme.org/acWebsite/about/ab_ACGMEPoliciesProcedures.pdf Accessed February 12, 2011.
  2. Mission Statement of Accreditation Council for Graduate Medical Education (ACGME) http://www.acgme.org/acWebsite/about/ab_ACGMEPoliciesProcedures.pdf Accessed February 12, 2011. [DOI] [PubMed]
  3. Nasca TJ, Philibert I, Brigham T, Flynn TC. The next GME accreditation system - rationale and benefits. N Engl J Med. 2012;366(11):1051–1056. doi: 10.1056/NEJMsr1200117. [DOI] [PubMed] [Google Scholar]
  4. Berenguer CM, Ochsner MG Jr, Lord SA, Senkowski CK. Improving surgical site infections: using National Surgical Quality Improvement Program data to institute Surgical Care Improvement Project protocols in improving surgical outcomes. J Am Coll Surg. 2010;210:737–733. doi: 10.1016/j.jamcollsurg.2010.01.029. [DOI] [PubMed] [Google Scholar]
  5. Lipitz-Snyderman A, Steinwachs D, Needham DM, Colantuoni E, Morlock LL, Pronovost PJ. Impact of a statewide intensive care unit quality improvement initiative on hospital mortality and length of stay: retrospective comparative analysis. BMJ. 2011;342:d219. doi: 10.1136/bmj.d219. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. ACGME Outcome Project Timeline. http://www.acgme.org/outcome/project/timeline/TIMELINE_index_frame.htm Accessed February 12, 2011.
  7. Kogan JR, Holmboe ES, Hauer KE. Tools for direct observation and assessment of clinical skills of medical trainees: a systematic review. JAMA. 2009;302:1316–1326. doi: 10.1001/jama.2009.1365. [DOI] [PubMed] [Google Scholar]
  8. Miller A, Archer J. Impact of workplace based assessment on doctors' education and performance: a systematic review. BMJ. 2010;341:c5064. doi: 10.1136/bmj.c5064. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Tamblyn R, Abrahamowicz M, Dauphinee D, Wenghofer E, Jacques A, Klass D, Smee S, Blackmore D, Winslade N, Girard N, Du Berger R, Bartman I, Buckeridge DL, Hanley JA. Physician scores on a national clinical skills examination as predictors of complaints to medical regulatory authorities. JAMA. 2007;298:993–1001. doi: 10.1001/jama.298.9.993. [DOI] [PubMed] [Google Scholar]
  10. Wenghofer E, Klass D, Abrahamowicz M, Dauphinee D, Jacques A, Smee S, Blackmore D, Winslade N, Reidel K, Bartman I, Tamblyn R. Doctor scores on national qualifying examinations predict quality of care in future practice. Med Educ. 2009;43:1166–1173. doi: 10.1111/j.1365-2923.2009.03534.x. [DOI] [PubMed] [Google Scholar]
  11. Papadakis MA, Teherani A, Banach MA, Knettler TR, Rattner SL, Stern DT, Veloski JJ, Hodgson CS. Disciplinary action by medical boards and prior behavior in medical school. N Engl J Med. 2005;353:2673–2682. doi: 10.1056/NEJMsa052596. [DOI] [PubMed] [Google Scholar]
  12. Papadakis MA, Arnold GK, Blank LL, Holmboe ES, Lipner RS. Performance during internal medicine residency training and subsequent disciplinary action by state licensing boards. Ann Intern Med. 2008;148:869–876. doi: 10.7326/0003-4819-148-11-200806030-00009. [DOI] [PubMed] [Google Scholar]
  13. Review Manager (RevMan 5) software. http://ims.cochrane.org/revman Accessed October 1, 2010.
  14. Reed DA, Cook DA, Beckman TJ, Levine RB, Kern DE, Wright SM. Association between funding and quality of published medical education research. JAMA. 2007;298:1002–1009. doi: 10.1001/jama.298.9.1002. [DOI] [PubMed] [Google Scholar]
  15. Reed DA, Beckman TJ, Wright SM, Levine RB, Kern DE, Cook DA. Predictive validity evidence for medical education research study quality instrument scores: quality of submissions to JGIM's Medical Education Special Issue. J Gen Intern Med. 2008;23:903–907. doi: 10.1007/s11606-008-0664-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Alshekhlee A, Walbert T, DeGeorgia M, Preston DC, Furlan AJ. The impact of Accreditation Council for Graduate Medical Education duty hours, the July phenomenon, and hospital teaching status on stroke outcomes. J Stroke Cerebrovasc Dis. 2009;18:232–238. doi: 10.1016/j.jstrokecerebrovasdis.2008.10.006. [DOI] [PubMed] [Google Scholar]
  17. Anderson KL, Koval KJ, Spratt KF. Hip fracture outcome: is there a "July effect"? Am J Orthop (Belle Mead NJ) 2009;38:606–611. [PubMed] [Google Scholar]
  18. Ayas NT, Norena M, Wong H, Chittock D, Dodek PM. Pneumothorax after insertion of central venous catheters in the intensive care unit: association with month of year and week of month. Qual Saf Health Care. 2007;16:252–255. doi: 10.1136/qshc.2006.021162. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Bakaeen FG, Huh J, Lemaire SA, Coselli JS, Sansgiry S, Atluri PV, Chu D. The July effect: impact of the beginning of the academic cycle on cardiac surgical outcomes in a cohort of 70,616 patients. Ann Thorac Surg. 2009;88:70–75. doi: 10.1016/j.athoracsur.2009.04.022. [DOI] [PubMed] [Google Scholar]
  20. Dhaliwal AS, Chu D, Deswal A, Bozkurt B, Coselli JS, Lemaire SA, Huh J, Bakaeen FG. The July effect and cardiac surgery: the effect of the beginning of the academic cycle on outcomes. Am J Surg. 2008;196:720–725. doi: 10.1016/j.amjsurg.2008.07.005. [DOI] [PubMed] [Google Scholar]
  21. Englesbe MJ, Pelletier SJ, Magee JC, Gauger P, Schifftner T, Henderson WG, Khuri SF, Campbell DA. Seasonal variation in surgical outcomes as measured by the American College of Surgeons-National Surgical Quality Improvement Program (ACS-NSQIP) Ann Surg. 2007;246:456–462. doi: 10.1097/SLA.0b013e31814855f2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Englesbe MJ, Fan Z, Baser O, Birkmeyer JD. Mortality in medicare patients undergoing surgery in July in teaching hospitals. Ann Surg. 2009;249:871–876. doi: 10.1097/SLA.0b013e3181a501bd. [DOI] [PubMed] [Google Scholar]
  23. Finkielman JD, Morales J, Peters SG, Keegan MT, Ensminger SA, Lymp JF, Afessa B. Mortality rate and length of stay of patients admitted to the intensive care unit in July. Crit Care Med. 2004;32:1161–1165. doi: 10.1097/01.CCM.0000126151.56590.99. [DOI] [PubMed] [Google Scholar]
  24. Ford AA, Bateman BT, Simpson LL, Ratan RB. Nationwide data confirms absence of 'July phenomenon' in obstetrics: it's safe to deliver in July. J Perinatol. 2007;27:73–76. doi: 10.1038/sj.jp.7211635. [DOI] [PubMed] [Google Scholar]
  25. Garcia S, Canoniero M, Young L. The effect of July admission in the process of care of patients with acute cardiovascular conditions. South Med J. 2009;102(6):602–607. doi: 10.1097/SMJ.0b013e3181a2f8ca. [DOI] [PubMed] [Google Scholar]
  26. Highstead RG, Johnson LS, Street JH, Trankiem CT, Kennedy SO, Sava JA. July--as good a time as any to be injured. J Trauma. 2009;67:1087–1090. doi: 10.1097/TA.0b013e3181b8441d. [DOI] [PubMed] [Google Scholar]
  27. Inaba K, Recinos G, Teixeira PG, Barmparas G, Talving P, Salim A, Brown C, Rhee P, Demetriades D. Complications and death at the start of the new academic year: is there a July phenomenon? J Trauma. 2010;68:19–22. doi: 10.1097/TA.0b013e3181b88dfe. [DOI] [PubMed] [Google Scholar]
  28. Kestle JR, Cochrane DD, Drake JM. Shunt insertion in the summer: is it safe? J Neurosurg. 2006;105:165–168. doi: 10.3171/ped.2006.105.3.165. [DOI] [PubMed] [Google Scholar]
  29. Phillips DP, Barker GE. A July spike in fatal medication errors: a possible effect of new medical residents. J Gen Intern Med. 2010;25:774–779. doi: 10.1007/s11606-010-1356-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Resnick AS, Disbot M, Wurster A, Mullen JL, Kaiser LR, Morris JB. Contributions of surgical residents to patient satisfaction: impact of residents beyond clinical care. J Surg Educ. 2008;65:243–252. doi: 10.1016/j.jsurg.2008.01.005. [DOI] [PubMed] [Google Scholar]
  31. Schroeppel TJ, Fischer PE, Magnotti LJ, Croce MA, Fabian TC. The "July phenomenon": is trauma the exception? J Am Coll Surg. 2009;209:378–384. doi: 10.1016/j.jamcollsurg.2009.05.026. [DOI] [PubMed] [Google Scholar]
  32. Smith ER, Butler WE, Barker FG. Is there a "July phenomenon" in pediatric neurosurgery at teaching hospitals? J Neurosurg. 2006;105:169–176. doi: 10.3171/ped.2006.105.3.169. [DOI] [PubMed] [Google Scholar]
  33. Soltau TD, Carlo WA, Gee J, Gould J, Ambalavanan N. Mortality and morbidity by month of birth of neonates admitted to an academic neonatal intensive care unit. Pediatrics. 2008;122:e1048–e1052. doi: 10.1542/peds.2008-0412. [DOI] [PubMed] [Google Scholar]
  34. Yaghoubian A, de VC, Chiu V, Lee SL. "July effect" and appendicitis. J Surg Educ. 2010;67:157–160. doi: 10.1016/j.jsurg.2010.04.003. [DOI] [PubMed] [Google Scholar]
  35. Ascher-Walsh CJ, Capes T. An evaluation of the resident learning curve in performing laparoscopic supracervical hysterectomies as compared with patient outcome: five-year experience. J Minim Invasive Gynecol. 2007;14:719–723. doi: 10.1016/j.jmig.2007.07.005. [DOI] [PubMed] [Google Scholar]
  36. Fok WY, Chan LY, Chung TK. The effect of learning curve on the outcome of Caesarean section. BJOG. 2006;113:1259–1263. doi: 10.1111/j.1471-0528.2006.01060.x. [DOI] [PubMed] [Google Scholar]
  37. Grotenhuis BA, Wijnhoven BP, Jamieson GG, Devitt PG, Bessell JR, Watson DI. Defining a learning curve for laparoscopic cardiomyotomy. World J Surg. 2008;32:1689–1694. doi: 10.1007/s00268-008-9622-9. [DOI] [PubMed] [Google Scholar]
  38. Ibrahim S, Tay KH, Lim SH, Ravintharan T, Tan NC. Analysis of a structured training programme in laparoscopic cholecystectomy. Langenbecks Arch Surg. 2008;393:943–948. doi: 10.1007/s00423-007-0269-z. [DOI] [PubMed] [Google Scholar]
  39. Jang TB, Casey RJ, Dyne P, Kaji A. The learning curve of resident physicians using emergency ultrasonography for obstructive uropathy. Acad Emerg Med. 2010;17:1024–1027. doi: 10.1111/j.1553-2712.2010.00850.x. [DOI] [PubMed] [Google Scholar]
  40. Jensen CD, Gilliam AD, Horgan LF, Bawa S, Attwood SE. Day-case laparoscopic Nissen fundoplication. Surg Endosc. 2009;23:1745–1749. doi: 10.1007/s00464-008-0178-4. [DOI] [PubMed] [Google Scholar]
  41. Lin YY, Shabbir A, So JB. Laparoscopic appendectomy by residents: evaluating outcomes and learning curve. Surg Endosc. 2010;24:125–130. doi: 10.1007/s00464-009-0691-0. [DOI] [PubMed] [Google Scholar]
  42. Daetwiler S, Guller U, Schob O, Adamina M. Early introduction of laparoscopic sigmoid colectomy during residency. Br J Surg. 2007;94:634–641. doi: 10.1002/bjs.5638. [DOI] [PubMed] [Google Scholar]
  43. Filippi CG, Schneider B, Burbank HN, Alsofrom GF, Linnell G, Ratkovits B. Discrepancy rates of radiology resident interpretations of on-call neuroradiology MR imaging studies. Radiology. 2008;249:972–979. doi: 10.1148/radiol.2493071543. [DOI] [PubMed] [Google Scholar]
  44. Goldmann K, Hechtfischer C, Malik A, Kussin A, Freisburger C. Use of ProSeal laryngeal mask airway in 2114 adult patients: a prospective study. Anesth Analg. 2008;107:1856–1861. doi: 10.1213/ane.0b013e318189ac84. [DOI] [PubMed] [Google Scholar]
  45. Itani KM, DePalma RG, Schifftner T, Sanders KM, Chang BK, Henderson WG, Khuri SF. Surgical resident supervision in the operating room and outcomes of care in Veterans Affairs hospitals. Am J Surg. 2005;190:725–731. doi: 10.1016/j.amjsurg.2005.06.042. [DOI] [PubMed] [Google Scholar]
  46. Krugman SD, Garfunkel LC, Olsson JM, Ferrell CL, Serwint JR. Does quality of primary care vary by level of training in pediatric resident continuity practices? Acad Pediatr. 2009;9:228–233. doi: 10.1016/j.acap.2008.12.010. [DOI] [PubMed] [Google Scholar]
  47. Lee MT, Hu P, Hsi SC, Liu KY, Chao HM, Chen YQ. Mortality rates under the care of junior and senior surgery residents in a surgical intensive care unit/neurologic intensive care unit: a 5-year retrospective cohort study at Taoyuan Armed Forces General Hospital. J Crit Care. 2008;23:550–555. doi: 10.1016/j.jcrc.2008.03.039. [DOI] [PubMed] [Google Scholar]
  48. Lowe T, Paoloni R. Sutured wounds: factors associated with patient-rated cosmetic scores. Emerg Med Australas. 2006;18:259–267. doi: 10.1111/j.1742-6723.2006.00822.x. [DOI] [PubMed] [Google Scholar]
  49. Nieder AM, Meinbach DS, Kim SS, Soloway MS. Transurethral bladder tumor resection: intraoperative and postoperative complications in a residency setting. J Urol. 2005;174:2307–2309. doi: 10.1097/01.ju.0000181797.19395.03. [DOI] [PubMed] [Google Scholar]
  50. Palan J, Gulati A, Andrew JG, Murray DW, Beard DJ. The trainer, the trainee and the surgeons' assistant: clinical outcomes following total hip replacement. J Bone Joint Surg Br. 2009;91:928–934. doi: 10.1302/0301-620X.91B7.22021. [DOI] [PubMed] [Google Scholar]
  51. Sagarin MJ, Barton ED, Chng YM, Walls RM. Airway management by US and Canadian emergency medicine residents: a multicenter analysis of more than 6,000 endotracheal intubation attempts. Ann Emerg Med. 2005;46:328–336. doi: 10.1016/j.annemergmed.2005.01.009. [DOI] [PubMed] [Google Scholar]
  52. Acun Z, Cihan A, Ulukent SC, Comert M, Ucan B, Cakmak GK, Cesur A. A randomized prospective study of complications between general surgery residents and attending surgeons in near-total thyroidectomies. Surg Today. 2004;34:997–1001. doi: 10.1007/s00595-004-2857-7. [DOI] [PubMed] [Google Scholar]
  53. Auerbach JD, Lonner BS, Antonacci MD, Kean KE. Perioperative outcomes and complications related to teaching residents and fellows in scoliosis surgery. Spine (Phila Pa 1976) 2008;33:1113–1118. doi: 10.1097/BRS.0b013e31816f69cf. [DOI] [PubMed] [Google Scholar]
  54. Barnes JA, Bunce C, Olver JM. Simple effective surgery for involutional entropion suitable for the general ophthalmologist. Ophthalmology. 2006;113:92–96. doi: 10.1016/j.ophtha.2005.06.039. [DOI] [PubMed] [Google Scholar]
  55. Baron R, Sujendran V, Maynard N. Should oesophagectomies be performed by trainees? The experience from a single teaching centre under the supervision of one surgeon. Ann R Coll Surg Engl. 2008;90:305–309. doi: 10.1308/003588408X285919. [DOI] [PMC free article] [PubMed] [Google Scholar]
  56. Borowski DW, Ratcliffe AA, Bharathan B, Gunn A, Bradburn DM, Mills SJ, Wilson RG, Kelly SB. Northern Region Colorectal Cancer Audit Group (NORCCAG) Involvement of surgical trainees in surgery for colorectal cancer and their effect on outcome. Colorectal Dis. 2008;10:837–845. doi: 10.1111/j.1463-1318.2007.01465.x. [DOI] [PubMed] [Google Scholar]
  57. Chaudhuri N, Grayson AD, Grainger R, Mediratta NK, Carr MH, Soorae AS, Page RD. Effect of training on patient outcomes following lobectomy. Thorax. 2006;61:327–330. doi: 10.1136/thx.2005.046847. [DOI] [PMC free article] [PubMed] [Google Scholar]
  58. Chiu CC, Wei PL, Wang W, Chen RJ, Chen TC, Lee WJ, Huang MT. Role of appendectomy in laparoscopic training. J Laparoendosc Adv Surg Tech A. 2006;16:113–118. doi: 10.1089/lap.2006.16.113. [DOI] [PubMed] [Google Scholar]
  59. Drain AJ, Ferguson JI, Wilkinson S, Nashef SA. Blood, sweat, toil, and tears of surgical training. Part I: Blood. Asian Cardiovasc Thorac Ann. 2007;15:307–309. doi: 10.1177/021849230701500408. [DOI] [PubMed] [Google Scholar]
  60. Ekanem AD, Udoma EJ, Etuk SJ, Eshiet AI. Outcome of emergency Caesarean sections in Calabar, Nigeria: impact of the seniority of the medical team. J Obstet Gynaecol. 2008;28:198–201. doi: 10.1080/01443610801912329. [DOI] [PubMed] [Google Scholar]
  61. Ferguson J, Walker W. Developing a VATS lobectomy programme--can VATS lobectomy be taught? Eur J Cardiothorac Surg. 2006;29:806–809. doi: 10.1016/j.ejcts.2006.02.012. [DOI] [PubMed] [Google Scholar]
  62. Fischer CP, Hong JC. Early perioperative outcomes and pancreaticoduodenectomy in a general surgery residency training program. J Gastrointest Surg. 2006;10:478–482. doi: 10.1016/j.gassur.2006.01.010. [DOI] [PubMed] [Google Scholar]
  63. Gundevia Z, Whalley H, Ferring M, Claridge M, Smith S, Wilmink T. Effect of operating surgeon on outcome of arteriovenous fistula formation. Eur J Vasc Endovasc Surg. 2008;35:614–618. doi: 10.1016/j.ejvs.2007.11.018. [DOI] [PubMed] [Google Scholar]
  64. Harris IA, Lin C. Orthopaedic trauma surgery performed by unsupervised and supervised trainees: complication rates compared. J Orthop Surg (Hong Kong) 2007;15:264–266. doi: 10.1177/230949900701500302. [DOI] [PubMed] [Google Scholar]
  65. Hollander DA, Vagefi MR, Seiff SR, Stewart JM. Bacterial endophthalmitis after resident-performed cataract surgery. Am J Ophthalmol. 2006;141:949–951. doi: 10.1016/j.ajo.2005.11.055. [DOI] [PubMed] [Google Scholar]
  66. Hsu GP, Morton JM, Jin L, Safadi BY, Satterwhite TS, Curet MJ. Laparoscopic Roux-en-Y gastric bypass: differences in outcome between attendings and assistants of different training backgrounds. Obes Surg. 2005;15:1104–1110. doi: 10.1381/0960892055002374. [DOI] [PubMed] [Google Scholar]
  67. Jain PK, Hayden JD, Sedman PC, Royston CM, O'Boyle CJ. A prospective study of ambulatory laparoscopic cholecystectomy: training economic, and patient benefits. Surg Endosc. 2005;19:1082–1085. doi: 10.1007/s00464-004-2170-y. [DOI] [PubMed] [Google Scholar]
  68. Jeganathan R, Kinnear H, Campbell J, Jordan S, Graham A, Gavin A, McManus K, McGuigan J. A surgeon's case volume of oesophagectomy for cancer does not influence patient outcome in a high volume hospital. Interact Cardiovasc Thorac Surg. 2009;9:66–69. doi: 10.1510/icvts.2008.195461. [DOI] [PubMed] [Google Scholar]
  69. Kanakala V, Bawa S, Gallagher P, Woodcock S, Attwood SE, Horgan LF, Seymour K. Outcome of patients in laparoscopic training courses compared to standard patients. Surgeon. 2010;8:132–135. doi: 10.1016/j.surge.2009.10.030. [DOI] [PubMed] [Google Scholar]
  70. Karagounis A, Asimakopoulos G, Niranjan G, Valencia O, Chandrasekaran V. Complex off-pump coronary artery bypass surgery can be safely taught to cardiothoracic trainees. Interact Cardiovasc Thorac Surg. 2006;5:222–226. doi: 10.1510/icvts.2005.117465. [DOI] [PubMed] [Google Scholar]
  71. Kauvar DS, Braswell A, Brown BD, Harnisch M. Influence of resident and attending surgeon seniority on operative performance in laparoscopic cholecystectomy. J Surg Res. 2006;132:159–163. doi: 10.1016/j.jss.2005.11.578. [DOI] [PubMed] [Google Scholar]
  72. Khan OA, Lin PF, Chaudhuri J, Nelson M, Elsmore S, Walsh T. Training outcomes in colorectal cancer surgery in a district general hospital. Acta Chir Belg. 2008;108:503–507. doi: 10.1080/00015458.2008.11680275. [DOI] [PubMed] [Google Scholar]
  73. Lim M, O'Boyle CJ, Royston CM, Sedman PC. Day case laparoscopic herniorraphy. A NICE procedure with a long learning curve. Surg Endosc. 2006;20:1453–1459. doi: 10.1007/s00464-004-2265-5. [DOI] [PubMed] [Google Scholar]
  74. Marinis A, Stamatakis E, Tsaroucha A, Dafnios N, Anastasopoulos G, Polymeneas G, Theodosopoulos T. Safety and effectiveness of outpatient laparoscopic cholecystectomy in a teaching hospital: a prospective study of 110 consecutive patients. BMC Res Notes. 2010;3:207. doi: 10.1186/1756-0500-3-207. [DOI] [PMC free article] [PubMed] [Google Scholar]
  75. Messina A, Villa E, Mhagna Z, Dalla TM, Cirillo M, Brunelli F, Quaini E, Troise G. Medium-term results of systematic off-pump coronary surgery performed by trainee surgeons. Eur J Cardiothorac Surg. 2010;38:380–386. doi: 10.1016/j.ejcts.2010.02.029. [DOI] [PubMed] [Google Scholar]
  76. Nguyen CT, Hernandez AV, Gao T, Thomas AA, Jones JS. Office based vasectomy can be performed by supervised urological residents with patient pain and morbidity comparable to those of a staff surgeon procedure. J Urol. 2008;180:1451–1454. doi: 10.1016/j.juro.2008.06.047. [DOI] [PubMed] [Google Scholar]
  77. Ogiso S, Yamaguchi T, Hata H, Kuroyanagi H, Sakai Y. Introduction of laparoscopic low anterior resection for rectal cancer early during residency: a single institutional study on short-term outcomes. Surg Endosc. 2010;24:2822–2829. doi: 10.1007/s00464-010-1057-3. [DOI] [PubMed] [Google Scholar]
  78. Paajanen H, Varjo R. Ten-year audit of Lichtenstein hernioplasty under local anaesthesia performed by surgical residents. BMC Surg. 2010;10:24. doi: 10.1186/1471-2482-10-24. [DOI] [PMC free article] [PubMed] [Google Scholar]
  79. Phillips JS, Vowler SL, Salam MA. Is training in endoscopic sinus surgery detrimental to patient outcome? J Surg Educ. 2007;64:278–281. doi: 10.1016/j.jsurg.2007.07.003. [DOI] [PubMed] [Google Scholar]
  80. Pyle JW, Angobaldo JO, Bryant AK, Marks MW, David LR. Outcomes analysis of a resident cosmetic clinic: safety and feasibility after 7 years. Ann Plast Surg. 2010;64:270–274. doi: 10.1097/SAP.0b013e3181afa4e9. [DOI] [PubMed] [Google Scholar]
  81. Robinson RP. The impact of resident teaching on total hip arthroplasty. Clin Orthop Relat Res. 2007;465:196–201. doi: 10.1097/BLO.0b013e3181576006. [DOI] [PubMed] [Google Scholar]
  82. Robson AJ, Wallace CG, Sharma AK, Nixon SJ, Paterson-Brown S. Effects of training and supervision on recurrence rate after inguinal hernia repair. Br J Surg. 2004;91:774–777. doi: 10.1002/bjs.4540. [DOI] [PubMed] [Google Scholar]
  83. Sivaslioglu AA, Demir B, Dolen Y, Gelisen O, Haberal A. Residents performance in transobturator tape procedures for stress urinary incontinence. Eur J Obstet Gynecol Reprod Biol. 2007;134:259–261. doi: 10.1016/j.ejogrb.2006.12.011. [DOI] [PubMed] [Google Scholar]
  84. Skrekas T, Mochtar CA, Lagerveld BW, de Reijke TM, van Velthoven RF, Peltier A, de la Rosette JJ, Laguna MP. Mentor-initiated approach in laparoscopic radical prostatectomy. J Endourol. 2006;20:831–835. doi: 10.1089/end.2006.20.831. [DOI] [PubMed] [Google Scholar]
  85. Stoica SC, Kalavrouziotis D, Martin BJ, Buth KJ, Hirsch GM, Sullivan JA, Baskett RJ. Long-term results of heart operations performed by surgeons-in-training. Circulation. 2008;118:S1–S6. doi: 10.1161/CIRCULATIONAHA.107.756379. [DOI] [PubMed] [Google Scholar]
  86. Teoh AY, Chong CN, Wong J, Lee KF, Chiu PW, Ng SS, Lai PB. Routine early laparoscopic cholecystectomy for acute cholecystitis after conclusion of a randomized controlled trial. Br J Surg. 2007;94:1128–1132. doi: 10.1002/bjs.5777. [DOI] [PubMed] [Google Scholar]
  87. Wan IY, Thung KH, Hsin MK, Underwood MJ, Yim AP. Video-assisted thoracic surgery major lung resection can be safely taught to trainees. Ann Thorac Surg. 2008;85:416–419. doi: 10.1016/j.athoracsur.2007.10.009. [DOI] [PubMed] [Google Scholar]
  88. Wong K, Duncan T, Pearson A. Unsupervised laparoscopic appendicectomy by surgical trainees is safe and time-effective. Asian J Surg. 2007;30:161–166. doi: 10.1016/S1015-9584(08)60016-0. [DOI] [PubMed] [Google Scholar]
  89. Woolson ST, Kang MN. A comparison of the results of total hip and knee arthroplasty performed on a teaching service or a private practice service. J Bone Joint Surg Am. 2007;89:601–607. doi: 10.2106/JBJS.F.00584. [DOI] [PubMed] [Google Scholar]
  90. Filippi CG, Meyer RE, Cauley K, Nickerson JP, Burbank HN, Johnson JM, Linnell GJ, Alsofrom GF. The misinterpretation rates of radiology residents on emergent neuroradiology magnetic resonance (MR) angiogram studies: correlation with level of residency training. Emerg Radiol. 2010;17:45–50. doi: 10.1007/s10140-009-0820-z. [DOI] [PubMed] [Google Scholar]
  91. Meyer RE, Nickerson JP, Burbank HN, Alsofrom GF, Linnell GJ, Filippi CG. Discrepancy rates of on-call radiology residents' interpretations of CT angiography studies of the neck and circle of Willis. AJR Am J Roentgenol. 2009;193:527–532. doi: 10.2214/AJR.08.2169. [DOI] [PubMed] [Google Scholar]
  92. Jaturapatporn D, Dellow A. Does family medicine training in Thailand affect patient satisfaction with primary care doctors? BMC Fam Pract. 2007;8:14. doi: 10.1186/1471-2296-8-14. [DOI] [PMC free article] [PubMed] [Google Scholar]
  93. Paul M, Nobel K, Goodman S, Lossy P, Moschella JE, Hammer H. Abortion training in three family medicine programs: resident and patient outcomes. Fam Med. 2007;39:184–189. [PubMed] [Google Scholar]
  94. Gueret G, Billard V, Bourgain JL. Fibre-optic intubation teaching in sedated patients with anticipated difficult intubation. Eur J Anaesthesiol. 2007;24:239–244. doi: 10.1017/S0265021506001475. [DOI] [PubMed] [Google Scholar]
  95. Barsuk JH, Cohen ER, Feinglass J, McGaghie WC, Wayne DB. Use of simulation-based education to reduce catheter-related bloodstream infections. Arch Intern Med. 2009;169:1420–1423. doi: 10.1001/archinternmed.2009.215. [DOI] [PubMed] [Google Scholar]
  96. Buckley JD, Joyce B, Garcia AJ, Jordan J, Scher E. Linking residency training effectiveness to clinical outcomes: a quality improvement approach. Jt Comm J Qual Patient Saf. 2010;36:203–208. doi: 10.1016/s1553-7250(10)36033-8. [DOI] [PubMed] [Google Scholar]
  97. Edelson DP, Litzinger B, Arora V, Walsh D, Kim S, Lauderdale DS, Vanden Hoek TL, Becker LB, Abella BS. Improving in-hospital cardiac arrest process and outcomes with performance debriefing. Arch Intern Med. 2008;168:1063–1069. doi: 10.1001/archinte.168.10.1063. [DOI] [PubMed] [Google Scholar]
  98. Ganzer R, Rabenalt R, Truss MC, Papadoukakis S, Do M, Blana A, Straub M, Denzinger S, Wieland WF, Burchardt M, Herrmann T, Stolzenburg JU. Evaluation of complications in endoscopic extraperitoneal radical prostatectomy in a modular training programme: a multicentre experience. World J Urol. 2008;26:587–593. doi: 10.1007/s00345-008-0291-y. [DOI] [PubMed] [Google Scholar]
  99. Isaacson G. Six sigma tympanostomy tube insertion: achieving the highest safety levels during residency training. Otolaryngol Head Neck Surg. 2008;139:353–357. doi: 10.1016/j.otohns.2008.06.012. [DOI] [PubMed] [Google Scholar]
  100. Lien HH, Huang CC, Liu JS, Shi MY, Chen DF, Wang NY, Tai FC, Huang CS. System approach to prevent common bile duct injury and enhance performance of laparoscopic cholecystectomy. Surg Laparosc Endosc Percutan Tech. 2007;17:164–170. doi: 10.1097/SLE.0b013e31804d44bb. [DOI] [PubMed] [Google Scholar]
  101. Lienard A, Merckaert I, Libert Y, Bragard I, Delvaux N, Etienne AM, Marchal S, Meunier J, Reynaert C, Slachmuylder JL, Razavi D. Transfer of communication skills to the workplace during clinical rounds: impact of a program for residents. PLoS One. 2010;5:e12426. doi: 10.1371/journal.pone.0012426. [DOI] [PMC free article] [PubMed] [Google Scholar]
  102. McMahon GT, Katz JT, Thorndike ME, Levy BD, Loscalzo J. Evaluation of a redesign initiative in an internal-medicine residency. N Engl J Med. 2010;362:1304–1311. doi: 10.1056/NEJMsa0908136. [DOI] [PubMed] [Google Scholar]
  103. Pandey S, Slawik S, Cross K, Soulsby R, Pullyblank AM, Dixon AR. Laparoscopic appendicectomy: a training model for laparoscopic right hemicolectomy? Colorectal Dis. 2007;9:536–539. doi: 10.1111/j.1463-1318.2007.01244.x. [DOI] [PubMed] [Google Scholar]
  104. Papadimos TJ, Hensely SJ, Duggan JM, Hofmann JP, Khuder SA, Borst MJ, Fath JJ. Intensivist supervision of resident-placed central venous catheters decreases the incidence of catheter-related blood stream infections. Patient Saf Surg. 2008;2:11. doi: 10.1186/1754-9493-2-11. [DOI] [PMC free article] [PubMed] [Google Scholar]
  105. Rogers GM, Oetting TA, Lee AG, Grignon C, Greenlee E, Johnson AT, Beaver HA, Carter K. Impact of a structured surgical curriculum on ophthalmic resident cataract surgery complication rates. J Cataract Refract Surg. 2009;35:1956–1960. doi: 10.1016/j.jcrs.2009.05.046. [DOI] [PubMed] [Google Scholar]
  106. Schmidt UH, Kumwilaisak K, Bittner E, George E, Hess D. Effects of supervision by attending anesthesiologists on complications of emergency tracheal intubation. Anesthesiology. 2008;109:973–977. doi: 10.1097/ALN.0b013e31818ddb90. [DOI] [PubMed] [Google Scholar]
  107. Thomas KG, Thomas MR, Stroebel RJ, McDonald FS, Hanson GJ, Naessens JM, Huschka TR, Kolars JC. Use of a registry-generated audit, feedback, and patient reminder intervention in an internal medicine resident clinic--a randomized trial. J Gen Intern Med. 2007;22:1740–1744. doi: 10.1007/s11606-007-0431-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  108. Warm EJ, Schauer DP, Diers T, Mathis BR, Neirouz Y, Boex JR, Rouan GW. The ambulatory long-block: an accreditation council for graduate medical education (ACGME) educational innovations project (EIP) J Gen Intern Med. 2008;23:921–926. doi: 10.1007/s11606-008-0588-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  109. Wayne DB, Didwania A, Feinglass J, Fudala MJ, Barsuk JH, McGaghie WC. Simulation-based education improves quality of care during cardiac arrest team responses at an academic teaching hospital: a case-control study. Chest. 2008;133:56–61. doi: 10.1378/chest.07-0131. [DOI] [PubMed] [Google Scholar]
  110. Woo BK, Ma AY. Psychiatric inpatient care at a county hospital before and after the inception of a university-affiliated psychiatry residency program. J Psychiatr Pract. 2007;13:343–348. doi: 10.1097/01.pra.0000290675.53170.54. [DOI] [PubMed] [Google Scholar]
  111. Ziemer DC, Doyle JP, Barnes CS, Branch WT Jr, Cook CB, El-Kebbi IM, Gallina DL, Kolm P, Rhee MK, Phillips LS. An intervention to overcome clinical inertia and improve diabetes mellitus control in a primary care setting: Improving Primary Care of African Americans with Diabetes (IPCAAD) 8. Arch Intern Med. 2006;166:507–513. doi: 10.1001/archinte.166.5.507. [DOI] [PubMed] [Google Scholar]
  112. Asch DA, Nicholson S, Srinivas S, Herrin J, Epstein AJ. Evaluating obstetrical residency programs using patient outcomes. JAMA. 2009;302:1277–1283. doi: 10.1001/jama.2009.1356. [DOI] [PubMed] [Google Scholar]
  113. Prystowsky JB, Bordage G. An outcomes research perspective on medical education: the predominance of trainee assessment and satisfaction. Med Educ. 2001;35:331–336. doi: 10.1046/j.1365-2923.2001.00910.x. [DOI] [PubMed] [Google Scholar]
  114. Young JQ, Ranji SR, Wachter RM, Lee CM, Niehaus B, Auerbach AD. "July effect": impact of the academic year-end changeover on patient outcomes: a systematic review. Ann Intern Med. 2011;155:309–315. doi: 10.7326/0003-4819-155-5-201109060-00354. [DOI] [PubMed] [Google Scholar]
  115. Farnan JM, Petty LA, Georgitis E, Martin S, Chiu E, Prochaska M, Arora VM. A systematic review: the effect of clinical supervision on patient and residency education outcomes. Acad Med. 2012. [DOI] [PubMed]
  116. Dijksterhuis MG, Voorhuis M, Teunissen PW, Schuwirth LW, ten Cate OT, Braat DD, Scheele F. Assessment of competence and progressive independence in postgraduate clinical training. Med Educ. 2009;43:1156–1165. doi: 10.1111/j.1365-2923.2009.03509.x. [DOI] [PubMed] [Google Scholar]
  117. Bumm K, Heupel J, Bozzato A, Iro H, Hornung J. Localization and infliction pattern of iatrogenic skull base defects following endoscopic sinus surgery at a teaching hospital. Auris Nasus Larynx. 2009;36:671–676. doi: 10.1016/j.anl.2009.03.004. [DOI] [PubMed] [Google Scholar]
  118. Dimick JB, Cowan JA Jr, Colletti LM, Upchurch GR Jr. Hospital teaching status and outcomes of complex surgical procedures in the United States. Arch Surg. 2004;139:137–141. doi: 10.1001/archsurg.139.2.137. [DOI] [PubMed] [Google Scholar]
  119. Kohn GP, Galanko JA, Meyers MO, Feins RH, Farrell TM. National trends in esophageal surgery--are outcomes as good as we believe? J Gastrointest Surg. 2009;13:1900–1910. doi: 10.1007/s11605-009-1008-2. [DOI] [PubMed] [Google Scholar]
  120. Ku TS, Kane CJ, Sen S, Henderson WG, Dudley RA, Cason BA. Effects of hospital procedure volume and resident training on clinical outcomes and resource use in radical retropubic prostatectomy surgery in the Department of Veterans Affairs. J Urol. 2008;179:272–278. doi: 10.1016/j.juro.2007.08.149. [DOI] [PubMed] [Google Scholar]
  121. Lee SL, Yaghoubian A, de Virgilio C. A multi-institutional comparison of pediatric appendicitis outcomes between teaching and nonteaching hospitals. J Surg Educ. 2011;68:6–9. doi: 10.1016/j.jsurg.2010.08.003. [DOI] [PubMed] [Google Scholar]
  122. Meguid RA, Brooke BS, Chang DC, Sherwood JT, Brock MV, Yang SC. Are surgical outcomes for lung cancer resections improved at teaching hospitals? Ann Thorac Surg. 2008;85:1015–1024. doi: 10.1016/j.athoracsur.2007.09.046. [DOI] [PubMed] [Google Scholar]
  123. Meguid RA, Brooke BS, Perler BA, Freischlag JA. Impact of hospital teaching status on survival from ruptured abdominal aortic aneurysm repair. J Vasc Surg. 2009;50:243–250. doi: 10.1016/j.jvs.2009.01.046. [DOI] [PMC free article] [PubMed] [Google Scholar]
  124. Taylor SF, Gerhardt RT, Simpson MP. An association between Emergency Medicine residencies and improved trauma patient outcome. J Emerg Med. 2005;29:123–127. doi: 10.1016/j.jemermed.2005.01.014. [DOI] [PubMed] [Google Scholar]
  125. Yaghoubian A, de Virgilio C, Lee SL. Appendicitis outcomes are better at resident teaching institutions: a multi-institutional analysis. Am J Surg. 2010;200:810–813. doi: 10.1016/j.amjsurg.2010.07.028. [DOI] [PubMed] [Google Scholar]
  126. Haan CK, Edwards FH, Poole B, Godley M, Genuardi FJ, Zenni EA. A model to begin to use clinical outcomes in medical education. Acad Med. 2008;83:574–580. doi: 10.1097/ACM.0b013e318172318d. [DOI] [PubMed] [Google Scholar]
  127. Cook DA, Andriole DA, Durning SJ, Roberts NK, Triola MM. Longitudinal research databases in medical education: facilitating the study of educational outcomes over time and across institutions. Acad Med. 2010;85:1340–1346. doi: 10.1097/ACM.0b013e3181e5c050. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Additional file 1

Complete search string for seaching databases for relevant studies.

Click here for file (70.3KB, PDF)
Additional file 2

All tables (S1a to S1d, S2 and S3) containing detailed information on all individual studies.

Click here for file (340KB, PDF)

Articles from BMC Medicine are provided here courtesy of BMC

RESOURCES