Residents Make Their Lists and Program Directors Check Them Twice: Reviewing Case Logs

The Challenge

Many Residency Review Committees (RRCs) require that residents complete case and procedure logs. For instance, the requirements for general surgery state “All… must enter their operative experience concurrently during each year of the residency in the ACGME [Accreditation Council for Graduate Medical Education] case log system.”^1(p15) This is not only a requirement for most procedural-based specialties but also a requirement for other specialties in which program directors and institutional officials must ensure that residents have “a satisfactory patient care experience that includes: sufficient numbers of patients, diversity of diagnoses, and acuity/complexity of the patients.”^2(p13) The intent of these requirements is to ensure adequate clinical and procedural experiences for residents before independent practice, with the caveat that completion of x number of cases or procedures does not ensure competence in a given individual.

The purpose of this article is to provide an annual template for program directors and those involved with internal reviews to use when reviewing case logs, as part of an overall program evaluation.

What Is Known

As the completion of some prescribed number of cases does not ensure clinical or technical competency in a particular resident, the practice of reviewing case logs is part of program evaluation, not resident assessment. The minimum case or procedure numbers have been set by the various RRCs to represent a minimum number of cases to ensure most trainees can practice safely and independently. To date, there is little empirical evidence underlying the minimum numbers; those numbers have been determined by panels of experts, such as RRC members. The bottom line is that current minimum numbers are the best available target and programs are held to those numbers by their RRC.³

For program evaluations and improvements to be made, the data on which they are made need to be as accurate as possible. Timely data entry is likely to increase accuracy and should be stressed to residents as a professional requirement similar to the timely record-keeping they will maintain in practice. Automating and simplifying data entry as much as possible may also increase accuracy. Rowe and colleagues⁴ demonstrated that computer-based logging increased compliance and accuracy. The findings from a study of case logs in anesthesiology suggest that, compared with data obtained from an automated record, manual case-log data entry resulted in up to 5% of cases not being captured by at least half of the residents.⁵ Therefore, efforts to automate data entry may result in more accurate logs and more time for residents to devote to nonclerical matters.

A second consideration in case log accuracy is the pressure felt by programs to meet requirements because programs are judged by that achievement. A recent survey of plastic surgery chief residents suggests that although, on occasion, residents were asked to recode cases to fit a certain requirement (presumably legitimately), none reported being asked to falsify their logs to meet requirements. In that study, most residents felt that their logs exactly or very closely represented their actual experience (S. J. Kasten, MD, MHPE, FACS, oral communication, January 9, 2012, on recent survey's unpublished data).

Common Program Requirements

Program directors must “provide each resident with documented semiannual evaluation of performance with feedback.”^6(p3) An important part of that review should be the review of case logs. This allows faculty to comment on a residents' experience to date; more important, the review will ensure that gaps are identified early, and actionable plans are developed during the ensuing 6 months. For programs in which more emphasis is traditionally placed on procedural numbers, more frequent checking is wise. Program directors or chief residents may assess case logs on an ongoing basis, particularly for categories or residents for whom there is a concern about “low numbers.”

The program should review not only individual trainee logs but also the overall log for the program. This review should be in concert with the annual program review to ensure the programs' goals and objectives are being met as well as the RRC requirements.⁷ An annual review in concert with a midpoint accreditation internal review will highlight issues early and allow for interventions before an RRC site visit⁸ (table).

TABLE.

Program Activities Relating to Case Logs

In addition to reviewing procedure logs, program directors must ensure trainees demonstrate competency in the domain of professionalism. Professionalism includes proper, timely, and accurate documentation. In assessing professionalism, one strategy is to evaluate compliance with logging procedures and cases. However, program directors should develop clear goals and objectives for that documentation before using that documentation for trainee evaluation;⁹ and they may want to randomly pull a few patient records to verify the accuracy of the case log data.

Finally, if a resident is receiving poor marks in a particular clinical problem, case logs can be used to determine whether the resident needs remediation versus more clinical experience in the area.

Rip Out action items

Program directors must:

1. Ensure residents and faculty understand the importance of tracking their experiences

2. Review procedure/case logs at least semiannually at the time of the trainee's formative review

3. View recording of procedure/case numbers and associated details as a professionalism competency

4. Review procedure/case logs as a critical aspect of the institutional internal review process

5. Share case and procedure data with faculty and residents regularly to facilitate plans to resolve inequities between trainees, as needed

What Can You Start TODAY

1. Have faculty and residents review the requirements for procedures and various case experiences.

2. Discuss with faculty and residents the importance of recording procedure and case numbers.

3. Develop professional standards in your own program to ensure compliance with recording by trainees.

4. If your RRC does not offer guidelines for adequate experience, elicit feedback from faculty and peers.

What Can You Do LONG TERM

1. Develop a competency approach to assess the quality as well as number of resident procedures.¹⁰

2. Before an RRC visit, the internal review process should examine your logs in detail to ensure a breadth of experience and note any obvious gaps.

3. At least annually, obtain trainee feedback about their experiences and any gaps in their experiences.

4. Contact your specialty organization for national procedural and experience guidelines.

5. Advocate with your specialty organization to ensure that the list of requirements reflects current practice.