Pathology Residents Comprise Inspection Team for a CAP Self-Inspection

Abstract

We report our experience at the University of Florida in which residents and fellows served as the inspection team for a College of American Pathologists (CAP) self-inspection. We aimed to determine whether the CAP self-inspection could serve as a learning opportunity for pathology residents and fellows. To prepare for the inspection, we provided a series of 4 lunchtime seminars covering numerous laboratory management topics relating to inspections and laboratory quality. Preparation for the inspection began approximately 4 months prior to the date of the inspection. The intent was to simulate a CAP peer inspection, with the exception that the date was announced. The associate residency program director served as the team leader. All residents and fellows completed inspector training provided by CAP, and the team leader completed the team leader training. A 20 question pre- and posttest was administered; additionally, an anonymous survey was given after the inspection. The residents’ and fellows’ posttest scores were an average of 15% higher than on the pretest (P < .01). The surveys as well as subjective comments were overwhelmingly positive. In conclusion, the resident’s and fellow’s experience as an inspector during a CAP self-inspection was a useful tool to learn accreditation and laboratory management.

Introduction

Participating in clinical laboratory inspections is an important component of a pathologist’s career, but incorporation of this activity into residency training is challenging. Accreditation and laboratory inspections may be briefly covered during laboratory management didactic seminars, which are often limited.¹ A survey of 54 program directors published in 2011 reported that 52% of programs devoted less than 20 hours of lecture time to laboratory management topics over the course of a 4-year training cycle.² However, about half of the programs surveyed reported use of a real or mock inspection as a teaching tool; and 96% reported that this was “moderately” or “highly” effective. The role of the resident in these inspections was not described and may have included sending 1 or 2 residents to a peer inspection, which our institution has also done in the past. This extraordinarily valuable experience is not broadly available for all trainees.

The importance of the accreditation process is emphasized by our professional organizations. Several milestones determined by The Accreditation Council for Graduate Medical Education and The American Board of Pathology relate to accreditation or could be achieved by participating in a self-inspection (see Table 1).³ One example is Systems-Based Practice 2: Lab Management: Regulatory and compliance: Explains, recognizes, summarizes and is able to apply regulatory and compliance issues (AP/CP), in which one attribute of a level-4 resident is “Participates in an internal or external laboratory inspection.” Of note, the American Board of Pathology cites “inspection and accreditation process” as a topic covered on the clinical pathology board exam.⁴ The Resident In-Service Exam administered by the American Society for Clinical Pathology⁵ also includes questions on these topics.

Table 1.

The Accreditation Council for Graduate Medical Education (ACGME) Milestones That Pertain to Accreditation or Could be Achieved by Participating in a Self-inspection.

Systems-based practice 2: Laboratory management: regulatory and compliance: explains, recognizes, summarizes, and is able to apply regulatory and compliance issues (AP/CP)
1	Knows that laboratories must be accredited
3	Understands the components of laboratory accreditation and regulatory compliance (Clinical Laboratory Improvement Amendments [CLIA] and others), either through training or through experience
3	Completes laboratory inspector training
4	Participates in an internal or external laboratory inspection
5	Participates in or leads internal or external laboratory inspections
SBP4: Laboratory management: quality, risk management, and laboratory safety: explains, recognizes, summarizes, and is able to apply quality improvement, risk management, and safety issues (AP/CP)
2	Understands the concept of a laboratory quality management plan
3	Interprets quality data and charts and trends
3	Demonstrates a knowledge of proficiency testing and its consequences
4	Reviews and analyzes proficiency testing results
SBP6: Laboratory management: technology assessment: explains, recognizes, summarizes, and is able to apply technology assessment (AP/CP)
2	Understands the need for a process in implementing new technology
3	Understands and describes the process of implementing new technology
4	Participates in new instrument and test selection, verification, implementation, and validation (including reference range analysis) and maintains a portfolio of participation in these experiences
PROF3: Professionalism: demonstrates responsibility and follow-through on tasks (AP/CP)
ICS1: Intradepartmental interactions and development of leadership skills: displays attitudes, knowledge, and practices that promote safe patient care through team interactions and leadership skills within the laboratory (AP/CP)
1	Demonstrates respect for and willingness to learn from all members of the pathology team
2	Works effectively with all members of the pathology team
2	Attends laboratory, departmental, or institutional committee meetings
3	Understands own role on the pathology team and flexibly contributes to team success through a willingness to assume appropriate roles as needed
3	Utilizes mechanisms for conflict resolution and helps to defuse and ameliorate conflict
3	Participates in groups to accomplish goals
4	Helps to organize the pathology team to facilitate optimal communication and coeducation among members

Methods

First, we designed our objectives for this project. By the time of graduation, all pathology residents and fellows will be able to:

1. participate in a College of American Pathologists (CAP) inspection;

2. explain the logistical aspects of an inspection including preparation and day of procedures;

3. demonstrate how to obtain and analyze evidence for each checklist item; and

4. demonstrate how to cite deficiencies and follow-up on citations.

Our hospital system includes approximately 1000 inpatient beds, 1 off-site emergency department, and numerous outpatient clinics. Our core laboratories process approximately 10 million tests per year, and our histology laboratory processes approximately 75 000 cases per year. Our institution is comprised of the UF Health Medical Lab and the UF PathLabs, which is a laboratory run by the Department of Pathology, Immunology, and Laboratory Medicine. Both sections have laboratories in the hospital and in an off-site building. Additionally, the hospital maintains laboratories in the clinic building and at a free-standing emergency department. There are 6 Clinical Laboratory Improvement Amendments (CLIA) licenses and a total of 76 CAP checklists. We have 16 residents (4 each per year) and 7 fellows (2 gastrointestinal pathology, 2 hematopathology, 1 neuropathology, 1 surgical pathology, and 1 cytopathology).

After determining an appropriate inspection date, the associate residency program director with clinical pathology focus (APD-CP) and the laboratory quality assurance coordinator worked together to design a 4-contact hour curriculum which took place over 4 lunch periods starting 11 weeks prior to the inspection date. See Figure 1 for the topics covered at each session. The associate program director with anatomic pathology focus (APD-AP) also participated by doing the team leader checklist and organizing the trainees at the off-site laboratory location. Residents and fellows were assigned various areas of the laboratory and were provided their checklists 7 weeks prior to the inspection. Checklists were assigned based on postgraduate year (PGY) level, rotations completed, and interests. Many checklists were grouped and assigned to 2 or more trainees, in some cases to a fellow and a resident to facilitate peer education. For example, a PGY-1 who had completed a 1-month transfusion medicine rotation was assigned to inspect the blood bank along with a PGY-4 resident. The core laboratory checklists were assigned to the 3 remaining PGY-1s (1 for hematology, 1 chemistry, and 1 urinalysis), and all were assigned to work together on the corresponding all-common checklist. The hematopathology fellows were assigned flow cytometry and human leukocyte antigen (HLA) labs, each gastrointestinal pathology fellow was assigned a laboratory general checklist, a PGY-4 who will be doing an informatics fellowship was assigned to the molecular lab, and so forth.

Figure 1.

Preparations for the inspection began approximately 2.5 months prior to the inspection. This diagram shows topics covered in 4 one-hour lunch meetings and required assignments due after each meeting.

At the first lunch meeting, a 20-question examination was administered. The same examination along with a 10-question anonymous survey was administered following the conclusion of the inspection. The questions are provided in the results section.

On the day of the inspection, processes were followed as if we were a visiting peer inspection team to give the trainees the complete experience. A welcome session started at 8:00 am. All pathologists, administrative staff, and laboratory supervisors and managers were invited to attend. A member of the host institution gave a welcome speech, and the team leader stated the goals and general process of the inspection as suggested in the CAP training modules. Host personnel and the inspection team introduced themselves. This session lasted approximately 30 minutes, after which the team began their inspections. The team leader had interviews scheduled with the hospital’s vice president of operations at 10:00 am and the chief medical officer at 11:30 am. Lunch was provided for the team at both the hospital location and the off-site laboratory location at 12 pm. The team leader (APD-CP) was present at the hospital location, and the APD-AP was present at the off-site laboratory. Each team member reported their status to these leaders who talked via a phone appointment at 12:15 pm. Arrangements were made for inspectors who were done or almost done to help those with a substantial amount of checklist items remaining.

A presummation meeting was held at 3:00 pm, which allowed ample time to discuss questions about possible deficiencies. Each inspector filled out the inspector summation reports (pink and yellow sheets) as appropriate. The summation meeting began at 4:30 pm with the same audience as those who attended the welcome meeting. The summation commenced with a brief speech by the team leader stating that the inspection team felt very welcome, the goals of the inspection were met, and many positive findings were seen in the laboratories. Then each inspector thanked the laboratory staff that he or she worked with and read aloud any deficiencies or recommendations that were found. Finally, a celebratory dinner party was held at the team leader’s house for all inspectors and laboratory staff.

Results

Table 2 shows the results of the pre- and posttest. The percentage of respondents answering with the correct option is shown. A total of 23 residents and fellows took the tests. The average score on the pretest was 64.6% (range 40-90, standard deviation 13.3) and the posttest was 80% (range 55-95, standard deviation 8.7, P < .01 by t test).

Table 2.

Quiz Questions with Percent of Participants Selecting the Correct Answer before (Pretest) and after (Posttest) the Inspection.

Question	Correct Answer	% Correct Pretest (n = 23)	% Correct Posttest (n = 23)
How often are CAP-accredited laboratories inspected? Once when they initiate patient testing Every year Every 2 years Every 4 years	C	73.9	95.7
Who performs CAP inspections? A designated individual employed by CAP A team of individuals from a peer institution The hospital’s Quality Officer Pathologists who volunteer to be on the CAP checklist committee	B	69.6	91.3
What does CLIA stand for? Clinical Laboratory Improvement Association Clinical Laboratory Inspection Association Clinical Laboratory Improvement Amendments Clinical Laboratory Improvement Act	D	52.2	39.1
Which of the following are required prior to an inspection? CAP team member training Creation of a CAP checklist Choosing a date with the director of the laboratory that is being inspected A visit to the laboratory by the inspection team’s leader	A	65.2	95.7
Which type of CLIA certificate does UF Health Shands and UF Path laboratories obtain? Provider performed microscopy Waived Compliance Accreditation	D	87.0	91.3
Which type of CLIA certificate is required for a family practice office performing waived tests (such as group A strep) and microscopy (such as vaginal wet preps)? Provider performed microscopy Waived Compliance Accreditation	A	34.8	52.2
What is reported to CAP when the laboratory does not meet a CAP standard? Delinquency Violation Deficiency Grade F Misdemeanor	C	69.6	91.3
Which level of complexity is anatomic pathology considered? Waived Moderate High Provider performed microscopy	C	78.3	91.3
What agency determines the level of complexity for each laboratory test? FDA WHO CAP CMS CLIA	A	26.1	47.8
What year was the last major revision to CLIA law? 1970 1988 1995 2000 2008 2016	B	34.8	60.9
What are the components of a CAP checklist item? Regulation, note, evidence of compliance Directive, discussion, recommendations Quality control, inspector name, laboratory manager name Guideline, evaluation score, endorsement	A	65.2	82.6
How is the date of the laboratory inspection determined? A specific date determined by the CAP Any date determined by the inspection team Any date determined by the inspection team within a CAP assigned 3-month window of time, excluding any “black out dates” issued by the laboratory being inspected A specific date chosen by the laboratory director	C	91.3	95.7
What is the difference between a phase 1 and phase 2 citation? Phase 2 are considered directly linked to possible patient harm and therefore the response must include evidence that an action plan was implemented Phase 1 are considered basis for ceasing operations of a laboratory Phase 2 are considered recommendations and do not require an action plan Phase 1 were corrected on site	A	73.9	95.7
What should be done if an inspector does not believe that the regulation was met but the laboratory supervisor disagrees? The citation should be listed The regulation should be corrected on site The inspector is considered the expert so the citation should be listed The supervisor can call CAP and discuss it with them during the inspection	D	56.5	39.1
How long do most inspections take? 1-3 days 1-2 weeks 1 month 1 year	A	95.7	100
What is the relationship between CAP and CLIA? They act entirely independently All laboratories are required to be accredited by CAP in alpha-lower to be CLIA accredited CAP has deemed status which means that CAP can inspect on behalf of CLIA CLIA standards are more stringent than CAP’s so most laboratories choose to be CAP accredited	C	43.5	69.6
How many checklists types are used during inspections and what are they? Four: laboratory-specific (technical), all common, lab general, team leader Three: laboratory general, universal technical, all lab-specific (technical) Two: laboratory general, team leader Five: laboratory general, team leader, all common, CAP general, universal technical	A	47.8	100
How is a CAP inspection concluded? The inspector does not disclose what he/she is going to report to CAP to the laboratory manager as this should be confidential A summation where all citations are read out loud A party including the inspecting team and personnel of the laboratory that is being inspected The team leader does a “final walk” through the entire laboratory	B	69.6	100
Which of the following do not require a response reported to CAP? Phase 1 Phase 2 Recommendations and corrected on site Delinquencies and violations	C	82.6	95.7
What should occur if an inspection team cannot find an individual with expertise in an area of the laboratory that they must inspect (ie, HLA testing)? Cross-train an individual for several months so that he or she is proficient enough to inspect that area Try to obtain an individual from a neighboring institution but if they cannot, contact CAP for a CAP-assigned inspector Exclude that area from the inspection and rely on the institution’s self-inspection The team leader will perform the inspection of that area	B	69.6	56.5

Abbreviation: CAP, College of American Pathologists.

Figure 2 shows the questions and results of our 10-question anonymous survey. Table 3 shows the comments that were obtained on the anonymous survey.

Figure 2.

Statements and results of the anonymous survey.

Abbreviations: CAP indicates College of American Pathologists; N/A, not applicable.

Table 3.

Residents’ and Fellows’ Comments from the Post-inspection Survey.

“I hope we can continue this participation in future years. Having hands-on experience was very educational and I think doing more things like this will help me feel more prepared to become a lab director.”

“Great experience for me! Would like to know more tips from experienced inspectors regarding how to successfully conduct an inspection.”

“This is a very good learning experience. I learned a lot, not only in lab management but also for the preparation of the board exam.”

“Excellent experience. Thank you for organizing it.”

“Overall this was a great experience and much preferred over lab management lectures. Admittedly, I did not feel competent to perform a lab inspection leading up to the scheduled date. However, on inspection day, I was surprisingly confident in my abilities to inspect our labs. Although we uncovered a significant number of deficiencies, I believe it was beneficial to the lab as well as the inspection team and it was an educational experience.”

“I think residents were spread out too thin. Next time, we should try to restrict beginners to two checklists only.”

“This overall was a great experience. I feel having 2 of these throughout training would be very helpful. We should continue doing these.”

“I think the CAP online training is too technical to be easily digested. Upon actually doing the inspection, I quickly realized what was necessary and the task became very straightforward. Overall this was a brilliant decision/idea and I greatly appreciate having participated. My confidence with lab management is also much greater.”

“Everything was good and I learned a lot. I was not really looking forward to doing the inspection, but now I feel like I can do this again and I have gained important skills. One note: All of the CAP training and the meetings made a lot of us scared—we could not understand the terminology. The most helpful things were the videos on the CAP website because the terminology was confusing to us.”

“Adding a lab tech/member or someone with more experience for review day of inspection would be nice, though not necessary.”

Discussion

The self-inspection is an ideal opportunity for residents to actively participate in the inspection process. Residents gained knowledge of the inspection process, laboratory accreditation, and technical processes of the laboratories they were assigned to inspect as evidenced by significantly higher posttest scores compared to pretest scores. Additionally, anonymous survey data showed that the residents and fellows enjoyed the experience. Most residents admitted to having very little knowledge about these subjects prior to the inspection. This was our first attempt and the initial exposure to any sort of inspection-related activities for several of our residents. If a program conducted this activity every other year, each resident would participate twice during their training. Residents in PGY-3 and -4 and fellows will be able to teach the PGY-1 and -2 residents and share their knowledge from previous inspections. The only caveat to this schedule is that some fellows come here for only 1 year of training; if that happens to be the off year, they would not gain this experience. Fellows who are here on the year of a self-inspection but graduated from a different program that does not offer experience or training in inspections would need to be recognized, as they may have the technical knowledge to mentor a junior resident but not working knowledge of the inspection process.

When members of our laboratory traveled as a peer inspection team last year, we invited our 2 chief residents to join as inspectors. Although they certainly enjoyed that experience and it helped them understand inspections, they stated that they wished that had completed this self-inspection and related curriculum prior to the peer inspection as a way to increase their confidence in their role and competence of compliance. Numerous junior faculty members in our department commiserated with that experience, feeling as if they had been “thrown in” to the inspection environment in the past and expressed a desire for hands-on experience while in training. Although the inspector training module provided by the CAP offers an excellent educational experience, being an inspector during a self-inspection complements the knowledge needed to serve as a peer inspector. Additionally, trainees had the opportunity to get to know our quality manager and were able to ask questions about accreditation in general or specific to the laboratories they were inspecting.

This process proved to be educational for more than just the trainees. The associate residency program directors took the leadership roles for this inspection—one acting as the overall team leader and the other participating as team lead for the off-site laboratory. Both had extremely limited training in inspections prior to this experience and saw this as a unique learning opportunity not only for their trainees but to achieve these milestones for themselves. This was an extraordinarily valuable experience for these junior faculty members and both report that they now feel much more prepared to serve on an inspection team or as a team leader in the future. Additionally, the act of being inspected is not nearly as daunting.

Given the novitiate of the team leaders, it was important to identify an experienced team member or point person for consultation. Our neuropathology fellow at the time of this inspection, Dr Meggen Walsh, served on the CAP Resident Forum Executive Committee, has participated in numerous inspections in the past and thus served as a valuable resource and role model for trainees. During the presummation conference, the group discussed numerous possible deficiencies and recommendations. Dr Walsh, along with the team leader, decided on the outcome of the deliberation. It was very helpful to have someone with experience in our group. If a program does this activity for the first time and does not have anyone with this degree of experience, it would be appropriate to ask a senior faculty member to participate especially during the working lunch and presummation where numerous questions will come up, even if this deviates from the simulation of a peer inspection.

The self-inspection also afforded an opportunity for other staff and faculty to “practice.” The lab quality assurance coordinator worked with both the laboratory team and the inspection team during the inspection, which was difficult but possible due to careful planning. The hospital administrator and chief medical officer were both interviewed by the team leader. The hospital administrator has many years of experience and was therefore able to provide feedback to the team leader. The chief medical officer had been in the position for only 4 weeks at the time of the inspection, so answering questions related to the laboratory and pathologists allowed him to gain familiarity of the interview and inspection process.

Importantly, residents and laboratory staff collaborated on a shared goal. While these groups of people work together on situations such as call questions and troubleshooting when there are problems, communication is often limited to phone and e-mail. The inspection gave them a chance to work together in person, in the laboratory. Interprofessional teamwork is an important skill for residents to learn. The nature of a self-inspection performed by trainees brings about a slight professional challenge: The trainees were tasked with citing their own mentors, teachers, and friends. Following the inspection, one laboratory director was vividly upset about a deficiency and stated that he did not think it was appropriate, and this evoked apprehension and self-doubt in the trainee who inspected this laboratory. While the appropriate route would be to contest this with the CAP, after discussion with the lab director we chose to strike it as “corrected on site” in order to avoid potential disagreement between the trainee and his mentor. In this regard, we felt that our self-inspection was formal enough to feel real yet informal enough to foster an atmosphere of teaching. The laboratory staff members were encouraged to give the trainee the time to search for deficiencies, but if one was missed, they were welcome to point it out as a teaching opportunity. We deviated from the simulation of a peer inspection process as little as possible, but we felt that announcing the date was necessary for logistical reasons. Also, we encouraged the residents to discuss checklist items with laboratory staff prior to the inspection. Since some of the laboratory tests and processes were foreign to them, we felt that allowing this time would be of educational value.

Future directions include examining resident’s performance on the laboratory management sections of the Resident In-Service Examination (RISE). Additionally, we can compare results (ie, deficiencies) of subsequent peer inspections as a means of assessing our performance as self-inspectors. It will be difficult to assess the true benefit of this activity, which is achieving a high level of comfort and expertise while on an inspection as a practicing pathologist, involving several years following graduation from our program. However, from the results of this preliminary study we are confident that these goals will be met.