Potency Analyses Provide Insight Into Student Aseptic Compounding Technique Errors

Abstract

Objective. To determine whether direct observational scores were predictive of the potency of pharmacy students’ compounded sterile preparations (CSPs) and to identify any misunderstandings students had regarding individual aseptic technique steps.

Methods. P1 students performed aseptic techniques during three observational encounters separated by two weeks. Students’ performances were evaluated using an observation-based rubric and were subject to potency analysis. The encounters were transferring a drug solution from a vial, an ampule, and a reconstituted powder to intravenous (IV) bags.

Results. The mean potency of the diphenhydramine (vial) and lidocaine (ampule) met the ±10% goal of expected potency. These results were significantly different from those of the ampicillin (reconstitution) encounter, which was outside the goal. The percentage of students meeting the potency goal was 59.3% for the diphenhydramine, 80.3% for the lidocaine, and 50.4% for the ampicillin encounters. The observation scores were significantly different between all three encounters. There were no correlations between the observational scores and the potency for any encounter regardless of whether or not the student met the goal potency. Although their observation scores were acceptable, up to 50% of students did not meet the potency goal for each of the three encounters.

Conclusion. The potency data provided the critical insight that P1 students were not adequately trained to account for pressurization when manipulating vials using aseptic compounding processes. The results suggest that both observation scores and potency analysis should be part of an overall assessment of student ability to compound sterile preparations.

INTRODUCTION

Pharmacy compounding is defined as the art and science of preparing personalized medications for patients.¹ Compounding allows pharmacists to work with patients and prescribers to customize a medication and meet a patient’s specific need. The recognition of the importance of compounding in schools of pharmacy has been highlighted by the Accreditation Council for Pharmacy Education (ACPE), American Association of Colleges of Pharmacy (AACP), and the National Association of Boards of Pharmacy (NABP).^2-4 However, none of their standards and requirements provide a pedagogical methodology that should be used to teach compounding skills in the classroom or the cleanroom.⁵

Pharmaceutical compounding intersects with patient care in both nonsterile and sterile preparations. In the arena of sterile compounding, adverse events, including death, have occurred when sterile compounds are contaminated with microbial organisms.^6,7 The United States Pharmacopeia (USP) sets standards for sterile compounding (USP General Chapter <797>, https://www.usp.org/compounding/general-chapter-797), and most state boards of pharmacy require pharmacists to comply with these standards. As such, providing instruction in sterile compounding is an essential responsibility of all schools of pharmacy.

The scope, frequency, timing, and methods by which pharmacy schools implement curricula on aseptic technique are specific to each program.⁸ Surveys of various schools of pharmacy have reported the frequency and depth of compounded sterile preparation education.^9-12 Hellmus and colleagues found that all of the 48 schools responding to their survey included some type of instruction on compounded sterile preparations (CSPs).¹¹ Only 70% allowed students to compound on their own (rather than in groups or not at all), and only 21% offered a standalone course on this topic. Students having experience in sterile compounding prior to entering school and students undergoing repeated assessment in school during a short period of time have been reported to show improved aseptic compounding skills.^5,13-16 As a corollary to experience and repeated testing within a small period of time, the issue of how frequently aseptic technique training should be included within an entire curriculum and the optimal points to include such training have also been discussed.^8,17

Sterile compounding courses are often short in duration and expensive to conduct which leads to inadequate training of pharmacy students in the use of aseptic techniques. Consequently, most sterile compounding instruction is limited to one semester and commonly taught in the P1 year. This creates the possibility that upon entering the workforce, graduates are three years removed from their aseptic technique training and may no longer possess the basic knowledge or skills. Ely and Birnie found that students did not retain nonsterile compounding skills for more than one year.¹⁸ Numerous course designs and evaluation schemes have attempted to maximize compounding instruction outcomes.^5,19 Newer technologies have also made use of simulations to optimize instruction time. Penguin Innovations (West Lafayette, IN) is developing a virtual interactive cleanroom simulation as an instructional additive or alternative for students to become familiar with the complexities of a cleanroom environment and to gain an overview of aseptic manipulation techniques.^20,21 Table 1 displays a compilation of published references for assessing sterile compounding instruction.^{8,12-17,19-34}

Table 1.

Assessment Methods Used in Schools of Pharmacy to Evaluate Aseptic Technique

Both course design and assessment criteria are important influences on the outcomes students achieve in a sterile compounding course. In the actual compounding of CSPs, there are three central factors that must be controlled to minimize microbial contamination: engineering controls, personnel control, and process control. Of the three factors, schools of pharmacy primarily focus on the teaching of process control, ie, aseptic techniques. Aseptic compounding techniques are typically evaluated by a variety of methods because educators like to address the differences in student learning preferences and needs. Direct observation of students’ aseptic techniques during the compounding process is a common evaluation method. However, direct observation only provides a partial evaluation of all of the aspects of sterile compounding, and is only useful if the observations are done one-on-one by personnel who are competent in the preparation and evaluation of compounded sterile preparations (CSPs).

One methodology for assessing students’ compounding competency of CSPs that is rarely described in the pharmacy education literature is the quantitative analysis of the potency of the active pharmaceutical ingredient (API).³⁵ Potency analysis is an objective measure of competency and does not require additional expenditures for compounding supplies. Furthermore, if students perform the analysis themselves, such an assessment requires negligible additional faculty time. Pharmaceutical analysis is required from a regulatory standpoint when assigning beyond-use dates (BUDs) for CSPs that are longer than the default dates given in USP <797>. However, two factors might limit a school-wide adoption of such methodology: first, the initial cost of the instrumentation; and second, faculty or instructor expertise in carrying out of valid pharmaceutical analysis methods.

A survey of compounding curricula in schools of pharmacy found that most survey respondents cited direct observation as their primary assessment approach, while only a small percentage (8%) of institutions used a quantitative method to evaluate student preparations.³⁶ Only one study has been published where potency analysis was part of an assessment scheme that also involved media fill analysis.¹⁷ Direct observation may be a good surrogate method of assessment for schools that do not have the ability to carry out analyses of student preparations, as observation does provide general feedback to students about the commonly missed or incorrectly performed manipulations required in aseptic techniques.

At the University of North Carolina Eshelman School of Pharmacy, the assessment of students’ compounding abilities using a multitude of assessment tools has remained a top priority in the curriculum for several years. One of the predominant assessment methods has been to determine the preparations’ potencies by pharmaceutical analysis.^1,37 The analyses have provided profound insight into pharmacy students’ abilities to accurately compound preparations. The analytical work has also enhanced student understanding of the quality of compounded preparations and encouraged them to formulate better preparations. As concluded in a previous study by Jih and Shrewsbury, having students analyze their own preparations increased their self-confidence and their competence in their compounding ability.³⁸

This study was undertaken to determine several answers to questions that are not currently reported in the pharmacy education literature. The first question we addressed was whether observational scores are good predictors of acceptable potency results. The second question was what percentage of students would successfully prepare a CSP within an acceptable potency range on the first attempt. The third question was whether support should be given to a recommendation that both observational scores and potency analysis be part of an overall assessment plan for pharmacy students compounding CSPs. The study was approved by the University of North Carolina Biomedical IRB Committee.

METHODS

First-professional year (P1) pharmacy students compounded three different sterile preparations during three separate observational encounters. The first encounter assessed proficiency in withdrawing and transferring 40 mg of diphenhydramine from a 1 mL (50 mg/mL) vial into a 50-mL normal saline (NS) intravenous (IV) bag. The second encounter occurred two weeks later and assessed student proficiency in breaking a 2 mL lidocaine (15 mg/mL) ampule, withdrawing 30 mg of lidocaine with a filter needle, switching to a 21 or 23 gauge needle, and injecting the contents into a 50 mL NS IV bag. The third encounter occurred two weeks following the second encounter and assessed the reconstitution of ampicillin (125 mg/vial) with 5 mL of NS, followed by withdrawal and injection of 110 mg of ampicillin into a 50 mL NS IV bag.

Prior to each observational encounter, students completed two in-class instruction sessions led by a teaching assistant (TA). In the first session, the TA taught proper aseptic techniques and an explanation of the packaging and devices to be used during the semester. In the second session, students practiced the aseptic techniques to be used specifically in the first encounter. Students were also given a copy of the observational rubric that would be used to assess their technique (Appendix 1). The observational rubric was modified after the first encounter to better characterize the techniques required for encounters two and three (Appendix 2). The TAs did not receive any specialized instruction on how to grade the students during each encounter. However, they were instructed to give either full credit or zero credit for each action item in the observational rubric. The majority of the TAs were licensed pharmacy residents from schools other than the Eshelman School of Pharmacy, with the remainder (three out of 12) being second- or third-year Eshelman School of Pharmacy students who demonstrated competency in the course during their first year of pharmacy school.

During each observational encounter, students were evaluated on their ability to interpret an IV order, gather and place supplies in the laminar airflow workstation (LAFW), and compound a sterile preparation using aseptic techniques within a suggested 15-minute time constraint. Garbing, hand hygiene, and cleaning of the LAFW were assessed previously and not included in these evaluations. Students were evaluated individually by a different TA from the one they had been assigned to for the semester. The TAs were not assessed for grading uniformity. Between encounters, students were given the opportunity to practice for the next encounter on their own time. Supplies were available during regular school hours for additional practice and the course instructor or TAs were available to provide further instruction.

Immediately following each observational encounter, students withdrew sufficient volume twice from their IV bag to produce two samples of 3 mL and placed the samples in methacrylate cuvettes. The samples were read on a spectrophotometer at the following wavelengths for each API: diphenhydramine (vial) at 270 nm, lidocaine (ampule) at 275 nm, ampicillin (reconstitution) at 265 nm. Each student recorded the absorbance of both samples on a log sheet. A laboratory staff member calculated the potency of each sample using the appropriate linearity assay data. The average of the two potency determinations was calculated. The goal average potency was defined as ±10% of intended concentration. The standard curves, which were prepared immediately prior to the evaluations, contained data points that matched the expected concentrations, and were corrected for the average overfill in a 50 mL normal saline IV bag (approximately 7 mL).

The observational scores from the rubrics and the potency values from the students’ self-analyses were organized in Excel by encounter and student name. All negative or zero potency values were excluded from further statistical analysis because a negative or zero value could indicate that no API was transferred into the normal saline bag or that the student incorrectly analyzed their preparation. In the entire study, this omission accounted for 2% of the total number of data pairs. The potency values and observational scores were matched per student, and a z test for two means was used in all comparisons to determine significance as the variance of each population size was known and each population size was greater than 30. A p=.05 was set as the level of significance.

RESULTS

Data were organized by type of observational encounter and by endpoint criteria. “All students” refers to all of the students who participated in the encounter that had a non-negative, nonzero potency value. “Students meeting potency endpoint” refers to students whose samples were within ±10% of expected potency, which was the long-standing laboratory standard.

The descriptive statistics for potency data are shown in Table 2. The mean potency of all students for the diphenhydramine and lidocaine CSPs met the goal of ±10% of expected potency. However, the mean potency for the ampicillin CSP fell below this mark. The mean potency of the students’ diphenhydramine and lidocaine CSPs did not differ significantly from one another. However, the potencies of both the diphenhydramine and lidocaine CSPs differed significantly from that of the ampicillin CSP. This suggested that the ampicillin reconstitution encounter was relatively more difficult for students.

Table 2.

Potency Results for All Encounters to Gain Insight Into Student Aseptic Technique Errors

The percentages of students that compounded their preparations within the potency endpoint were 59.3% for the diphenhydramine (withdrawn from a vial), 80.3% for the lidocaine (withdrawn from an ampule), and 50.4% for the ampicillin (reconstituted from a powder). There were no significant differences between the groups’ averaged potencies. The standard deviation (SD) of the three encounters was the same order of magnitude, suggesting a “consistent” number of students might have experienced difficulties with these skills between each encounter. Both the kurtosis and skewness of the data improved in this group compared to that for all students, indicating a more normal distribution of potency values. There was also a substantial decrease in the range of potencies.

The descriptive statistics of the observational scores for each encounter are shown in Table 3. The number of observational results differed slightly from the number of potency results because some TAs returned the rubrics to students prior to the data collection. All possible mean comparisons for all students and students meeting the potency endpoint were significantly different. The standard deviations and ranges for both groups were similar in magnitude, which suggests that the variances were similar regardless of potency value. However, a trend towards improvement in the observational score was seen as students in both groups progressed throughout the encounters.

Table 3.

Observational Scores for All Encounters to Gain Insight Into Student Aseptic Technique Errors

Table 4 shows the breakdown of points awarded to all students for the performance of each action item listed on the aseptic technique rubrics (Appendix 1 and Appendix 2). Of these items, action item 2, “Assembles necessary supplies; syringe and needle sizes are appropriate to task,” had the lowest SD for all encounters, whereas, action item 6, “Withdraws contents of vial or glass ampule appropriately,” had the highest standard deviation for all encounters. Action item 2 was likely the least difficult skill to perform, while action item 6 was likely the most difficult. Action item 6 may correspond best with potency values as this action item was the only point where API volume was manipulated. The same results were seen with the observational data of the students meeting the potency endpoint, ie, action item 2 had the lowest SD and action item 6 had the highest SD (data not shown).

Table 4.

Observational Scores by Action Item for All Encounters to Gain Insight Into Student Aseptic Technique Errors

Correlations of observational scores and potency values were constructed in each encounter to determine if a relationship existed such that the students with the higher observational scores also compounded preparations that met the potency endpoint. The correlation plots for all students and students meeting the potency endpoint are shown in Figures 1 and 2, respectively. The correlation equations of fit and the corresponding R² value are shown in Table 5. There was no correlation between the observational score and the measured potency in any encounter for either group. Correlation coefficients suggested that less than 10% of the variation in the measured potency could be attributed to the observational score.

Figure 1.

Correlations Between Observational Scores and Potency Values for all Students

Figure 2.

Correlations Between Observational Scores and Potency Values for Students Meeting Potency Endpoint

Table 5.

Correlation Equations of Observational Scores and Potency Values for All Encounters

DISCUSSION

The ability to work in a LAFW using aseptic techniques is a fundamental skill that pharmacy students must have, whether they compound CSPs themselves or evaluate someone else’s compounding procedures. Direct observation is the most common evaluation tool used to assess students. One methodology rarely used in the direct assessment of a student’s compounding proficiency is the quantitative determination of the API’s potency. A central objective of this study was to determine if sample analysis provided better insights into the students’ ability to compound acceptable CSPs.

The results showed a lack of correlation between the measured potency and the observational values. One consideration that may have overshadowed the correlation predictions was the use of manufactured products. The concentrations of the APIs in commercially available products meet USP standards: the diphenhydramine hydrochloride injection and ampicillin for injection were to be ±10% of label, and the lidocaine hydrochloride injection should have been ±5%. Varying the potency values by these percentages to simulate the possible contribution variability in the API concentration did not improve the lack of correlations between the measured potency and observational values.

The other explanation that would account for the potency variability was a failure of the student to withdraw the correct volume of solution from the vial or ampule before transferring the drug solution to the IV bag. This would occur if the student withdrew the incorrect volume or if the volume was correct based on the placement of the plunger within the syringe barrel, but the student had not adequately removed the air bubbles. With the ampicillin reconstitution encounter, variability could have occurred at two points: if the wrong volume of diluent was added to the vial to be reconstituted; and if the wrong volume of reconstituted solution was removed from the vial before transferring the drug solution to the IV bag.

Also, students may have had difficulties knowing how to correctly determine (by calculation) the volume of solution to be withdrawn from the vial. The diphenhydramine withdrawal and ampicillin reconstitution encounters were designed such that the students did not withdraw the entire vial contents for the preparation. The lidocaine encounter required the student to withdraw the total volume contained in the ampule. Even for students who could not determine the correct volume to withdraw from the ampule, they may have known to withdraw approximately all of the contents of the ampule into their syringe. It may be prudent to consider a more complicated medication order for the ampule encounter. Obtaining student feedback on the difficulties they had with each encounter would be helpful to elucidate this point.

In each encounter, action item 6 had the greatest variability (ie, SD) of all of the observational scores. Consideration was given to the possibility that action item 6 might directly correlate with potency variability. The difficulty with such a correlation analysis in this study was that action item 6 had only two possible values: no score (ie, 0) or the maximum score for that item. Therefore, the scale of the observational scores was limited compared to the scale in potency values, and a correlation would be of limited validity. Thus, the mean potency for the two possible observational scores was determined for both groups (ie, all students and students meeting the potency endpoint) in all three encounters. In the group of all students, the mean potency values for the diphenhydramine and lidocaine encounters were not significantly different regardless of the respective action item 6 score. However, the mean potency values for the diphenhydramine and lidocaine encounters were different from the ampicillin reconstitution encounter for both possible action item 6 scores. This was the same pattern as seen for this group using the total observational score instead of action item 6. For the students meeting the potency endpoint, the only statistical difference between the two observational scores was found in the ampicillin encounter. This was not the pattern seen when using the total observational score. This result suggested that action item 6 may be a key indicator in the students’ inability to obtain the correct volume of drug solution to transfer to the IV bag during the ampicillin reconstitution encounter. However, which of the possible scenarios the student failed to execute correctly cannot be determined from this result.

Action item 6 required the student to “withdraw the contents of the vial or ampule appropriately,” which again confirmed that this one procedural step was the most critical step of the entire process if the goal was to have an acceptably potent CSP. Evaluators likely observed errors in the volume of drug solution withdrawn during the assessment of students. However, prior to this study, the significance of action item 6 was not appreciated in the procedural scheme of the whole encounter.

One specific aspect of volume withdrawal that students found difficult was the pressurization of vials. For the diphenhydramine withdrawal and the ampicillin reconstitution encounters, students had to work with the pull-back pressure within the vial to withdraw the drug solution if the vial was not correctly pressurized. However, the broken ampule was an open system and thus removed the potential for a pull-back pressure on the syringe. During the process of adding normal saline to the ampicillin powder for reconstitution, students may have inadvertently added additional air to the powder vial or failed to withdraw the same volume of air after adding the diluent, which would have “over pressurized” the vial. Then, when the students again added air to help withdraw the reconstituted ampicillin solution, the vial may have been pressurized even more and created a favorable gradient for the solution to “leak” out of the vial through the hole the needle had made in the vial septum.

The other factor in working with vials that students often fail to appreciate is that the volume of air going into and out of the vial must equal the volume of solution moving into or out of the vial. Many times students will add the appropriate volume of solution into the vial, but will just wait for the syringe plunger to “equalize the air pressure” and then remove the needle-syringe from the vial. Because the plunger equalizes at a smaller air volume than the solution volume that was added, the vial became over pressurized, which again led to the possibility of the preparation “leaking” out of the vial. One approach to avoid this common problem is to use the technique of “milking” the vial in order to avoid the large momentary excesses in air pressurization that typically lead to preparation leakage. This technique repeatedly adds small increments of air to the vial and removes an equivalent small increment of solution. This technique minimizes pressure changes inside the vial while reconstituting the powder with a diluent or removing reconstituted drug solution.

One objective for the course was for students to be able to perform aseptic technique skills at a level that an observer would judge as “appropriate.” However, the addition of potency determinations to the aseptic technique encounters indicated that many students could not meet the goal of compounding a preparation that would deliver the right amount of drug to a patient. Potency analysis would not fully examine all aspects of student aseptic technique, but its use does provide additional insight that eludes the limits of visual observation. However, the study indicated that two action items should be added to the observational rubric: verify that the solution volume in the syringe is correct after the removal of air bubbles, and verify that the dosage to be withdrawn is correct.

The authors understand that having laboratories analyze student-compounded preparations as part of the assessment of students’ aseptic technical skills would place a significant burden on the course instructor and the school. First, there would be the nominal cost of the additional syringes and needles needed to remove the samples from the CSP IV bags for analysis. In this study, linearity standard curves were used for the spectrophotometric analysis, so standards were prepared that only required a few additional IV bags and vials and ampules. The analysis process for each encounter required an initial development time, but the time to prepare for each encounter’s laboratory was minimal. The students analyzed their own preparations, which required that they remove a sample from the IV bag and place the solution into a cuvette, so the analysis time was minimal for students as well.

However, while the observational scores highlighted a specific action item that was a major deficiency in the aseptic technique skills needed to compound acceptable CSPs, the potency analysis provided very critical and specific insights into which manipulative steps were being performed incorrectly. For example, the data clearly demonstrated that students had improved potency when completing the ampule encounter compared to that achieved in the other two encounters, which involved working with pressure inside of vials. The data also reveal that the reconstitution encounter required students to understand that they needed to withdraw an equivalent volume of air from the vial after adding the diluent to avoid problems in subsequent steps because of excess pressure in the vial.

These outcomes showcase the importance of using both observational scores and potency analysis to judge student competency in compounding CSPs. The action item from the observational scores will indicate the difficulty or problem the student is experiencing during the compounding process, and the potency analysis will show how to overcome the difficulty or problem. The synergy of the two assessment methods provides a sound pedagogical approach to ensure compounding competency for students.

Appendix 1.

Observational Rubric Used in the First (Vial) Encounter

Appendix 2.

Observational Rubric Used With Second (Ampule) and Third (Reconstitution) Encounters