A High-Stakes Assessment Approach to Applied Pharmacokinetics Instruction

Abstract

Objectives

To identify instructional and assessment problems leading to pharmacy students' failure to retain pharmacokinetics abilities into the experiential year and develop an instructional methodology and abilities-based assessment tool to address the problem.

Methods

Pharmacokinetic instructional methods were assessed and an abilities-based assessment tool was developed and utilized as a requirement for curricular progression. Both the instructional methodology and the assessment tool were evaluated using abilities-based outcomes and faculty surveys.

Assessment

Both instructional methods and assessment methods improved student pharmacokinetic skill performance in the direct patient-care environment.

Conclusions

Continual assessment, modification, and implementation of teaching methods and the adoption of a high-stakes abilities-based assessment impacted student learning in a problem-based, integrated course.

INTRODUCTION

The doctor of pharmacy (PharmD) degree offered at the Auburn University Harrison School of Pharmacy (AUHSOP) has evolved from a postgraduate program to a track-in program (2 years prepharmacy, 2 years BS pharmacy, 2 years PharmD), to a first professional degree doctoral program (2 years prepharmacy and 4 years pharmacy). AUHSOP has developed a comprehensive approach to instruction based upon recommendations of the American Association of Colleges of Pharmacy's Commission to Implement Change in Pharmaceutical Education,¹ in an effort to link educational outcomes to specific pharmacy practice responsibilities. To strengthen the link between factual content, pharmacy practice abilities, and patient care outcomes, a multidisciplinary, integrated curriculum was designed for the third year of our 4-year program. Because problem-based learning (PBL) is a student-centered, interactive educational process that promotes problem solving, reflective thinking, communication skills, and lifelong learning, it was thought to be the appropriate pedagogy for this portion of the curriculum.^2,3 Thus, the entire third year was comprised of a series of 10 multidisciplinary pharmacotherapeutic courses taught predominately utilizing a problem-based format. These pharmacotherapeutic courses were systems-based (eg, cardiology, pulmonary, etc.) and designed to integrate pharmacokinetics, pharmacology, medicinal chemistry, therapeutics, pharmaceutics, and psychosocial aspects of pharmacy practice utilizing small, facilitated student groups supplemented by periodic discussions involving the entire class.

When we combined PBL with an interdisciplinary curriculum we noted a pedagogical problem not previously well described in the literature: when skills that had previously been assured by specific, independent courses in narrowly defined areas of clinical competence became only a small portion of a student's overall grade within an integrated course, students could “choose” to become competent in that particular skill. When these individual courses or disciplines are collapsed into an integrated curriculum, a potential disadvantage might be fewer opportunities for assessment regarding abilities-based curricular outcomes. In addition, when course grades are a product of comprehensive examinations representative of the integrated coursework and are the only means of summative assessment, a student can progress through the curriculum without demonstrating competence in a given area of expertise. We found this to be particularly true for those skills most difficult to master.

At AUHSOP, our teaching methodology significantly impacted the educational outcomes in clinical pharmacokinetics. Employers, residency directors, and students had historically praised AUHSOP graduates' working knowledge in all basic science areas and especially in the area of applied pharmacokinetics. Following the initiation of the first professional degree PharmD program and an integrated, problem-based learning curricular model, our feedback changed significantly. As a part of AUHSOP's formal Outcomes Assessment Program, focus group sessions were held with preceptors in order to identify programmatic strengths and weaknesses. In 2001, preceptors identified a steep decline in our students' ability to apply pharmacokinetic concepts in patient care environments. To address this deficiency, in 2002, our instructional methods, past and present, were assessed and modified utilizing collaborative action research. As a result, a new instructional methodology and a novel abilities-based assessment tool were developed. The focus of this article is to describe the problems encountered in this area, our analysis of the problem, the approach taken to address the problem, and an evaluation of the solution implemented.

DESIGN

Action research was utilized to evaluate and address the educational problem in our curriculum: a steep decline in the ability of students to apply pharmacokinetic principles in patient care environments. Action research endeavors to improve the quality of a program or an action by using a spiral process that alternates between action and critical reflection.⁴ It is a powerful scientifically based research tool especially well suited for educational research, because it aligns research with practice.⁵ The primary goals of action research are development and change.⁶ It typically is designed and conducted by educators who analyze their own instructional methods in order to improve their teaching practices. The 6 cyclical steps include: identifying the problem and setting an outcome goal; analyzing the problem; determining why there are differences between the goals and the outcomes; formulating hypothetical interventions and changes; implementing changes; and reflecting on outcomes.⁴ Continual assessment followed by reflection, evaluation, and change has been advocated for both curricular and individual course planning in pharmacy education⁷; therefore, action research was utilized to evaluate and refine our curriculum in order to improve the students' ability to apply pharmacokinetic principles in the patient care environment.

Problem Analysis

During the shift from the postgraduate PharmD program to the first-professional degree PharmD program, several significant changes were made in the curriculum that affected pharmacokinetic instruction. Previously, in the postgraduate PharmD program, students were required to complete a basic biopharmaceutics course as part of the BS program, followed by an advanced biopharmaceutics course, and an applied (or clinical) pharmacokinetics course prior to embarking on advanced pharmacy practice experiences (APPEs). In the curricular design of the first-professional degree program, the previously taught biopharmaceutics course was decreased in number of credit hours, and a limited portion of this coursework was incorporated into a “new” pharmacokinetics course. Applied pharmacokinetics as an individual course was eliminated entirely. Applied pharmacokinetic principles were integrated into, and became only a small portion of, a series of 10 third-year multidisciplinary pharmacotherapeutic courses taught predominately utilizing a problem-based format. Brief descriptions of past and present pharmacokinetics-related courses are outlined in Appendix 1.

A panel of pharmacy practice and basic science faculty members was convened to identify possible reasons for the decline in the observed pharmacokinetic practice abilities following these curricular changes. After a critical review of the differences between the current and previous curricula, examination of the teaching methodology for clinical pharmacokinetics in the new curriculum, and reflection on the examination results on the applied pharmacokinetics portion of the comprehensive examinations, the following hypotheses emerged as potential reasons for poor outcomes:

1. Failure to coordinate the instructional format utilized in the pharmacotherapeutic sequence with previous basic science coursework. It was hypothesized that non-alignment might lead to student confusion and unsystematic approaches to problem solving. Collaboration across disciplines is thought to be an important component of successful curricular change.⁸

2. Lack of a consistent, standardized approach to pharmacokinetic problem-solving among different instructors in the 10-course sequence.

3. Significant variation in teaching approaches among instructors.

4. Lack of repetition. For most of the targeted drugs, only 1-3 problems or cases were solved prior to examination.

5. Ability of students to “choose” whether or not to become competent in the skill, since the pharmacokinetic examination made up only a small percentage of the overall course grade.

6. Lack of professional desire to become competent in this particular skill. Students seem to select or preferentially learn skills based on their predetermined concept of relevance to an anticipated practice setting upon graduation vs. learning to prepare for any possible future pharmacy practice. Many believed that they would practice in an environment where the skills would not be utilized.

Development of a New Teaching Methodology

Following analysis of the problem, a new teaching methodology and assessment program were designed to directly address the potential reasons identified for the decline in pharmacokinetic skills, as well as the global learning objectives developed for the course (Table 1). This course was based, in part, on the course previously described by Beck in which students were provided a series of patient cases and required to analyze and identify patient data that might impact drug dosing, assess pharmacokinetic data, make decisions regarding patient care (drug dosing and monitoring), and communicate their recommendations effectively.⁹

Table 1.

Global Learning Objectives for Applied Pharmacokinetics

^aSkills that must be mastered/demonstrated to pass the Abilities-Based Student Performance Assessment

In contrast to the uncoordinated, nonstandardized, nonintegrated approach to pharmacokinetics instruction, which theoretically led to the decline in student abilities, a consistent, systematic approach to pharmacokinetic problem-solving was developed. First, a single faculty member was chosen to implement the new teaching methodology during this year of the curriculum. This new approach consisted of a series of pharmacokinetic workshops designed to parallel pharmacotherapeutic discussions in the PBL 10-course integrated pharmacotherapy sequence of the third year.

During those modules with complementary pharmacokinetic drug topics, weekly or biweekly 2-hour supplemental instruction workshops were scheduled for case-based problem sets and discussion. For instance, vancomycin pharmacokinetics was paired with the “Infectious Diseases” module, and digoxin pharmacokinetics was paired with the “Cardiology” module. Attendance and preparation for these workshops was completely voluntary (40%-70% of the students in the class were typically present). This new teaching approach included the following activities:

Global, systematic review of drug facts.

The first pharmacokinetic workshop in each of the courses was designed to compel students to gather the drug facts necessary to develop an in-depth understanding of the drug. Each student was assigned to independently review a tertiary reference of his choice to develop a global understanding of the pertinent drug using a list of guiding questions provided at the beginning of the course. To make the process more manageable, each group was assigned one aspect/question to present to the class. The purposes of this assignment were to (1) develop a list of “need to know” parameters about any drug the student might encounter, (2) develop a summary of pertinent facts to be used when solving problems, and (3) encourage participation and discussion. Facts presented were evaluated though Socratic discussions, difficult-to-understand points were clarified, and inconsistencies between other coursework or the literature were discussed and addressed. Important points not brought forward for discussion were either discussed by the professor or assigned as further research homework.

Distribution and use of the “Peripheral Brain.”

Following the review of drug facts workshop, the basic facts (such as population parameters) presented by the students were distilled into a fact sheet called a “peripheral brain.” Formulas important for problem solving were added by the instructor. The use of a standardized peripheral brain helped foster a consistent and standardized approach to problem solving by promoting the use of standard population parameters and formulas and assured coordination with previous basic science coursework. Though this peripheral brain was prepared by the instructor prior to the course, the instructor ensured that all of the drug facts were actually first presented by some member of the class (see above). The purposes of preparing a standardized peripheral brain for the students rather than having them generate one on their own were (1) to demonstrate a method for compiling large quantities of information in a usable format, (2) to standardize formulas and facts used in the course, and (3) to streamline instruction. These purposes were reiterated to the class after gathering the facts for each drug. The facts and formulas on the peripheral brain were intended to be utilized as a tool by the students when working on workshop problems sets (described in the next section) and for answering pharmacokinetic questions on both the end of unit pharmacotherapy module examinations and the student performance assessment (SPA) described later in this paper; the peripheral brain never served as an answer key.

Problem sets.

To foster learning by repetition, multiple pharmacokinetic problem sets were developed. The case-based problems mirrored real patient scenarios and required critical thinking, evaluation of the dosing regimens, and patient monitoring. Workshop problems throughout the pharmacotherapeutic module sequence were designed to promote one standard approach to problem-solving which was repeated again and again to promote long-term retention. These workshops were designed to be progressive in nature. Each problem was designed to build upon concepts utilized in previous problems and to include an array of the most important clinical scenarios that pharmacy practitioners might encounter. In general, problem sets began with problems that required patient evaluation and calculation of initial doses utilizing population parameters in uncomplicated patients and progressed toward calculation and utilization of patient-specific parameters to individualize drug dosing.

Workshop activities.

Students were required to attempt to solve each case independently or in groups prior to class. This requirement forced students to assess their own ability to make a clinical assessment, think critically, and accurately perform mathematical calculations. In class, each problem was addressed and worked through in a prescribed stepwise manner. This procedure was designed to emphasize and illustrate the systematic thinking process and assessment process required to solve pharmacokinetic problems accurately in clinical situations. At each step, at least one student was required to provide an answer or an assessment. Those students who had not worked the problem correctly were given time to actually do the thinking or calculation required to complete each step during the workshop session. Answers or approaches that varied were compared and contrasted in an open forum. Discussions of multiple “correct” approaches to a single problem, best approaches, and worst approaches allowed students to expand their knowledge and appreciate clinical decision-making and its corresponding medication-related outcomes. By giving students opportunities to solve seemingly similar cases with different solution paths, they had opportunities to build and reinforce connections between important concepts.

Program Evaluation.

In order to determine whether students' pharmacokinetic skills actually improved following 3 years of this new teaching and assessment methodology, formal evaluations of competence were elicited from fourth-year inpatient preceptors. We felt that preceptor perceptions of student abilities were the most important measure available to us to determine whether skills learned in the classroom were actually transferable to the patient care environment. A survey was developed for online responses utilizing the online program Flashlight Online CTL Silhouette System.¹⁰ The online survey was conducted 9 months into the fourth-year experiential program (January 2006). The survey was approved by the Institutional Review Board of Auburn University.

Assessment of Pharmacokinetics Abilities-Based Outcomes

To limit the ability of the student to “choose” to become competent in applied pharmacokinetics and to encourage a professional desire to become competent in this skill, a 3-step approach to assessment was developed and utilized.

Self-assessment.

Self-assessment, one of AUHSOP's school-wide outcomes, was encouraged. During the workshops no tests or quizzes were administered. Students were expected to come to class prepared, utilize workshop sessions to determine their own level of understanding and ability, identify any deficiencies, and correct deficiencies when they occurred. Students who were unprepared for the workshops were encouraged to participate but were unable to use the workshops as an opportunity for self-assessment.

End-of-course examination.

Students were given a “final” examination at the end of each systems-based pharmacotherapeutic course. Illustrating the cyclical nature of the action research process, our utilization of this examination has evolved over time. Before the initiation of the entry-level program (prior to 1997), the clinical pharmacokinetics portion of the end-of-course final examination accounted for 5%-10% of the overall course grade. With initial implementation of this teaching method (2003), the clinical pharmacokinetics portion of the end-of-course final examination still accounted for 5%-10% of the overall course grade, but in addition, all students were required to pass a comprehensive abilities-based standard performance assessment specifically aimed at assessing pharmacokinetic problem-solving abilities. Currently, the applied pharmacokinetics end-of-course examination accounts for 10% of the overall course grade. A list of the required basic competencies/skills is provided to the students prior to the examination (Table 1). The end-of-course examination is now graded by 2 methods: (1) a traditional scoring method and (2) a set of abilities-based criteria designed to assess the most basic pharmacokinetic competencies and skills (called Level 1 Skills). To demonstrate competency in the kinetics component of each course, a score of 70% by the traditional scoring method and demonstration of 90% accuracy of Level 1 skills are required. Students who score ≤ 69.4% on any examination by the traditionally graded method, or score < 90 % at Level 1 on any abilities-based assessment end-of-course examination, receive an incomplete in that course and are required to take a comprehensive abilities-based examination at the end of the fall semester (the SPA). The traditional scoring method is utilized to calculate the final course grade. If a student is required to take and successfully completes the comprehensive abilities-based examination (SPA), the grade of “incomplete” is changed to the letter grade previously earned for the course.

Standard Performance Assessment.

Performance-based assessment using pre-established criteria and student feedback has been advocated as a tool to enhance students' ability to perform clinical tasks.¹¹ When most professors grade examinations, partial credit for some correct portion of an answer is granted. We found that with “traditional” grading methods, a student could recommend an inappropriate course of action or even a potentially fatal dose of a drug in response to an examination question, yet “pass” the examination by achieving an acceptable percentage grade.

To evaluate and assure student competency as well as evaluate the curricular changes, the first Standard Performance Assessment (SPA) was administered at the end of the fall semester 2002 following 6 of the modular, interdisciplinary courses in which most of the pharmacokinetic skills should have been developed and/or refined (the most pertinent of these first 6 modules included “Renal,” “Infectious Diseases I,” “Cardiology I,” and “Pulmonary”). After the curricular changes were instituted, the SPA was administered annually following completion of these 6 modules. All students were initially required to take the SPA. Now, only those students who do not demonstrate minimal competency on the end-of-module examinations are required to take this comprehensive examination. The SPA consisted of patient-cases followed by 1-2 generalized prompting questions such as: “Based upon the information provided, would you dispense the aminoglycoside as ordered? Why or why not? If not, what would you recommend to the physician?” Generalized prompts were used in order to assess critical thinking and problem-solving skills in addition to mathematical skills.

A list of “minimal competencies” that students were required to achieve to pass the course and a detailed answer key were developed and agreed upon by a 3-member committee for each problem on the SPA. The medications covered by the SPA were aminoglycosides, vancomycin, digoxin, and theophylline.

The SPA developed was designed to evaluate 3 skill levels (Table 1). This provided a means of both summative and formative feedback. A passing score at Level 1 allowed the student to proceed through the curriculum and begin APPEs following the successful completion of all other coursework. A passing score for Level 1 indicated that the student could perform the most basic pharmacokinetic skills: calculation of population pharmacokinetic parameters, evaluation of the prescribed dose utilizing population parameters, modification of the prescribed dose utilizing population parameters, and recommendation of an initial dose utilizing population parameters. Performance at Levels 2 and 3 demonstrated increasing proficiency of application of pharmacokinetics principles in more complex scenarios with consideration of patient-specific parameters and evidence of increasing expertise in clinical assessment, monitoring, and sound decision making. Level 2 and 3 skills were evaluated to provide formative feedback for skill assessment and self-remediation.

Students were required to pass the SPA in order to proceed to fourth-year APPEs. Completed SPAs were graded prior to the start of the following spring semester. Students who received a failing score were required to remediate and retake a similar but different examination, the Challenge SPA. Passing the Challenge SPA allowed the student to proceed through the curriculum. Failure to pass the Challenge SPA resulted in a referral to the Committee on Admissions and Academic Requirements (CAAR). The Committee's actions have been variable but have included requiring a student to take a third examination or a specialty pharmacokinetics APPE early in the fourth-year APPE sequence, with formal feedback provided by the preceptor regarding the student's abilities.

ASSESSMENT

Of 63 students taking the SPA, 41 (65%) passed at the most basic level on the first attempt. None passed at Level 3 (Table 1). Nineteen of 22 students who failed the examination took a voluntary, intensive, 6-week remedial course in the spring of 2003. All of these students passed the SPA at Level 1; 30% passed the SPA at Level 3. This group of the weakest students passed the examination with scores much superior to that of those taking the examination the first time.

Further corrective action was taken as described earlier in the section “Development of a New Teaching Methodology.” In the fall semester of 2003, a series of voluntary workshops were instituted and coordinated with the “Infectious Diseases Part I,” “Pulmonary,” “Cardiology Part I,” and “Neurology-Psychiatry” problem-based, integrated modules. Following a SPA at the end of the fall semester of 2003, a disconcerting 35% of the students again failed to pass the initial examination at Level 1 competence and were required to self-remediate and retest before proceeding to APPEs.

In 2004, the program was not changed appreciably, and again a significant number (54%) of students failed to pass the initial SPA examination. These results led to further curricular change. The pharmacokinetics component became an officially recognized part of each pharmacotherapeutic module. In addition, our current testing and pass/fail policy as described was adopted.

Though a large number of students in the reported years failed the SPA on first attempt, our requirement for self-remediation and re-testing achieved our programmatic goal and assured that 100% of the students eventually passed the high-stakes SPA at Level 1 (demonstrating competency in the most basic pharmacokinetic skills) prior to embarking upon APPEs.

One important addition to our educational process was a set of carefully designed problem-based workshops. Because attendance to these was voluntary, and attendance was never taken, we are unable to report the demographics of the student population who routinely attended the workshops and correlate that data with first-attempt SPA performance.

Preceptor Surveys

The online survey required inpatient preceptors to estimate the percentage of students who could independently perform specified pharmacokinetic-related tasks during the first 3 clinical APPEs. Possible responses were 90%-100%, 80%-89%, 70%-79%, 60%-69%, or <60%. There was a 49% preceptor response rate. Eighty-one percent of current preceptors felt that more than 70% of students could accurately calculate population pharmacokinetic parameters. Eighty-six percent felt that more than 70% of students could utilize population parameters to calculate an initial dose; 67% believed that greater than 70% could evaluate a prescribed regimen; and 72% believed that greater than 70% could recommend a new dose or modify an existing regimen. Preceptors (62%) indicated that the PharmD students were independent and self-directed in performing pharmacokinetic calculations in patient care (Table 2). Preceptors who had been working prior to May 2003 (66 % of responders completed both survey instruments) were also asked to compare the pharmacokinetic abilities of students prior to and after the implementation of the new pharmacokinetic instructional and assessment methods. Preceptors who addressed these items felt that more students were competent after institution of the new methods. Statistical tests of significance for independent samples and dependent samples could not be performed, because some preceptors completed both survey instruments and other preceptors completed only one survey instrument.

Table 2.

Percentage of Preceptors Who Agreed That Greater Than Seventy Percent of Pharmacy Students Could Perform a Required Ability

Informal Preceptor Evaluations

In 2003 and 2004 during informal discussions with individual preceptors, discussions at faculty meetings, and group discussions at regional preceptor meetings, preceptors reported a significant improvement in pharmacokinetic skills demonstrated in the clinical environment and felt that students were now “well-prepared” for APPEs in this area. These comments were in stark contrast to the previous perceptions. Most preceptors felt that they no longer had to try to “teach” basic pharmacokinetics concepts during clinical APPEs (which they felt they could not do, given the time constraints of teaching and practice). Preceptors who had been teaching at this institution longer than 10 years seemed to be most keenly aware of these skill changes. In addition, the Curriculum Committee received no further communiqué reporting continued deficiencies in this skill.

DISCUSSION

Our cyclical improvement process using action research has demonstrated that teaching improvements can lead to significant learning improvements. Cyclic reevaluation and change led us to the adoption of a significant philosophical change: a focus on ability-based outcomes rather than ability components.

Over the past 4 years, our teaching methods for applied pharmacokinetics have been refined and modified based on our intense and ongoing review. The key to success of the program began with a critical and honest analysis of our educational program to determine what was and was not working and to design a curriculum that addressed the deficiencies.

In the analysis of the potential reasons for the decline in pharmacokinetic abilities, 6 hypotheses emerged. We attempted to address each. Specifically, we collaborated with the basic science disciplines to coordinate the instructional format. In an effort to standardize the approach to pharmacokinetic problem-solving and provide consistency, we designated a single faculty member to assume the role as instructor for all clinical pharmacokinetics in the third year of the problem-based learning curriculum. Since pharmacokinetics is now integrated into the third year of the curriculum and not an individual course, the instructional approach to clinical pharmacokinetics now couples problem-based, active learning with voluntary workshops. For this specialized content and in response to the deficits in abilities, this supplemental instruction approach was taken to ensure accuracy of problem-solving, to provide time for repetition of skills via multiple problem sets, and to address misconceptions in learning.

Coupled with thoughtful integration, consistency, and maintenance of a systematic approach, our high-stakes, independent abilities-based assessment tool assured student competency prior to entering the patient-care environment. Following the adoption of the first professional degree PharmD program, many pharmacy educators have expressed concerns about the possibility of what has been called “the inevitable dumbing down” of the curriculum as we moved from small classes of almost uniformly “high achievers” to large classes composed of students with a much wider distribution of intrinsic motivation and abilities. Students are practical. From their perspective, the tests are the curriculum. When the content requirements of a curriculum are extremely demanding and when evaluations are designed in a way that students can strategically omit significant content and still “succeed,” wise students often choose to do so. With no high-stakes consequences (the pharmacokinetics questions on the end of module examinations represented only 5%-10% of the overall grade), this portion of the examination was dismissed by students who ultimately passed the course and progressed in the curriculum. The high-stakes, abilities-based assessment tool (SPA) limited the ability of a student to “choose” to become competent in this particular skill even though it was folded into a multidisciplinary course. Successful performance on the SPA was a requirement for progression to fourth-year APPEs and assured minimal student competency.

Both our instructional methods and assessment methods have directly impacted student performance. The best measure of a student's patient care skills after leaving the classroom is preceptor perception. Both informal discussions and responses on formal surveys indicated that the instructional changes improved the students' abilities to provide pharmacokinetic pharmaceutical care in the direct patient-care environment.

We acknowledge several limitations of the preceptor survey. A 49% response rate may have influenced the overall results. Due to the anonymity of the data collection method, we were unable to analyze in a paired fashion the responses of preceptors who answered the survey regarding both prior and current student performance. In addition, the survey instrument did not allow for written comments to enumerate specific deficits observed relating to these abilities. Since practitioners themselves may use different pharmacokinetic equations, some may have misjudged student abilities when students chose a different, but appropriate method to solve a particular problem. Our survey did not allow us to ascertain whether differences between preceptor and appropriate student mathematical approaches ever resulted in clinically significant variations to drug regimen design or drug therapy management. We plan to acquaint preceptors with our educational approach as well as modify the survey instrument for further analysis. While the informal responses by preceptors are positive and support improved abilities in the patient care setting, we are concerned that on the online survey, 38% of preceptors still reported that students could not independently use pharmacokinetic calculations to provide pharmaceutical care. Since all students must demonstrate basic pharmacokinetic competence prior to beginning APPEs, we know this skill had been mastered; nevertheless, we must determine why a significant percentage of students were not competent or confident in performing pharmacokinetic calculations in the clinical setting.

Though our action research was designed to evaluate and improve curricular design, there may be reasons for a decline in any given skill other than curricular change. A lack of professionalism may explain some of our observations. Scores on end-of-course scores examinations may indicate that students may not be driven by a professional desire to achieve skill competence. Indeed, students often “choose” to postpone this learning and instead defer this preparation to the end of fall semester, electing to take the SPA. Even then, a large percentage of students fail to pass the SPA initially and have to retake this examination prior to beginning APPEs.

During the action research process, we found that successful integration is difficult and requires work and coordination. Though integration is a simple word and a goal that we often strive for, as educators we seldom seem to accomplish this goal. This teaching methodology was carefully designed to integrate material from first- and second-year coursework with our third-year multidisciplinary courses. Developing this consistency took time, patience, and perseverance. Maintaining consistency in style and application from previously taught introductory courses decreased student frustration and streamlined education. Working with instructors in concurrent courses assured that student learning was enhanced and supported rather than sterile and not applicable to other course material. Integration for application is a key to our success. To assure this success, faculty members must be willing to work together to develop both a standardized and an integrated approach.

In schools of pharmacy with class sizes over 100, the traditional design of problem-based or case-based learning with small groups of 6 to 8 students, administration, staffing, and coordination of these activities is expensive and often logistically impossible. However, finding expert facilitators committed to and capable of helping students meet all of the learning objectives is often the greatest challenge. When facilitated in a large group as in this model, resources are wisely utilized and logistical problems are resolved. In fact, we have found that a large group facilitation can stimulate discussions that help all students meet all of the learning objectives. Our workshops were relatively easy to schedule and manage and could easily be incorporated at any school of pharmacy with the proper personnel. The inherent difficulty in abilities-based examinations of this type is the time necessary for the administration and assessment/grading of that examination. Grading is labor-intensive and requires 40-80 man-hours for each of the end-of-course pharmacotherapy examinations. Grading the SPA required at least this much time. To streamline grading we have developed organized keys which include multiple acceptable answers. Also, we only require only those students who fail to adequately demonstrate the skill during each of the end-of-course examinations to take the comprehensive SPA examination. Clerical staff members cannot be used to grade examinations because graders must understand the topic and methodology well enough to make discretionary decisions even with a well designed and detailed key. We have employed fourth-year students who work directly with the professor to grade the first 3-5 examinations to assure that most of their questions are discussed and important subtleties in grading are appreciated and to assure standardization of grading. During the grading process, any questionable answers are flagged. The administrating professor evaluates flagged examination questions as well as the entire examination of any student failing to meet the criteria for passing. Utilizing these strategies, our instructional and assessment modalities were quite manageable.

Due to the success observed with this teaching and assessment method for applied pharmacokinetics, we plan to consider implementing a similar type of abilities-based assessment with independent examinations and consequences for progression for other difficult-to-integrate topics in our third year. We are in the process of integrating biostatistics and clinical research design into this third year of the curriculum and may do so in a similar manner.

CONCLUSION

Our method of applying action research to solve a curricular problem coupled with the utilization of an abilities-based assessment tool could be utilized to address a deficiency in any skill identified as important in a pharmacy curriculum. It could be used for any part of a curriculum that is perceived as being “lost” or fragmented through integration of coursework. Following significant curricular changes associated with the switch from a dual degree program to a first-professional degree PharmD program, we identified an important decline in demonstrable applied pharmacokinetics abilities. After a critical review of the differences between the current and previous curricula, examination of the teaching methodology for clinical pharmacokinetics in the new curriculum, and reflection on the examination results on the applied pharmacokinetics portion of the comprehensive examinations, we developed a list of potential reasons for the decline in pharmacokinetic skills. An instructional method was designed to directly address each of these potential curricular flaws. A high-stakes abilities-based assessment tool was developed and utilized as a requirement for curricular progression. Both changes in abilities-based outcomes and faculty surveys demonstrated a significant improvement in student pharmacokinetic skill performance in the direct patient-care environment following these instructional and assessment changes.

Appendix 1.