Skip to main content
NCBI home page
The Canadian Journal of Hospital Pharmacy logoLink to The Canadian Journal of Hospital Pharmacy
. 2014 Jul-Aug;67(4):304–307.

Are Standardized Knowledge Assessment Exams the Best Way to Assess a Pharmacist’s Competence and Should They Be a Mandatory Componentof Licence Renewal?

PMCID: PMC4152971  PMID: 25214663
Can J Hosp Pharm. 2014 Jul-Aug;67(4):304–305.

The “Pro” Side

Katelyn Halpape 1, Elaine Lo 1, Jillian Reardon 1, Ricky Turgeon 1, Sarah N Stabler 2

Imagine this all-too-common scenario: You are a pharmacist working in the intensive care unit (ICU) of a tertiary care hospital. As you finish writing an order related to a complex therapeutic drug monitoring issue, the patient in the next bed says, “Hey, you’re a pharmacist, right? I’d like to quit smoking—could you tell me what’s out there for me?” How do you respond? Do you simply tell the patient that you are an ICU pharmacist who deals only with hyper-acute disease management issues, or do you uphold your professional duty and use the full breadth of your knowledge and training to make some recommendations?

In an environment in which patients present with increasingly complex medical issues, it is crucial that all pharmacists maintain and utilize their professional competencies, as outlined by the National Association of Pharmacy Regulatory Authorities.1 Although each provincial health act requires that the relevant pharmacy college establish a continuing education program to uphold these standards, such programs are not uniform across the country. As the most consistent and objective form of evaluation, a standardized knowledge assessment exam is the best means of ensuring maintenance of these competencies and should be a mandatory component of licence renewal across Canada. In the aforementioned scenario, had the ICU pharmacist maintained competency standards by preparing for a knowledge assessment exam, he or she might have been better equipped to identify the drug-related issues associated with quitting smoking, develop a pharmaceutical care plan, and communicate the plan to the patient and the care team.

For the purpose of this debate, we are using the accepted definition of a knowledge assessment exam: a standardized, written exam designed to evaluate a pharmacist’s application of the knowledge and skills required to solve drug therapy problems and provide direct patient care. This exam format includes, but is not limited to, multiple-choice and short- and long-answer questions that collectively evaluate all levels of knowledge, from simple recall to synthesis and evaluation, as outlined in the taxonomy of educational objectives described by Bloom and others.2

Opponents of the standardized knowledge assessment exam might argue that pharmacists working in niche clinical areas no longer require “entry-to-practice” knowledge. To this we would respond that even highly specialized pharmacists are relied upon for their general knowledge of pharmacotherapies. Pharmacists’ greatest asset is not only the depth but also the breadth of their pharmacotherapeutic knowledge and skills, regardless of setting. The failure of any pharmacist to demonstrate entry-level competency compromises the quality of patient care and projects an inconsistent image to health professional colleagues, as well as to the public. Additionally, maintenance of foundational knowledge confers the benefit of increased flexibility and ease of transitions between clinical roles that inevitably occur over the span of a pharmacist’s career.

A common misconception is that the main purpose of the knowledge assessment exam is punitive, to serve as a slap on the wrist to those not meeting standard competencies. In actuality, one of the key and likely underappreciated benefits of a standardized exam is its utility as an individualized quality assurance measure. Failing the exam should prompt initiation of a remedial process involving personal reflection and identification of knowledge deficits. The most memorable life lessons are those learned from making a mistake. Would pharmacists not best be served by learning through errors on a test rather than errors involving a patient? Conversely, success on the knowledge assessment exam can serve as positive validation that expected competencies are being maintained.

Evident in the pharmacy profession’s innovative spirit is the fact that other health care professions are moving toward similar application of a standardized knowledge assessment exam. In 2014, the College of Physical Therapists of British Columbia is implementing a standardized, written, case-based knowledge assessment exam that employs a nonpunitive approach to filling competency gaps.3 Starting in 2015, occupational therapists in British Columbia will also be required to undertake a similar exam to maintain licensure.4

In practical terms, a written exam is cheaper and logistically simpler to implement, administer, and standardize than proposed alternatives, such as practice peer reviews (also known as job samples) and Observed Structured Clinical Examination–style assessment (also known as job simulations). A review of assessment methods for physician competence found that, in a 4-hour test, multiple-choice and short-answer questions had better reliability than other examination methods, including job simulations and job samples.5 Job samples rely on assessors trained in the selected subspecialty to identify deficiencies in the practitioner’s knowledge or skill, which in some instances limits the pool of potential assessors to a small group.6 This situation may in turn lead to colleagues evaluating each other, an inherent source of bias. Additionally, practice reviews rely on a limited sample of cases, which are impossible to standardize in terms of difficulty and competencies being evaluated. Simulations as a form of evaluation are extremely resource-intensive, as made evident by the 17-member committee,7 the large number of trained personnel, and the high cost to new practitioners required to administer such an evaluation just once for entry to Canadian pharmacy practice.

Given that there is no gold standard method for assessing competence, the choice of assessment method must be based on practical and reliable measures.5 As part of a project designed to develop an assessment method to evaluate the continuing competence of pharmacists, Fielding and others8 created and pilot-tested a written examination. They demonstrated that this examination provided a reliable and valid assessment of pharmacists’ general knowledge.

As lifelong learners, pharmacists should not view standardized knowledge assessment exams simply as a mandatory requirement of licensure; rather, they should embrace such exams as a means to positively affect their individual and collective growth as professionals. Would it be sensible to purchase an expensive car and never take it to a mechanic to see if it requires a tune-up? Similarly, is it sufficient to rigorously test future pharmacists via the Pharmacy Examining Board of Canada’s written evaluating examination and an Observed Structured Clinical Examination and then send these new graduates forward into the tumultuous and ever-changing world of pharmacy without ever pausing to re-evaluate and monitor their skills in a standardized way?

To quote the “Oath of a Pharmacist” (as adopted by the American Pharmacists Association and widely used across Canada), “I will accept the lifelong obligation to improve my professional knowledge and competence.”9 Pharmacy is a self-regulated profession; therefore, a mandatory mechanism is needed to ensure maintenance of the basic competencies that are foundational to the profession. An objective, validated measure, such as a knowledge assessment exam, will help to ensure that pharmacists are maintaining an acceptable level of competence.

Footnotes

Competing interests: None declared.

References

Can J Hosp Pharm. 2014 Jul-Aug;67(4):305–307.

The “Con” Side

Sarah Burgess 1, Natalie LeBlanc 1, Daniel Rainkie 1, Craig Roels 1, Sam Louie 2

The Pharmacy Examining Board of Canada sets standards for entry to practice, re-entry to practice, and continuing competence for all pharmacists in Canada. However, assessment of competence is legislated by provincial legislation, which the colleges in each province must follow to ensure provincial practice standards are being met. The colleges of pharmacists in British Columbia and Ontario have incorporated standardized knowledge assessment exams into their relicensure processes, whereas Alberta has recently removed the requirement for pharmacists to complete these exams. With the expansion of pharmacy practice and the nontraditional career paths being taken by many pharmacists, we believe this assessment method is not the best way to determine a pharmacist’s competence.

We define “knowledge assessment exam” as an open-book, handwritten or computer-based multiple-choice exam1,2 and “competence” as the professional knowledge, skills, and abilities and the appropriate professional values and attitudes of a pharmacist.3

There are parallels between a driver’s licence and a pharmacist’s licence with regard to renewal. Upon entry into practice, pharmacists are required to pass a written exam to assess knowledge and a practical exam to assess their application of that knowledge. Likewise, to obtain a driver’s licence, individuals are required to pass both a written and a practical exam. After obtaining a driver’s license, most individuals are not required to pass any kind of assessment upon licence renewal but are expected to remain proficient in all aspects of driving. If an individual’s competence to drive is called into question, a more meaningful assessment of driving abilities, such as an in-car driving test, would be necessary. Likewise, a pharmacist’s competence cannot be adequately assessed by a knowledge assessment exam; an evaluation of the person’s day-to-day practice is a more appropriate approach.

Pharmacists’ health care colleagues and other professionals are not required to complete a knowledge assessment exam for licence renewal. For example, engineers in British Columbia must undertake 80 professional development hours per year, which can take the form of courses or participation in mentorship opportunities.4 Similarly, BC nurses do not complete a knowledge assessment exam but are required to design and implement a professional development plan based on self-assessment of their learning needs and peer feedback.5 Like pharmacists, these professionals are well respected, have substantial responsibilities and are required to maintain competence in their respective fields; however, knowledge assessment exams are not a requirement for licence renewal.

Knowledge assessment exams are flawed in their ability to assess the application of knowledge. In a study comparing open-book and closed-book examinations in a group of medical students, Schumacher and others6 found that students in the open-book arm of the study had significantly higher results (mean 61.18% versus 54.29%; p < 0.01). Given the open-book nature of the knowledge assessment exam, it may not provide a true measure of pharmacists’ application of knowledge, but rather may indicate how well they can utilize the references provided. The Objective Structured Clinical Examination (OSCE) has been proposed as a more appropriate assessment, because it uses practice-based, real-world scenarios. In a Canadian study conducted to determine how well the knowledge assessment exam predicted scores on an OSCE, 21% of participants were considered “false positives”,2 which means that 1 of every 5 participants passed the knowledge assessment exam but went on to fail the OSCE. While proponents favouring the knowledge assessment exam might argue that it helps to protect the public, such a substantial false-positive rate should be concerning to members of the community and to pharmacy regulatory bodies.2

Finally, the knowledge assessment exam does not provide a complete assessment of a pharmacist’s competence. Such an evaluation is much more complex than answering open-book, multiple-choice questions. The knowledge assessment exam cannot evaluate communication, organizational skills, values, or attitudes, but these skills are essential in the provision of pharmaceutical care. Furthermore, a pharmacist’s competence in one area does not necessarily mean that he or she will be competent in another area. Using the earlier example of drivers’ licences, it is worth remembering that a long-haul truck driver, an ambulance driver, and a motorcyclist each require a different licence. Having one knowledge assessment exam for the variety of drivers on the road could be viewed as inappropriate. Similarly, pharmacists are involved in many nontraditional roles such as academia, management, industry, and research, and a “one size fits all” type of assessment is therefore not appropriate. Miller’s “pyramid of competence” is a 4-tiered model, with “knowledge” as the base, followed by “knows how”, “shows”, and finally “does” at the peak.7 With a knowledge assessment exam, a pharmacist’s competence is never assessed beyond the bottom layer of the pyramid. Application of knowledge at the peak, the “does” level, is where pharmacists should be evaluated to obtain a true measure of competence and to identify those at risk for unsafe practices.

Methods of relicensure for pharmacists are heterogeneous across Canada. All provinces have a continuing education component, with British Columbia using a combination of a knowledge assessment exam and continuing education and Ontario having introduced practice peer review. The College of Pharmacists of British Columbia has proposed replacing the knowledge assessment exam with focused practice reviews to assess the competence of pharmacists.8 The feasibility of practice reviews has been assessed by means of a cost impact analysis, and the process of creating a standardized process for conducting these reviews is underway in British Columbia. This type of assessment is individualized to the examinee and allows pharmacists to demonstrate the application of their standards of practice.

We propose the following as an alternative to knowledge assessment exams: a practice review, conducted every 5 years, which would focus on medication reviews, patient education, and appropriate documentation of interventions, with peer feedback and training for individuals not meeting the standards of practice. In addition, each pharmacist should have a continuous professional development portfolio, based on a framework developed in the United Kingdom, which incorporates the relevance and impact of portfolio contents,9 submitted yearly for a total of 15 hours of professional development. Ten percent of pharmacists would be audited yearly, and a mandatory continuous training development workshop, with re-evaluation the following year, would be provided for those not meeting the standards.

In conclusion, given the diversity of pharmacy practices, there is no single exam that can adequately capture whether or not competence is being achieved and sustained. We have identified weaknesses in the rationale, methods, and results of the knowledge assessment exam. Our proposed model is individualized and outcome focused. It has the ability to improve practice while maintaining professional autonomy, which should be the focus for all pharmacy regulatory bodies across Canada.

The Pro and Con articles for this issue’s “Point Counterpoint” column were developed from a debate held in fall 2013 as part of the course “Advanced Pharmacy Administration: Topics in Contemporary Practice,” Doctor of Pharmacy program, Faculty of Pharmaceutical Sciences, The University of British Columbia.

Footnotes

Competing interests: None declared.

References

Articles from The Canadian Journal of Hospital Pharmacy are provided here courtesy of Canadian Society of Healthcare-Systems Pharmacy

RESOURCES