Abstract
OBJECTIVES
This study aimed to implement a web-based pediatric education program designed for pharmacists who participate in neonatal and pediatric order verification at a community-based health system and to evaluate the success through measuring outcomes related to both comfort and competence of pharmacists in pediatric and neonatal pharmacotherapy.
METHODS
This prospective quality improvement study assessed changes in confidence and competence from before to after education. Eight educational modules were designed to provide education based on the needs of this institution. All pharmacists who participate in neonatal and pediatric order verification were eligible for inclusion throughout the health system. Time in the verification queue for pediatric and neonatal medication orders was compared for before to after education as an objective surrogate marker for comfort and competence. A provider survey was conducted before and after education to assess the providers' perspective of the quality and necessity of pharmacist-provider interactions.
RESULTS
All confidence scores showed statistical improvement from before to after education (p < 0.001). Before to after education competency scores significantly improved (median 77% [IQR, 69%–85%] to 100% [IQR, 92%–100%]; p < 0.01). The module with the lowest mean score (87%) was module 4 (Antibiotics Part 1), and the one with highest number of retakes (24 retakes from 16 different pharmacists) was module 5 (Antibiotics Part 2).
CONCLUSIONS
Targeted web-based education effectively improved both confidence and competence among health-system pharmacists to provide pediatric and neonatal care in a community hospital.
Keywords: competence, confidence, education, pediatric pharmacy, pediatrics
Introduction
Most pharmacists receive little to no formal training in pediatric and neonatal pharmacotherapy.1,2 The Accreditation Council for Pharmacy Education recognizes pediatric training as one of the science foundation elements essential to the development of pharmacists.3 The Accreditation Council for Pharmacy Education recommends that students understand essential patient care issues for different patient age groups, including the pediatric and neonatal population.4 Prescott and colleagues5 conducted a study on the extent to which pediatrics is taught within Doctor of Pharmacy programs. This study reported that student pharmacists receive a mean of 21.9 ± 22.9 hours (range, 1–153 hours) of pediatric lectures, with most pediatric topics not introduced until the third year within pharmacy programs in the United States. Because of the variability of pediatric exposure within pharmacy programs, there is inconsistency in content knowledge and confidence among pharmacists. There are also few data on targeted pharmacist education and educational design.1
In 2015 the PPAG recommended that any pharmacist caring for children in a hospital setting should be able to demonstrate proficiency in core knowledge and skills before practicing independently.1,6 The American Society of Health-System Pharmacists (ASHP) has released recommendations for providing pediatric services in hospitals and health systems. The ASHP states that pharmacy departments should provide adequate training for all staff members who may be called upon to provide care to pediatric patients. It is noted that staff development should address the needs of the department and various pediatric populations served in the institution. The education should be current, frequently scheduled, and easily accessible, noting that online tools are a convenient source for education and development. This recommendation included a minimum set of core competencies to help direct knowledge and skills needed for pharmacists. These core competencies include pharmacokinetic and dynamic differences, weight-based dosing and calculations, fluid and nutrition requirements, common diseases and drugs, drug information resources, pharmacogenomics, and specialized drug preparation and administration techniques for pediatric patients based on age.7
There are few data addressing the needs of adult learners for pharmacy education programs within the workplace. The foundational principles of the adult learning theory emphasize the importance of applicability of content to a learner's individual goals and objectives. The principles also describe the importance of autonomy in growth and learning.8 These principles for adult learners and professional recommendations were used when developing this pediatric education program.
This prospective, observational pediatric pharmacy educational program was Institutional Review Board approved and conducted at Parkview Health in Fort Wayne, IN. Parkview Health is an 8-hospital, community-based health system with a level 2 pediatric trauma center, 20 general pediatric beds, 7 pediatric intensive care unit beds, and a level 3 neonatal intensive care unit with 31 beds. All pharmacists who participate in the order verification of pediatric and neonatal orders employed by the health system were eligible for participation, including pharmacists with primary roles in distributive, clinical, outpatient, and ambulatory care areas. Clinical pharmacists at this institution participate in team rounds on patient care floors, conduct research, and train pharmacy residents and students.
At Parkview Health, previous experience with neonatal or pediatric pharmacotherapy is not a requirement for participation in order verification for these age groups. Opportunities for exposure and training during new hire orientation are limited. The purpose of this study was to implement a pediatric education program designed for this institution and to evaluate its success through measuring outcomes related to both the comfort and competence of pharmacists regarding pediatric and neonatal pharmacotherapy.
Materials and Methods
The educational program was designed by the authors of this study and was created specifically for Parkview Health. Previous studies were reviewed for information regarding design of the program.9–11 Eight educational modules were developed through Microsoft PowerPoint (Redmond, WA). Because this program was implemented for all 8 hospitals within the health system, the modules were uploaded to a SharePoint (Microsoft) website and were accessible online for all participants. Each of the authors independently reviewed each module for appropriateness. The program began with an introduction to pediatrics and medication safety modules. These modules had to be completed before a pharmacist could attempt any of the remaining clinical topics. Admission diagnosis codes were accessed from 2015 to 2017 to determine the most prevalent causes for admission and were subsequently used to create the clinical topics to be covered through education (Table 1). All modules were accredited for continuing education from the Indiana State Board of Pharmacy. Each module was followed by a quiz that was created through Microsoft Forms that was also accessible on the SharePoint website. The number of quiz questions ranged from 6 to 11, depending on the length of the module, and addressed objectives presented at the beginning of the module. Quizzes were allowed to be repeated as many times as needed to achieve a score greater than 80%. Questions and answer selections were shuffled for each retake to prevent the memorization of answers. Participants never received answers to the questions until after all participants completed the study. Patient cases were built in the electronic health record training environment to simulate real-world dilemmas pharmacists may encounter in their daily workflow.
Table 1.
Module Topics
| Module | Topic |
|---|---|
| 1 | Introduction to Pediatrics |
| 2 | Pediatric Medication Safety |
| 3 | Respiratory Conditions |
| 4 | Antibiotics Part 1 |
| 5 | Antibiotics Part 2 |
| 6 | Diabetic Ketoacidosis |
| 7 | Seizures |
| 8 | Introduction to Neonatology |
A demographic assessment was required by the participating pharmacists and included the following information: current pharmacy degree, years in pharmacy practice, years of inpatient hospital pharmacy experience, years of outpatient pharmacy experience, primary pharmacy work location, pediatric student rotation exposure, pediatric residency rotation exposure, hours of lecture/didactic training in pediatrics, didactic elective completion, and any pediatric-related continuing education credits completed in the last year. The primary outcome of this study was change in confidence and competence from before to after education. A pre-education confidence and pre-education competence assessment was required before the modules were available. After completion of the 8 modules, a post-education confidence and post-education competence assessment was completed. These post-assessments were released 2 weeks after recommended module completion. Confidence was assessed using a 5-point Likert scale (1 = low confidence, 2 = low to moderate confidence, 3 = moderate confidence, 4 = moderate to high confidence, 5 = high confidence). A 13-question competence assessment was designed by the authors with questions related to the objectives from each of the modules; this was also created through Microsoft Forms. Competency assessment answers were not available to the participants after completion, and retakes were not permitted. After completion of the 8 modules, the same competency assessment was repeated. The questions were designed to be similar to the individual module quiz questions; however, no questions were repeated from quiz to competence assessment. Using the online platform, time stamps for completion were noted for each participant to ensure the preassessments were completed before the modules or post-assessments.
Secondary outcomes included mean score for each module, number of retakes for each module, changes in perceptions of phone call quality and interactions between pediatric and neonatal providers and pharmacists, and before to after order verification time. The pediatric and neonatal providers were surveyed to evaluate the changes in perceptions of phone call quality and interactions between providers and pharmacists. Verification time of pediatric and neonatal orders was also compared before education to after education to evaluate changes in order verification times as another means of measuring competence. A difference in order verification times was used with the hypothesis that increased knowledge by staff may shorten verification times. These final 2 end points, the provider survey and order verification time, were used as surrogate markers in an attempt to link real-world outcomes with the impact of education. Additionally, participant feedback was surveyed at the end of the program.
The Wilcoxon signed rank test was used to compare the results of the primary outcomes. Secondary outcomes were assessed using descriptive statistics. The priori level of significance was 0.05.
Results
A total of 100% of pharmacists (n = 76 of 76) who were eligible for inclusion in the study participated in the education program. Most of the pharmacists have been practicing in pharmacy for more than 5 years (5.1–10 years, 26%; 10.1–20 years, 29%; >20 years, 28%). Most pharmacists practiced primarily in a distributive role (46%), followed by 42% in a clinical role, 9% in an outpatient role, and 3% in an ambulatory care role. Pharmacists who held a Doctor of Pharmacy degree (78%) were more common at this institution than pharmacists with a Bachelor of Science degree in pharmacy (22%). Similarly, 56% of pharmacists recalled receiving <2 hours of pediatric lecture during pharmacy school and more than 60% of pharmacists recalled having no additional pediatric pharmacy experience apart from didactic lecture hours.
Each confidence question improved significantly pre-education to post-education (p < 0.001), and the results are displayed in Table 2. Pre-education competence to post-education competence scores increased significantly (p < 0.01) and are shown in Table 3. The modules with the lowest mean score (87%) and highest number of retakes (24 retakes from 16 different pharmacists) were modules four and five, respectively. Both modules 4 and 5 were pediatric antibiotic sections, which consisted of acute otitis media, urinary tract infections, skin and soft tissue infections, intra-abdominal infections, community-acquired pneumonia, and meningitis.
Table 2.
Pre-education to Post-education Confidence Assessment
| Confidence Questions | Likert Scale* | |
|---|---|---|
| Pre Scores, Median (IQR) | Post Scores, Median (IQR)† | |
| You are receiving a drug information question regarding medication dosing from a nurse/physician for a pediatric patient. How confident are you in answering this medication dosing question? | 3 (2–4) | 4 (4–4) |
| How confident are you in knowing where to look for appropriate medication information in order to answer a pediatric drug information question? | 4 (3–4) | 5 (4–5) |
| You are receiving a drug information question regarding route of administration from a nurse/physician for a pediatric patient. How confident are you in answering this route of administration question? | 3 (3–4) | 4 (4–5) |
| Do you feel you have adequate pediatrics education to answer pediatric medication questions? Please rate your confidence level to this question. | 3 (2–3.5) | 4 (4–4) |
| What is your overall confidence with verifying orders for pediatric patients? | 3 (3–4) | 4 (4–4) |
| Do you feel you have adequate pediatrics education to verify medication orders for pediatric patients? Please rate your confidence level to this question. | 3 (3–4) | 4 (4–5) |
| Neonatal questions | ||
| You are receiving a drug information question regarding medication dosing from a nurse/physician for a neonatal patient. How confident are you in answering this medication dosing question? | 2 (2–3) | 4 (3–4) |
| How confident are you in knowing where to look for appropriate medication information in order to answer a neonatal drug information question? | 3 (3–4) | 4 (4–5) |
| You are receiving a drug information question regarding route of administration from a nurse/physician for a neonatal patient. How confident are you in answering this route of administration question? | 3 (2–3) | 4 (3–4) |
| Do you feel you have adequate neonatal education to answer neonatal medication questions? Please rate your confidence level to this question. | 2 (1.5–3) | 4 (3–4) |
| What is your overall confidence with verifying orders for neonatal patients? | 3 (2–3) | 4 (3–4) |
| Do you feel you have adequate neonatal education to verify medication orders for neonatal patients? Please rate your confidence level to this question. | 3 (2–3) | 4 (3–4) |
* 1 = low confidence; 2 = low to moderate confidence; 3 = moderate confidence; 4 = moderate to high confidence; 5 = high confidence.
† p < 0.01.
Table 3.
Pre-education to Post-education Competence Assessment
| Competence | Scores %, Median (IQR) | p value |
|---|---|---|
| Pre-education assessment | 77 (69–85) | < 0.01 |
| Post-education assessment | 100 (92–100) |
Order verification times are shown in Table 4. A difference was not detected comparing 2017 and 2018 data. Because of a low physician survey completion rate (9%; n = 4 of 47 of all providers) we were unable to detect a statistical difference in phone call quality and interactions; however, the authors noted a subjective improvement in number and quality of phone calls from survey comments.
Table 4.
Mean Module Scores and Number of Retakes
| Module | Topic | Score, %, Mean ± SD | Number of Retakes* | Number of Pharmacists |
|---|---|---|---|---|
| 1 | Introduction to Pediatrics | 91 ± 9 | 6 | 5 |
| 2 | Pediatric Medication Safety | 94 ± 9 | 5 | 5 |
| 3 | Respiratory Conditions | 92 ± 9 | 5 | 5 |
| 4 | Antibiotics Part 1 | 87 ± 9 | 9 | 6 |
| 5 | Antibiotics Part 2 | 89 ± 9 | 24 | 16 |
| 6 | Diabetic Ketoacidosis | 96 ± 9 | 5 | 4 |
| 7 | Seizures | 98 ± 5 | 6 | 5 |
| 8 | Introduction to Neonatology | 91 ± 9 | 8 | 8 |
* Retake if <80%.
Table 5.
Pediatric and Neonatal Order Verification Time, in Minutes
| Area | January, Mean (Range) | February, Mean (Range) | March, Mean (Range) | |||
|---|---|---|---|---|---|---|
| 2017* | 2018† | 2017* | 2018† | 2017* | 2018† | |
| Pediatric | 16 (1–101) | 22 (1–170) | 22 (1–122) | 22 (1–138) | 16 (1–208) | 20 (1–174) |
| PICU | 14 (1–250) | 21 (1–138) | 17 (1–111) | 16 (1–155) | 11 (1–69) | 12 (1–144) |
| NICU | 16 (1–176) | 18 (1–138) | 17 (1–261) | 15 (1–143) | 17 (1–138) | 13 (1–156) |
* Pre-education times.
† Post-education times.
Discussion
The web-based program designed specifically for this community hospital increased the confidence and competence of pharmacists. The main objective of this program was to address the ASHP and PPAG recommendations to provide training and to ensure pharmacists have the baseline knowledge required to care for pediatric patients safely and effectively.
Despite the benefit shown in this program, the lack of validated end points for assessing knowledge gain is a limitation to this study. The study used assessments created by the authors of this study for confidence and competence to have targeted education and were therefore not previously validated. We were unable to assess end points related directly to improvement of patient care; therefore, no conclusions can be made to demonstrate this education improved patient outcomes. Surrogate markers were used, including order verification times and a provider survey, in an attempt to link real-world outcomes with the impact of education. There is no clear explanation for the trends shown in the order verification times, and no difference was detected. The ability to detect a difference may be affected by the substantial amount of new education and processes that was implemented in the medical center during this time. In a previous study conducted by Meyers and Costello-Curtin,2 some pharmacists reported feeling more overwhelmed after completing the pediatric education program, and this could also be a potential factor in the longer order verification times. Another explanation for increased order verification time could be due to more attention and inquiry due to issues with orders that may have been previously overlooked prior to implementation of this education. Although the provider survey conducted had low completion rates, the concerns noted with the quality and number of phone calls received by pharmacists will be addressed in the future by creating scripting for phone calls to allow for more direct, pertinent discussion with providers. This educational program consisted of 8 hours of education provided over a total of 10 weeks, and in the future allowing the education to be spread across a longer time may be beneficial. Because of study time constraints, there was limited post-education time for the assessment of order verification times and the provider survey. This limited time may have contributed to low provider completion rates along with the inability to detect a difference in the order verification times, because many pharmacists were still completing the education during that time.
Positive feedback was received from pharmacists who participated in the education program. Pharmacists stated that they felt it enhanced their ability to provide patient care and improved patient outcomes in neonatal and pediatric patients. In particular, pharmacists felt the targeted nature of the education, which was specific to our institution, was helpful. This supports the current understanding of adult learning, because adult learners prefer to acquire information that is applicable and practical. For the education, the electronic health record was incorporated, which addressed another foundational principle of adult learning because it applied to the pharmacists' daily workflow. This education was created in an online format and was easily accessible to the entire pharmacy department. This allows for ongoing education and ease in updating the modules as practice and guidelines change in the future. Most module topics in this program specifically addressed the pediatric patient population, with only module 8 addressing neonates. The participants in this study frequently reported higher confidence scores for pediatric patients compared with neonatal patients from before to after education. With future education, the neonatal population may warrant more focus to address this lower confidence. Additionally, modules 4 and 5 had the lowest overall scores with the highest number of retakes. Both modules 4 and 5 contained the most disease states with the most clinical scenarios presented. Quiz questions for these modules were generated based on “real-life” scenarios in the emergency department setting for antibiotic dosing. This required extensive electronic health record involvement with minimal provider documentation for guidance, which may be less familiar for pharmacists not primarily in a clinical role.
Based on feedback from the participants in this educational initiative, the pharmacy department plans to incorporate this pediatric education model to maintain the staff's pediatric knowledge base, and completion of these modules will be a requirement for all new hires in the future. Continual feedback, updates, and improvement will enhance the quality of the education and improve outcomes. In the future, a formal assessment in change of neonatal and pediatric medication safety-related events after implementation of this program will be considered.
In conclusion, this study demonstrated statistically significant improvement in confidence and competence related to pediatric and neonatal pharmacotherapy after implementation of a pediatric education program.
Acknowledgments
This research was presented as a platform presentation at the Great Lakes Pharmacy Resident Conference at Purdue University in West Lafayette, IN, in April 2018 and as a poster at the American College of Clinical Pharmacy Virtual Poster Session in May 2018.
ABBREVIATIONS
- ASHP
American Society of Health-System Pharmacists
- NICU
neonatal intensive care unit
- PICU
pediatric intensive care unit
Footnotes
Disclosures Drs Ferguson, Shields, Young, and Trovinger declare no conflicts or financial interest in any product or service mentioned in the manuscript, including grants, equipment, medications, employments, gifts, and honoraria. Dr Kovey reports non-financial support and personal fees from the American College of Clinical Pharmacy outside the submitted work.
REFERENCES
- 1.Boucher EA, Burke MM, Johnson PN et al. Minimum requirements for core competency in pediatric pharmacy practice. J Pediatr Pharmacol Ther. 2015;20(6):481–484. doi: 10.5863/1551-6776-20.6.481. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Meyers R, Costello-Curtin J. Implementing a pediatric pharmacy educational program for health-system pharmacists. Am J Pharm Educ. 2011;75(10) doi: 10.5688/ajpe7510205. Article 205. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Accreditation Council for Pharmacy Education Accreditation standards and guidelines for the professional program in pharmacy leading to the doctor of pharmacy degree. https://www.acpe-accredit.org/pdf/FinalS2007Guidelines2.0.pdf Accessed September 19, 2019.
- 4.Accreditation Council for Pharmacy Education Guidance for the accreditation standards and key elements for the professional program in pharmacy leading to the doctor of pharmacy degree. https://www.acpe-accredit.org/pdf/GuidanceforStandards2016FINAL.pdf Accessed September 19, 2019.
- 5.Prescott WA, Dahl EM, Hutchinson DJ. Education in pediatrics in US colleges and schools of pharmacy. Am J Pharm Educ. 2014;78(3) doi: 10.5688/ajpe78351. Article 51. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Bhatt-Mehta V, Buck ML, Chung AM et al. Recommendations for meeting the pediatric patient's need for a clinical pharmacist: a joint opinion of the pediatrics practice and research network of the American College of Clinical Pharmacy and Pediatric Pharmacy Advocacy Group. J Pediatr Pharmacol Ther. 2012;17(3):281–291. doi: 10.5863/1551-6776-17.3.281. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Eiland LS, Gumpper KF, Heigham MK et al. ASHP-PPAG guidelines for providing pediatric pharmacy services in hospitals and health systems. Am J Health Syst Pharm. 2018;75(7):e136–e150. doi: 10.2146/ajhp170827. [DOI] [PubMed] [Google Scholar]
- 8.Cooper AZ, Richards JB. Lectures for adult learners: breaking old habits in graduate medical education. Am J Med. 2107;130(3):376–381. doi: 10.1016/j.amjmed.2016.11.009. [DOI] [PubMed] [Google Scholar]
- 9.Morris JL, Knoderer CA. Assessment of web-based training modules on learning facilitation for advanced pharmacy practice experiences in pediatrics. J Pediatr Pharmacol Ther. 2011;16(3):210–215. doi: 10.5863/1551-6776-16.3.210. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Gerber P, Carr R. A pharmacokinetics module taught within a pediatrics pharmacotherapy course. Am J Pharm Educ. 2013;77(6) doi: 10.5688/ajpe776126. Article 126. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Tofil NM, Benner KW, Worthington MA et al. Use of simulation to enhance learning in a pediatric elective. Am J Pharm Educ. 2010;74(2) doi: 10.5688/aj740221. Article 21. [DOI] [PMC free article] [PubMed] [Google Scholar]