Ability of Pharmacy Students, Pharmacists and Pharmacy Support Staff to Manage Childhood Fever via Simulation

Abstract

Objective. To ascertain how pharmacy students (novices) and pharmacy staff (experts) respond to a childhood fever scenario.

Methods. Data were collected from 65 second year students and 51 fourth (final) year students in an over-the-counter fever scenario during assessment tasks. Data from pharmacy staff were collected via mystery shopping conducted over nine weeks between March and October 2015. All encounters were immediately scored by the trained simulated client, and immediate feedback was provided for pharmacy staff and fourth year students. Questioning scores and proportions of competent participants were collected in all groups. Statistical comparative analyses were made between fourth year students and pharmacy staff. Pharmacy staff scores were also tracked over time.

Results. Second year students performed well, achieving a median questioning score of 100%. Conversely, pharmacy staff scored 22%. A large proportion of the fourth year students and pharmacy staff achieved appropriate outcomes (92% and 65%, respectively); however, a smaller proportion of second year students performed well (52%). The pharmacy staff achieved statistical improvements over time for median questioning scores.

Conclusion. Protocol compliant questioning appears to decline with experience. However, experienced counselors are more likely to provide appropriate patient advice. Further improvements in outcomes can also occur when staff are provided feedback and coaching. Mystery shopping simulations can be used as a valuable educational tool.

INTRODUCTION

The normal temperature of the human body is widely accepted as being 37°C.¹ It is regulated via the hypothalamus using a series of feedback mechanisms to ensure that a constant basal temperature remains.² During fever, there is an elevation of the body temperature due to the activation of physiological and behavioral responses reacting to invading microorganisms.² Although there has been much debate as to what temperature indicates fever, it is now widely accepted that temperatures of 38°C or above signal fever.³

Fever is described to have three main phases. The first phase is elevation of body temperature. This is achieved physiologically via vasoconstriction of blood vessels, reducing heat loss from the surface of the skin, and shivering whereby muscle contractions produce heat. The second phase of fever is characterized by a plateau in the elevated body temperature – the amount of heat lost is equal to the amount produced. The third phase of fever is the return of body temperature to normal. This is achieved via vasodilation and sweating allowing for the loss of excess heat.⁴

Fever is an extremely common childhood symptom of illness and accounts for up to 20% of pediatric visits to doctors.² Fever is usually caused by viral infections, the most common being an upper respiratory tract infection (URTI).² It is a natural and, more importantly, a beneficial response to invading microorganisms, which is often misunderstood by patients and caregivers. An elevated body temperature can cause an inhibition of growth and denaturation of proteins in some microorganisms,⁵ thus, preventing their pathogenic activity. It has also been found that an increased body temperature is able to enhance the performance of a host’s defenses including effects on innate and acquired immunity and cytokine expression.⁶ However, the positive and beneficial effects of fever remain unknown by the majority of the public and ‘fever phobia’ is experienced instead.⁷

‘Fever phobia’ describes the unnecessary worry and fear experienced by the caregivers of a febrile child.⁸ It is perceived by many that fever is a dangerous disease that has the potential to cause brain damage, blindness, and seizures if left untreated, death.⁹ These misconceptions will often lead to the mismanagement and inappropriate treatment of fever, such as the use of inappropriate medicines, frequent temperature readings and overuse of health care services.¹⁰ An example of this includes the routine use of antipyretics given to febrile children even though there is no evidence to suggest they reduce the morbidity of fever.¹¹ Antipyretics such as acetaminophen (paracetamol) are able to moderately lower the core body temperature by approximately 1-2°C and may provide some analgesic relief,¹² although strong evidence for this is still lacking. Often febrile children do not exhibit signs of distress, however symptoms such as muscle pain, headaches and restlessness can be experienced and this becomes more common at temperatures of above 41°C.² It is in cases such as these that antipyretic medicine seems a reasonable option.¹³

An Australian study using a simulated scenario with caregivers showed that only 23% of caregivers interviewed managed the fever scenario appropriately, with many using antipyretics before they were needed and making dose errors.¹⁴ El-Rhadi stated that “Parental education is critical in the management of febrile children.”² As antipyretic medicines for children are available from community pharmacies, this is one avenue where parental education could be given. Community pharmacies are located in most towns and cities, the majority of which offer pharmacist consultations that run on a ‘drop in’ basis, making access to professional health care knowledge easily accessible to the public.^15,16 Community pharmacies can play an important influential role in the education and management of minor ailments. This is done through the supply of non-prescription medicines, patient advice and referral to other health care practitioners when appropriate.^17,18 Therefore, it is important to ensure that pharmacists and pharmacy support staff are well-equipped to manage different disease states and ailments, including childhood fever, encountered in the pharmacy and that they follow evidence-based treatment guidelines to ensure that their patients receive the best possible health care.^19,20

An objective of pharmacy education curricula is to adequately prepare pharmacy graduates to provide appropriate advice for non-prescription medication requests. Undergraduate students at The University of Sydney in their second year of the Bachelor of Pharmacy (BPharm) degree are trained in communication skills and several minor ailment topic areas, including fever. This didactic learning followed by the opportunity to role-play in class culminates in an oral-simulated patient-examination to assess if students can put their knowledge and skills into practice. In the fourth year of the BPharm curriculum, students undertake a series of oral assessments in their final semester including an ‘on the spot’ assessment testing their skills in handling non-prescription medicine requests. These assessments further test the retention of skills learned in the second year. In order to determine whether pharmacy curriculum adequately prepares pharmacy graduates for dealing with non-prescription medication requests, including fever, it is important to assess student competency.

The aim of this study was to ascertain how competent pharmacy students and pharmacy staff members are to manage childhood fever using community pharmacy-based simulated scenarios. The specific objectives were to measure current clinical competency of students (novices) in assessment tasks, as well as current clinical competency in practice by observing the practice of pharmacists and pharmacy support staff (experts) using patient simulation over time. Furthermore, in order to identify where to target future intervention, our secondary objective was to compare student results with pharmacist and pharmacy support staff results.

METHODS

Ethics approval to collect student data was obtained via The University of Sydney Human Research Ethics Committee. Approval to collect data from pharmacist and pharmacy support staff was obtained through a separate application to the same research committee.

Data were collected from pharmacy students undergoing a four year Bachelor of Pharmacy degree at The University of Sydney between July 2015 and November 2016. The data were collected from students across two different year groups.

Students enrolled in the ‘Pharmacy Practice’ (PHAR2822) in their second year of study undergo a five-minute oral open book examination which contributes to 30% of the final unit grade. The unit of study PHAR2822 explores the role of the pharmacist as a primary care provider for disease states and the provision of management options, including non-pharmacological recommendations. Content in this unit of study is disseminated via lectures, tutorials, online learning and podcasts from August through November (Semester two) and culminates in the oral assessment where students are randomly allocated to one of four scenarios from 10 possible topic areas.²¹ These scenarios consist of a symptom-based request and students are able to refer to lecture notes and textbooks during the examination. Students were scored using a rubric where their assessment, analysis and advice, including body language, rapport and phrasing, of the scenario were all scored. The rubric was based on the International Pharmaceutical Federation (FIP) patient counselling rubric.²² All examiners were familiar with the scenarios and had a University qualification in pharmacy. Examinations were audio-recorded for audit.

Data from students allocated a fever examination scenario in November 2016 were collated and analyzed. Due to the use of a different scoring system for these second year students compared to the fourth year students and pharmacy staff, direct statistical comparisons were not made between second year students and the other groups. However, descriptive statistics have been reported alongside data from the other groups.

Student competency data on fever management were also collected from students enrolled in the fourth and final year of the BPharm degree at The University of Sydney. These students were enrolled in the Professional Practice unit (PHAR4821). This unit of study consolidates learning from previous courses across the four years of the curriculum, through the presentation and solving of clinical and ethical problems.²³ Between August and November in both 2015 and 2016, students were allocated to eight competency assessments, one of which was handling a request for a non-prescription medicine. Fifty-one students were randomly allocated to receive the fever scenario. These students were given an ‘on the spot’ assessment (maximum 30 minutes for assessment and feedback) on handling a fever scenario in their tutorial class that accounted for 5% of their final grade for that unit. The assessment was scored using a standardised data collection score sheet based on the WHAT-STOP-GO protocol, as part of the Pharmaceutical Society of Australia’s Standards for the Provision of Pharmacy Medicines and Pharmacist Only Medicines in Community Pharmacy.²⁴ This score sheet had been based on a previous mystery shopping project conducted in community pharmacy.²⁵ Consistent with the second year examinations, these assessments were performed by individuals holding a pharmacy degree from a university, were trained in the scenario, and the interactions were also audio-recorded to validate scoring across the four assessors.

In order to measure the current clinical competency of pharmacy staff in community practice, data were gathered during a mystery shopping study where third year BPharm students from The University of Sydney acted as mystery shoppers. This study took place in the Sydney metropolitan region between March and October 2015. A total of 36 individual pharmacies were recruited and 10 different common ailment scenarios were used, including fever. Each participating pharmacy was randomly allocated a common ailment and was visited by a different mystery shopper each week for nine consecutive weeks. For the purpose of this study only the data from the fever scenarios were extracted. The fever scenario data collection sheet was in the same format as the data collection sheet used for the fourth year students’ competency assessment. The following scenario variables were altered each week: who the medicine was for (child from 2-6 years of age), the product requested (brand or therapeutic class, ibuprofen or acetaminophen-containing liquid product), and the symptoms the patient was experiencing. Student mystery shoppers underwent a two-day practical and theoretical training program in order to prepare for the mystery shopping visits. This included conducting role-plays, providing training for both the mystery shop and also the provision of feedback to staff, filling in of data collection sheets and familiarizing students with the scenarios. This training was to improve consistency in data collection and performance as a mystery shopper between students. Mystery shoppers conducted the visits by entering the community pharmacy and requesting the product directly, either from a pharmacist or pharmacy support staff. Each pharmacy had consented to take part in the study, and each individual staff member had also provided consent to be audio-recorded prior to the visits taking place. Staff were made aware of the study but were not given exact times they mystery shopping would take place or which scenario they would be allocated. If a staff member was encountered that had not provided consent to audio-recordings, then they were immediately erased; and the data were destroyed. At the end of the interaction mystery shoppers purchased any products recommended to them by the staff member. Mystery shoppers then exited the pharmacy and completed the data collection sheet to score the staff member. Once complete, the mystery shopper re-entered the pharmacy to provide immediate feedback which was also audio recorded. Mystery shoppers also scored other aspects of the interaction; counseling on the use of the medicine, perceived rapport built with the consumer, provision of written or verbal information, and referral to a practitioner and who served the mystery shopper. Data from these mystery shopping visits were collected from audio recordings of the visits and scored by registered pharmacists in the research team. Visits with missing or inaudible recordings were excluded from analyses and this report.

For all of the score sheets (second year, fourth year and pharmacy staff) each criterion was scored yes (2 points), partial (1 point), no (zero points), or not applicable. The score was converted to a percentage, accounting for the number of non-applicable items in each particular case. To determine whether an overall appropriate outcome had been reached, students/pharmacy staff could either: refer the patient (fourth year scenarios), provide appropriate dosing based on weight or not provide a product but ensure that appropriate fever management and non-drug advice was given (second year and pharmacy staff scenarios).

Data obtained from the mystery shopping visits, second year BPharm oral examinations and fourth year BPharm assessments were entered into Microsoft Excel 2016 for Macintosh OSX (Microsoft Corp, Redmond, WA). Data were then descriptively analyzed to obtain the median questioning scores (WHAT scores) with respective interquartile ranges, and overall appropriateness of each case. The WHAT score is made of scores for eliciting the appropriate information including who the medicine is for, how long the patient has had the symptoms, the actual symptoms, and what other treatment they have tried or are currently taking for any medical conditions.

Statistical analyses of the data were performed using IBM SPSS 24.0 (SPSS Inc., Chicago, IL). The pooled data were first tested for normality and then several analyses were performed accordingly. Non-parametric independent-sample median tests were performed to determine the difference in median what scores between all groups of subjects as well as between pairs of groups of subjects with the exception of the second year data which was not compared due to the different scenario and score sheet used. Two-tailed point-biserial correlations were calculated to determine if there were any significant differences between the overall appropriate outcome of the student and pharmacy staff groups and pairs of groups. Finally, a two-tailed Spearman’s rank correlation was undertaken to ascertain if there was a relationship between visit number and WHAT score for the pharmacy mystery shopping cases.

RESULTS

Student (novice) results in questioning scores and proportion of student overall appropriateness.

The second year students achieved a median WHAT score of 100% (IQR=88-100) signifying that a vast proportion of the second year students attained full marks in consumer questioning. Fourth year students achieved a median WHAT score of 78% (IQR=67-89) (Table 1). In second year students, 52% managed the scenario appropriately, whereas 92% of fourth year students provided an appropriate outcome (Table 2).

Table 1.

Median Questioning Scores and IQR for Pharmacy Staff and Fourth Year Students

Table 2.

Proportion of Individuals Achieving an Appropriate Outcome

Forty community pharmacy mystery shopping visits were eligible for analysis from available audio recordings. When looking at the median WHAT scores per visit we observed a median baseline score of 11% (IQR=3-216) which peaked at week 8, with pharmacy staff achieving a median WHAT score of 44% (IQR=11-94). A Spearman’s rank correlation demonstrated that median WHAT scores significantly improved over time from visit 1 to visit 9 (p<.001) (Table 3).

Table 3.

Pharmacy Staff Performance Over Time

As shown in Table 1, the median WHAT score for all visits regardless of level of staff member or visit number was 22% (IQR=11-56). When looking solely at scores for pharmacists, the median was 33% (IQR=14-56; N=31), and for pharmacy support staff it was 22% (IQR=11-27; N=22).

Table 3 shows that the proportion of pharmacies providing the consumer with an appropriate outcome (either correct weight/based dosing or non-pharmacological advice or both) varied from as high as 100% of pharmacies managing the scenario to as few as 20%.

The proportion of visits resulting in an appropriate outcome regardless of staff level was 65% (N=26). When looking at pharmacists and pharmacy staff individually it was found that pharmacists had a higher chance of providing an appropriate outcome at 76% (N=29) than the pharmacy support staff at 36% (N=11). However, these proportions were not found to be statistically different ( p=.02).

Using non-parametric independent samples median tests it was determined that there were statistically significant differences between the median WHAT scores for fourth year students and both groups of pharmacy staff (p<.001). Post hoc tests showed that there were statistically significant differences between the median WHAT scores of groups of both groups of pharmacy staff and fourth year students, but not between pharmacists and pharmacy support staff (Tables 1 and 4).

Table 4.

Comparison of Median Questioning Scores Between Groups

Using point-biserial correlation, there were statistically significant differences in the proportions of subjects demonstrating an appropriate outcome for the consumer. The proportion of fourth year students managing the scenario correctly was significantly higher than pharmacists and pharmacy support staff (Tables 2 and 5).

Table 5.

Comparison of Proportions of Individuals Achieving an Appropriate Outcome by Group

DISCUSSION

This study allowed us to evaluate the questioning scores and overall competency in managing childhood fever scenarios in students (novices) and in pharmacy staff of both pharmacists and assistants (experts). The second year students achieved a median WHAT score of 100%, however only 52% of the students achieved an overall appropriate outcome. The fourth year students had the highest percent of participants achieving an appropriate outcome at 92% and a median WHAT score of 78%. The pharmacy staff scored the lowest median WHAT score of all the groups tested with pharmacists scoring 26%, and the pharmacy support staff 11%, however the percentage of participants achieving an appropriate outcomes was 65% overall with 76% of pharmacists managing the scenarios appropriately and 36% of support staff providing appropriate weight-based or non-pharmacological advice. The difference in proportions of appropriateness between pharmacists and pharmacy support staff was significantly different. Looking at pharmacy data over time, the median WHAT score significantly improved over time from visit 1-9.

Second year students performed well at adhering to the WHAT section of the WHAT-STOP-GO protocol and were able to ask the majority of the questions outlined in the score sheet, as their median WHAT score was 100%. Thus, it is the second year pharmacy students that are gaining the most information about their specific fever scenario. Based on this, it could be assumed that they are in the best position possible to provide the correct advice and counseling to achieve an appropriate outcome; however, we can see that this is not the case, as a vast proportion of these novices were not able to use the information gathered and apply their knowledge, resulting in only 52% of the cohort achieving an appropriate outcome. The behavior exhibited by the second year students demonstrates that second year students “know”²⁶ how to ask appropriate questions in a self-care pharmacy scenario, based on memorizing the WHAT section of the WHAT-STOP-GO protocol. However, without the corresponding content knowledge “knows how” and the “shows how”²⁶ of calculating pediatric doses resulting in nearly half of them unable to appropriately manage the scenario. The fourth year students, who are perhaps higher level novices, have progressed to the higher stages of “Miller’s Pyramid of Clinical Competence”²⁶ and are exhibiting clinical competencies in all aspects at the “shows how”’ level. Although the median WHAT score was 78%, the proportion of students obtaining an appropriate outcome was 92%. This suggests that while the fourth year students have gathered less information, the information obtained has been applied correctly to advise and manage the given fever scenario.

One of the possible reasons for the dramatic improvement of appropriate outcome scores from the second year to the fourth year could be increased experience and repetition when dealing with non-prescription medicine requests. As the clinical placements (professional experience) program does not commence until the third year of the degree program, continuing into the fourth and final year, it can be hypothesized that exposure to the work environment and being presented with ‘real-life’ opportunities to practice clinical assessment and decision making may account for the improvement in appropriateness seen in fourth year students.

This, coupled with the added experience and learning of two extra years of BPharm curriculum may allow the fourth year student to improve in the their information processing and decision making when dealing with non-prescription medicine requests, such as requests for products to manage fever.

When looking at the data collected from the pharmacy staff during the mystery shopping program, the median WHAT scores observed for both the pharmacists and the pharmacy support staff were significantly lower than the fourth year student scores, achieving only 26% and 11% respectively. As the majority of pharmacy staff performed so poorly when questioning the mystery shopper, gaining very little information about their particular fever scenarios, it could be predicted that there would be a high percentage of inappropriate outcomes. Adversely, the proportion of pharmacy staff attaining an appropriate outcome were 76% (pharmacists) and 36% (pharmacy support staff) showing that a large number of pharmacists were able to manage the fever scenarios correctly despite asking less questions. The WHAT score results and the corresponding appropriateness observed in this study is interesting, with the highest WHAT scores in second year not corresponding to appropriate outcomes, and the lower WHAT scores in pharmacy not necessarily predicting the appropriate outcome. This is in contrast to previous research using mystery shopping assessments. Schneider and colleagues showed a correlation between questioning and appropriate outcome in a mystery shopping study involving a request for salbutamol for asthma.²⁷ Watson and colleagues also found a similar correlation when examining the supply of non-prescription antifungals for vaginal thrush.²⁸ Collins and colleagues also showed a correlation between appropriate referral to a medical practitioner and the number of questions asked.²⁹ What we did observe, however, is that in pharmacy scenarios that ended in inappropriate advice, the number of questions was two or less. Perhaps the variation seen in the lack of correlation suggests that for different scenarios there are key questions that must be asked, rather than perhaps all questions. For example, all pharmacy students, pharmacist and pharmacy support staff are trained to ask about other medicines and other medical conditions a patient has.

Whilst these are important safety questions for many scenarios, in the case of childhood fever and a request for acetaminophen, these questions were not crucial to ask in order to arrive at an appropriate outcome (correct dosing, non-pharmacological actions, or referral). This perhaps suggests that there may need to be adapted protocols for different types of requests, and perhaps this adaption is occurring naturally with scores decreasing from second year through to practice. In fact, a recent study by Akhtar and Rutter, showed that the protocol used in the UK known as WWHAM was not necessarily a good predictor of outcomes in a vignette given to pharmacists and that a rethinking of protocols and the use of decision making algorithms may be warranted in the future, particularly as one moves from novice to expert.³⁰

Furthermore, a factor that also should be taken into consideration when looking at the student scores is that ‘tests and examinations drive student learning,’ a statement first coined by Muijtjens and colleagues.³¹ Both student groups were primed as they were aware that they were being graded on their performance, unlike the pharmacy staff that were aware that they would be partaking in the mystery shopping scheme, but didn’t know when the mystery shopper was going to enter the pharmacy. It has been found that testing students leads to an increase in the amount of study time and improvements in study strategies; thus, student scores are expected to be higher compared to the pharmacy staff as they were more prepared.³²

The pharmacist and pharmacy staff results found in the study are also similar mystery shopping studies, often finding that pharmacy staff do not ask a sufficient amount of questions.^19,33,34 There may be several reasons for the lack of questioning in a community pharmacy setting. One of these reasons is the increased pharmacy productivity. There has been an increase in the percentage of the public that a single community pharmacy needs to provide for,³⁵ as well as continual increases in the amount of non-prescription medication sales,³⁶ thus increasing the burden on community pharmacies and potentially decreasing the amount of time available to be spent with consumers. Further, as the Australian community pharmacy system does not operate on an appointment system,³⁷ pharmacy staff may be more inclined to be increasingly efficient with their patient consultations especially around periods of day such as lunch time as they are unable to predict when the pharmacy will get busy, and they may want to provide the same continuity of care for all of their patients.

Lastly, it has been found that patients often don’t want advice when purchasing non-prescription medicines. In one paper it was found that two of the main reasons for not wanting advice was that the customers had “used the medicine before with good results” and “had already received advice elsewhere.”³⁸ With the mix of clientele, it is extremely difficult for pharmacists and support staff to determine which consumers need assistance and those that don’t, further highlighting the importance of some initial assessment questions. However, through the use of mystery shopping, our study did show an improvement in the performance of pharmacy staff over time. When looking at the median WHAT scores per visit it was found that there was a significant improvement from visits 1-9 despite fluctuations.

The high proportion of visits resulting in an appropriate outcome for pharmacies and fourth year students was encouraging. As fever is a symptom that all children will suffer and all caregivers will need to manage, it is promising that weight-based as opposed to age-based dosing was seen in a large proportion of the encounters in this study. However, this weight-based dosing advice contrasts with parents actual actions seen in a previous study¹⁴ so understanding how a caregiver interprets dosing information is important research to pursue. It should also be noted that many children with fever do not require antipyretic treatment and that over treatment of fever due to fever phobia and misconceptions is not just a caregiver issue but many health professionals also lack appropriate knowledge.^39,40 Our scenarios could be modified in future to see if indeed pharmacists and staff are equipped with evidenced-based knowledge to assist caregivers with more documented and scored data on the non-drug advice given as these are also important aspects of fever management.

In this study the data collection tools, such as the scoring rubrics, were similar for fourth year students and pharmacy staff cohorts, allowing data to be easily analyzed and compared; however, it should be noted that the scoring rubric for the second year students did contain some differences to accommodate for the marking criteria needed to assess their performance based on the curriculum; hence, these data were unable to be directly compared. Similarly, the way in which the data were collected was comparable across the cohorts by using role-play environments. However, one limitation was that not all the cohorts received the same degree of difficulty when dealing with their given scenarios. The second year students received a ‘symptom-based request’ scenario as opposed to ‘direct product request’ scenario that the fourth year students and pharmacy staff received.⁴¹

Symptom-based requests have been found to evoke more advice and counseling from pharmacy staff members than a product based request making this an easier scenario to handle,⁴² yet, they are still poorly executed by many second years. It must also be considered that fourth year students and pharmacy staff had the product packaging in front of them which may have aided them in calculating appropriate weight-based doses; whereas, the second year students did not have access to the packaging and had to rely solely on their notes or their memory as well as available product strengths. Further, as already alluded to, the students in this study were aware that they were being assessed on a particular day and were therefore able to prepare for the encounter in contrast to the pharmacy staff giving students an advantage over scoring.

CONCLUSION

The findings from this study suggest that final year pharmacy students are appropriately educated and equipped to manage and advise on fever scenarios and have moved from a novice to an expert learner. In practice, patients may receive varying levels of questioning from experts when presenting with a fever scenario; however, despite less questions, they may still receive appropriate weight-based dose information. Incorporating mystery shopping and simulation as an educational tool for the ongoing assessment and improvement of pharmacy staff may allow for enhanced learning and an improved and constant level or care.