Abstract
Objective
To design and evaluate a preregistration course utilizing asynchronous online learning as the primary distance education delivery method.
Design
Online course components including tutorials, quizzes, and moderated small-group asynchronous case-based discussions were implemented. Online delivery was supplemented with self-directed and face-to-face learning.
Assessment
Pharmacy graduates who had completed the course in 2004 and 2005 were surveyed. The majority felt they had benefited from all components of the course, and that online delivery provided benefits including increased peer support, shared learning, and immediate feedback on performance. A majority of the first cohort reported that the workload associated with asynchronous online discussions was too great. The course was altered in 2005 to reduce the online component. Participant satisfaction improved, and most felt that the balance of online to face-to-face delivery was appropriate.
Conclusion
A new pharmacy preregistration course was successfully implemented. Online teaching and learning was well accepted and appeared to deliver benefits over traditional distance education methods once workload issues were addressed.
Keywords: asynchronous, preregistration, online, online discussion, Australia
INTRODUCTION
Pharmacy graduates in many countries are required to complete a period of supervised practice before becoming eligible for registration as a pharmacist. In Australia, graduates must complete the equivalent of 1 full-time year of supervised preregistration training (internship) and a part-time preregistration course that is designed to assist them in integrating their undergraduate learning into real-world practice and transition from student to independent competent pharmacist.1 They must also pass written and oral competency-based registration examinations.
Since preregistrants usually work full-time and may be many miles from an education provider, courses usually use distance education as a delivery method. Preregistration education in Australia has historically been delivered through traditional distance education methods such as self-directed learning and written assignments, supplemented by face-to-face education. With traditional distance education modalities, preregistrants are not easily able to connect with their fellow students and teachers to share experiences and learn from each other, and they may feel isolated and unsupported. The development of online teaching and learning technologies has allowed distance education to become more dynamic and interactive, therefore overcoming some of these issues.2,3
Although aspects of pharmacy education have been delivered online in various ways over recent years,2,4,5 no published literature describes online teaching and learning in a pharmacy preregistration course. This paper describes the development and evaluation of the first 2 years of the Monash University Pharmacy Preregistration (Internship) Course, which includes a significant online distance education component.
DESIGN
The course concept and initial framework was designed by staff members from the Monash University Department of Pharmacy Practice. An advisory committee consisting of a representative of the pharmacy registering authority, a hospital pharmacist, a community pharmacist, a rural pharmacist, and a preregistrant was established to guide course development. The curriculum was based on Australian competency standards, practice standards, and guidelines for pharmacists. Teaching modalities were selected to enable course objectives to be met while also maximizing student interaction and taking into consideration concerns in the profession about the suitability of online course delivery. A pharmacist with experience in hospital and community pharmacy and pharmacy education was engaged to develop and implement the curriculum. Course learning objectives are described in Table 1. The course was approved by the local registering authority, the Pharmacy Board of Victoria.
Table 1.
Preregistration Course Learning Objectives
Curriculum Delivery
Online course components were delivered using WebCT (Universal Learning Technology, Boston, MA), a Web-based learning management system that allows students to access course materials and online discussions at any time from any computer with Internet access.
Asynchronous online discussion was chosen as the main distance education modality because it provides group interaction so that students can share ideas and experiences, and it enables students to interact at any time convenient for them. In contrast to face-to-face meetings, asynchronous discussion also gives students an equal voice and time to research the topic and consider their answers before responding.
Self-directed and face-to-face activities were also included to supplement the online component and address topics that could not be covered online (eg, communication skills). The overall course structure is shown in Table 2.
Table 2.
Course Structure 2004-2005
Curriculum Components
Small group online discussions.
Discussion groups included 10 to12 preregistrants from a mix of hospital and community practices in metropolitan and rural areas, plus a practicing pharmacist as discussion moderator.
Discussion modules were developed for asynchronous use over 3 to 4 week blocks, with a 1-week break between blocks. Modules were designed to generate discussion about professional practice issues, reinforce and expand undergraduate knowledge, and help preregistrants apply their knowledge in day-to-day practice. Most topics were presented as case studies, which encouraged students to analyze real-life situations and develop problem-solving skills while sharing ideas, experiences, and practices. Cases presented scenarios in which therapeutic, legal, ethical, social, and workplace-based issues arose. They were divided into parts, and moderators contributed further information from the discussion guide to progress the cases. Two cases ran concurrently in each discussion block. Preregistrants were expected to log on 3 to 4 times each week and contribute to each case at least twice per week. Moderators assessed students' performances based on quality and regularity of contributions.
Moderators were pharmacists recruited from hospital and community practice. They were provided with a comprehensive manual including a discussion guide to inform their responses and build consistency across groups. They received face-to-face tuition on moderating online discussion groups and using WebCT. Their role is summarized in Table 3.
Table 3.
Role of Online Discussion Moderators
Online open discussion forum.
An open discussion forum, moderated by the course director, was accessible to all course participants. There were no set topics and contribution was optional. It was primarily for preregistrants to share information and experiences, and discuss practice-related questions.
Pharmaceutical calculations module.
The calculations module was developed following concerns from the profession and the registration authority about pharmacy graduates' ability to perform calculations. Computer-assisted tutorials were developed, along with multiple-choice practice quizzes, and an assessment quiz. Questions were randomly generated to ensure that each quiz was unique. Upon quiz completion, preregistrants received immediate computer-generated feedback. For practice quizzes, this included their scores and fully worked answers, and for assessment quizzes, they received only their scores.
Pharmacy law online module.
The law module consisted of hyperlinks to pharmacy law-related resources (eg, local pharmacy legislation and practice-guidelines), a series of practice multiple-choice quizzes and an assessment quiz. These were designed to supplement pharmacy law content delivered through online discussion cases and face-to-face seminars. As with the calculations quizzes, preregistrants received immediate computer-generated feedback upon completion of each quiz.
Face-to-face seminars.
Seminars consisting of lectures and small group tutorials were held at the university in multi-day blocks to minimize travel for nonmetropolitan students. Topics included pharmacy law, evidence-based practice, primary healthcare, therapeutics, problem solving, and communication and counseling. Tutors were experienced pharmacists recruited from hospital and community practice. They were provided with comprehensive notes to ensure consistency between groups. To further develop preregistrants' communication skills, some tutorials included role playing with lay people who simulated patients in typical healthcare scenarios.12
Small group meetings.
For many years prior to this course, pharmacist-facilitated face-to-face small group meetings had been used in Australian preregistration education as a forum for preregistrants to share experiences and learn from each other and practicing pharmacists. Although we believed that moderated small group online discussions could replace these meetings, they were included in the 2004 course to address doubts expressed by the profession during course development. Preregistrants were assigned to a local group, but this was not always possible for rural preregistrants.
Practice oral examination.
Pharmacy students first encountered competency-based oral examinations in their preregistration year, and to assist in developing their oral examination technique, practice examinations were developed and provided to preregistrants' workplace preceptors, along with administration instructions and an answer guide.
National Prescribing Service (NPS) case study and audit.
The NPS (www.nps.org.au) is the service agency that ensures the quality use of medicines for Australia's National Medicines Policy. The NPS develops resources and activities to assist health professionals to provide evidence-based patient care. Each year the NPS provides clinical case studies and practice audits. Participants receive feedback and expert commentary, allowing comparison of their responses and practices with other participants and experts. Preregistrants were required to participate in 1 case study and 1 practice audit as part of the course.
EVALUATION AND ASSESSMENT
Two hundred thirty-one preregistrants completed the course over 2004-2005 (Table 4). An evaluation questionnaire and reply-paid envelope were mailed to preregistrants following the final registration examination (approximately 1 month after course conclusion) in each of these years. The questionnaire included multiple-choice questions and space for free-text comments. No reminders were sent to nonresponders because many would have relocated after registration. At the end of 2004 only, questionnaires were also sent to the online discussion moderators to assess their perceptions about the effectiveness of this education modality.
Table 4.
Preregistrant Demographics
Responses for 2004, along with feedback from the course advisory committee, tutors/moderators, and preceptors, informed changes to the 2005 course (Table 2). Responses from 2005 were compared with those from 2004 to evaluate the impact of changes made.
The response rate to the preregistrant evaluation was 40% (n = 48) in 2004 and 39% (n = 43) in 2005. The response rate to the moderator evaluation was 89% (n = 17).
Small group online discussions.
In 2004, 57% of preregistrants perceived they had benefited either greatly or moderately from the online small group discussions (Figure 1). Most preregistrants indicated that regular interaction with other preregistrants (79%) and with practicing pharmacists (86%) contributed to their learning and professional development. Rural preregistrants in particular benefited from this mode of course delivery. One student commented: “The online discussions were brilliant, as you were only an email/posting away from other students and everyone had interesting and different pointers that could be learnt.”
Figure 1.
Preregistrant's Perceived Benefit of Individual Program Aspects
Concerns were raised in 2004 about the workload associated with online discussions, with 56% of preregistrants indicating it was excessive. Preregistrants had 3 primary concerns: (1) A perceived need to log on more frequently than recommended. Some preregistrants felt they needed to log on every day to keep up or to “get in first” when new material was posted by the moderator. (2) The misconception that quantity or frequency of postings rather than quality was most important for assessment (although regular contributions were required, assessment was weighted toward quality of postings and ability of preregistrants to contribute constructively to a professional discussion). This led to some preregistrants making trivial contributions or repeating things, which increased reading time and caused frustration. (3) The relatively short 1-week break between most discussion blocks, leading to fatigue later in the year. Another factor contributing to repetitious postings, and therefore impacting preregistrants' experiences, was their difficulty adjusting to discussion-based assessment instead of the more familiar assignment-based assessments. Many preregistrants treated the discussions like an assignment in which there was one correct answer so if someone else posted a correct answer first, they felt there was nothing they could contribute. The students struggled to understand the requirement to discuss the issues raised and the various ways the scenario could be handled in practice, comment on each others' responses, and share related workplace experiences and practices. Most preregistrants (90%) reported logging on to discussions 2 to 5 days per week (54% logged on the recommended 3 to 4 days per week). Most (82%) reported spending 2 to 4 hours per week reading, researching, and contributing to the discussion topics.
The moderators' performance was important in the preregistrants' discussion experience. Although in 2004 most preregistrants rated the moderators' performance as excellent, very good, or good (4%, 29%, and 47%, respectively); 17% felt it was only fair; and 3% rated it as poor.
In 2005, to address preregistrants' concerns about workload, and a perceived imbalance between online and face-to-face course delivery, the number of discussion blocks was reduced from 8 to 7, and the break between blocks increased from 1 to 2 weeks on average. Additional explanation was provided to preregistrants about effective online discussion participation and assessment criteria, along with additional training to help moderators address preregistrants' concerns. Discussion topics were reviewed to ensure maximal scope for dialogue on each topic. These changes contributed to an improvement in preregistrants' perceptions of the small group discussions. In 2005, 79% reported great or moderate benefit from discussions (Figure 1). Feedback about moderator performance also improved: excellent (9%), very good (47%), good (40%), fair (5%), and poor (0). Although fewer preregistrants felt the workload was excessive in 2005 (32% vs. 56%), some remained concerned about workload and moderator performance.
Thirteen (76%) moderators felt small group online discussions were either greatly or moderately beneficial for the pre-registrants. Fourteen (82%) felt that providing a structured forum for preregistrants to regularly interact with peers from different practice settings was greatly or moderately valuable, and 14 (82%) felt this educational method was more effective than traditional written distance education assignments. Moderators commented that advantages of online discussion over written assignments were that it forced students to discuss topics instead of working independently, reduced potential for plagiarism, and provided access to peer support, sharing of experiences and resources, and timely feedback.
Open discussion forum.
Although participation was not mandated, the open discussion forum was frequently used by preregistrants, enabling them to share experiences and ask questions about any aspect of their practice and examination preparation. In 2005, 86% of preregistrants felt they benefited from this forum (17% great benefit, 43% moderate benefit, 26% a little benefit). This question was not included in the 2004 evaluation.
Pharmaceutical calculations module.
In 2005, the calculations module was modified to eliminate less practice-relevant calculations (eg, buffer solutions) and introduce a clinical calculations section (eg, creatinine clearance, body surface area). The module also changed from CD-ROM to WebCT delivery. These changes resulted in a substantial improvement in satisfaction (Figure 1). In 2005, 89% of preregistrants felt the module had increased their confidence with calculations (this question was not asked in 2004). Immediate feedback upon practice quiz completion was highly valued, with 92% of preregistrants rating this as very helpful. One student commented, “Immediate feedback for the quizzes was great – I only expected answers to be provided but working out showing steps taken to achieve the answer was very helpful indeed. It meant I could without delay work out what I did wrong.”
Pharmacy law online module.
More online quizzes and comprehensive Web links for legislation and guidelines led to improved satisfaction with the law module between 2004 and 2005 (Figure 1). In 2005, 71% of preregistrants felt the module had increased their confidence about pharmacy law matters (question not asked in 2004). As with the calculations module, immediate feedback was highly valued (84% said very helpful).
Small group meetings.
Although preregistrants found these meetings were beneficial (88% reported moderate or great benefit), timing and travel was problematic for many due to full-time employment, shift work, or dispersed location. Consequently, many preregistrants missed meetings or swapped groups. For this reason, and with the positive response to the online small group discussions in 2004, the face-to-face small group meetings were removed from the 2005 course.
Seminars.
To redress a perceived imbalance between online and face-to-face delivery, the number of seminar days was increased in 2005. One day was added to compensate for the reduced number of small group online discussions and another in lieu of small group face-to-face meetings. Instead of 3-day seminar blocks, 2-day blocks were introduced to reduce the gap between seminars and overcome preregistrant fatigue noted on the third consecutive day. The proportion of time spent in small-group activities was also increased (from 56% in 2004 to 67% in 2005). These changes led to increased satisfaction with this aspect of the course (Figure 1). Small group sessions using simulated patients were highly rated in both years.12
Practice oral examinations.
The opportunity to do a practice examination was highly rated in both years (Figure 1) and based on feedback, the number of examinations was increased from 1 to 2.
National Prescribing Service activities.
In 2005, 99% of preregistrants felt they had obtained some benefit from the NPS case study (38% great benefit, 40% moderate benefit, 21% a little benefit) and 98% felt they had benefited from the NPS self-audit (43% great benefit, 31% moderate benefit, 24% a little benefit). This question was not included in the 2004 evaluation.
Balance of online versus face-to-face course delivery.
Preregistrants were asked to rate how they felt about the online versus face-to-face balance. Of the 2004 cohort, 58% felt there was too much online delivery. When asked “if the amount of online discussion was reduced, would you prefer it to be replaced with written assignments, more seminars or more small group meetings?” 50% indicated a preference for seminars, 48% for small group meetings, and 13% for assignments (some preregistrants selected more than 1 option).
As a result of changes to the course structure described above, in 2005 the proportion of preregistrants who felt there was too much online delivery fell to 36% (Figure 2).
Figure 2.
Preregistrants' Perception of Amount of Online Course Content
DISCUSSION
A pharmacy preregistration course based on asynchronous online delivery was successfully introduced and well accepted by course participants and the profession. Course evaluation suggested that online teaching and learning in this preregistration context provided benefits including shared learning, peer support and immediacy of feedback.
Based on our experience, and supported by published research,3 online discussion is a powerful and flexible learning tool which delivers sound educational outcomes. Online discussion participation is central to the e-learning experience in terms of building understanding and providing social and contextual support.6,7 Establishing a strong online community has been identified as an important component to the success of distance learners.3,8
When designing the course we felt that online case-based discussions involving peers and practicing pharmacists would more effectively support learning than independent written assignments. We cannot assess whether this was the case since we have no comparative evaluation data from preregistration courses that use written assignments. However, feedback from preregistrants and moderators (most of whom had significant experience in preregistration training and preceptorship), and our own observation and experience, suggests this is likely. Most preregistrants felt the discussion groups assisted their professional development. They valued online discussion as a tool to explore professional issues, share experiences, and connect with colleagues. In contrast to face-to-face meetings or tutorials, asynchronous discussion groups allowed participants to interact with peers more frequently, at their convenience, and with an equal voice.
Regardless of the relative effectiveness of online delivery versus traditional distance education methods, there is little doubt that online delivery was successful in reducing preregistrant isolation and providing a forum for sharing experiences and learning from peers. Around 80% of preregistrants indicated they benefited from regular interaction with peers and pharmacists. The ability of technological course innovations to reduce preregistrant isolation is also supported by the increase in rural preregistrants choosing this course in 2005 compared with 2004 (Table 3).
Although preregistrant perceptions of the benefits of small-group online discussions improved between 2004 and 2005, this aspect of the course was rated lowest in both years relative to other components (Figure1). It is not known whether individually completed written assignments would have rated any higher, but in 2004 only 13% of preregistrants indicated that they would have preferred them in place of some of the online discussions. This suggests that online discussions are an acceptable alternative, and preregistrants likely will rate any workload-intensive distance education modality lower than face-to-face modalities.
Our findings suggest that workload impacted on preregistrant satisfaction with online discussions. However, the time that preregistrants spent participating in the online discussions was close to what we had expected (2 to 3 hours), so from the course provider's perspective the workload was not overly excessive. We felt the adverse feedback was primarily related to preregistrants' misunderstandings about discussion requirements and how to participate effectively, along with fatigue from repetitious postings and inadequate breaks between discussions. Especially in 2004, almost-continuous online discussions required more consistent input compared with written assignments. Although preregistrants considered this unfavorable, course providers saw consistent contribution as advantageous to learning. Some preregistrants felt compelled to respond within a self-imposed timeframe of daily, or even multiple times per day, which created pressure for them. This is partly because they treated discussion cases like a traditional assignment, and wanted to post the correct answer instead of discussing the scenario. Because small group discussions were assessed, some preregistrants felt compelled to contribute even when they had nothing new or constructive to add, sometimes leading to repetition and increased workload.
Transitioning from assignment-based assessment to assessment based on online discussion participation was a challenge for some preregistrants. At the outset, it is important for preregistrants to be informed of, and understand, the objectives and assessment criteria for such activities. This may become less of a problem in future years as online discussions become increasingly used at the undergraduate level in Australian pharmacy courses.
Moderators play a vital role in managing online discussions by regulating the discussion and shaping group behavior and culture.3,6,10,13 They instill enthusiasm, provide feedback, and make discussions more efficient by directing students against chasing false threads or engaging in inconclusive debates.9-11 Moderators also model appropriate group interaction, subject engagement, and message posting.6 Our experience strongly suggests moderator performance contributes to participant satisfaction and therefore learning. Preregistrants reported highest satisfaction when moderators contributed regularly – posting at the beginning to start the discussion; during to guide the discussion, deliver feedback, and provide expert comments; and at the end to summarize. Providing feedback was considered critical (eg, acknowledging good contributions, pointing out errors in posts if not picked up by another preregistrant), as was ensuring all preregistrants had an opportunity to contribute (eg, by directing questions to noncontributors). The importance of the moderator to the success of online discussions underlines the need to provide them with adequate training, guidance, feedback, and remuneration.
Pharmaceutical calculations and pharmacy law modules were the other significant online components, and these were developed as innovative solutions to deficiencies noted in preregistrants' abilities, as well as assisting preregistrants to prepare for their registration examinations. Almost 100% of preregistrants felt they benefited from these modules; particularly highly valued were immediate feedback and worked answers in the practice quizzes – features not easily deliverable with paper-based distance education formats.
Achieving an optimal balance of online versus face-to-face course delivery was important. Online education, like most distance education methods, cannot adequately address all the skills and attitudes required by a novice health professional. Despite regularly logging on and engaging with the course materials, online learners can feel disconnected from others participating in the course in the absence of face-to-face contact. Distance education programs therefore usually need to be supplemented with some form of face-to-face education.4 The challenge of determining what proportion of a new course can be delivered online has been highlighted by other researchers.4 In this course, online delivery was supplemented by face-to-face seminars. Initially about two-thirds of the course was delivered online, and although most preregistrants felt they benefited from the online components, about half wanted more face-to-face activities. Following review of the first year of the course, the online component was decreased to about 57%. Following these changes, about two-thirds of preregistrants were happy with the balance.
This evaluation has some limitations. First, we have assessed only course participants and moderators' views. We have not formally assessed the views of the profession or the pharmacy registering authority, although both are represented on the course advisory committee and have therefore had significant input into course development, review, and implementation. The profession's engagement with the course is also reflected in the significant interest shown by pharmacists wanting to moderate online discussions and tutor at seminars. The registering authority's satisfaction with the course is indicated by its annual reapproval of the program as 1 of 2 preregistration courses available to local pharmacy graduates. Second, we have no data on the course's impact on graduates' performance in their competency-based registration examinations, because the registering authority does not make this available. Finally, the questionnaire response rate was relatively low, so a possibility exists that results are unrepresentative of the entire preregistrant cohort.
SUMMARY
A new pharmacy preregistration course incorporating a significant asynchronous online component was developed, implemented, and evaluated. The successful implementation and largely positive reception of the course illustrates the feasibility and suitability of this method of delivery for preregistration education.
ACKNOWLEDGEMENTS
The authors acknowledge the contributions of the advisory group members in the development of the course. The support and input of Professor Roger Nation, Head, Department of Pharmacy Practice, Monash University, is also acknowledged.
Jennifer L. Marriott and Gregory Duncan developed the original model for the course and had an advisory role in course implementation. Rohan A. Elliott was course director from 2003 to 2006 and, with assistance from Angela Calandra, developed and implemented the course. Rohan A. Elliott conducted the course evaluation and analysis.
REFERENCES
- 1. Pharmacy Board of Victoria. Guidelines for good pharmaceutical practice. Parkville: Victoria: Pharmacy Board of Victoria; 2004. http://www.pharmacybd.vic.gov.au/cmsdocs/guidelines2004.pdf. Accessed March 18, 2009.
- 2.Burton S, Anderson C. Using the Internet to develop an international learning community of pharmacists. Pharm World Sci. 2002;24(5):172–174. doi: 10.1023/a:1020541403026. [DOI] [PubMed] [Google Scholar]
- 3.Swan K. Learning effectiveness online: what the research tells us. In: Bourne J, Moore JC, editors. Elements of Quality Online Education, Practice and Direction. Needham, MA: Sloan Center for Online Education; 2003. pp. 13–45. [Google Scholar]
- 4.Chung U. Maintaining quality pharmaceutical education in the digital age. Am J Health-Syst Pharm. 2003;60:943–946. doi: 10.1093/ajhp/60.9.943. [DOI] [PubMed] [Google Scholar]
- 5.Hunter TS, Deziel-Evans L, Marsh WA. Assuring excellence in distance pharmaceutical education. Am J Pharm Educ. 2003;(3):67. article 94. [Google Scholar]
- 6.Littleton K, Whitelock D. Guiding the creation of knowledge and understanding in a virtual learning environment. Cyberpsychology Behav. 2004;7(2):173–181. doi: 10.1089/109493104323024438. [DOI] [PubMed] [Google Scholar]
- 7.Hedberg JG. Ensuring quality e-learning: creating engaging tasks. Educ Media Int. 2003;40:175–186. [Google Scholar]
- 8.Conrad D. Building and maintaining community in cohort-based online learning. J Dist Educ. 2005;20(1):1–20. [Google Scholar]
- 9.Oliver M, Shaw GP. Asynchronous discussion in support of medical education. J Asynchronous Learning Networks. 1993;7(1):56–67. [Google Scholar]
- 10.Wu D, Hiltz SR. Predicting learning from asynchronous online discussions. J Asynchronous Learning Networks. 2004;8(2):139–152. [Google Scholar]
- 11.Mazzolini M, Maddison S. When to jump in: the role of the instructor in online discussion forums. Comput Educ. 2007;49(2):193–213. [Google Scholar]
- 12.Nestel D, Calandra A, Elliott RA. Using volunteer simulated patients in development of pre-registration pharmacists: Learning from the experience. Pharm Educ. 2007;7(1):35–42. [Google Scholar]
- 13.Beaudin B. Keeping online asynchronous discussions on topic. J Asynchronous Learning Networks. 1999;3(2):41–53. [Google Scholar]