Abstract
Background
The Americas Hepato-Pancreato-Biliary Association (AHPBA) and the Australian and New Zealand Hepatic, Pancreatic and Biliary Association (ANZHPBA) are developing an online distance learning curriculum to facilitate an interactive didactic experience for hepatopancreatobiliary (HPB) fellows in the operationalization of existing HPB fellow curricula. Two needs assessment surveys were carried out to identify the optimal structure and process for deployment in fellow education.
Methods
A 22-question survey querying fellows' learning styles and current and anticipated use of learning tools was disseminated electronically to 38 North American and Australasian HPB fellows. A follow-up 20-question survey was administered to assess fellows' feelings regarding online content.
Results
Response rates were 55% (n = 21) for the first survey and 42% for the second (n = 16). In the first survey, 67% of respondents claimed familiarity with the required HPB curriculum, and 43% indicated dissatisfaction with current personal study strategies. A total of 62% (n = 13) reported studying with focused clinical relevance versus using a prescribed curriculum (n = 1, 5%). Fellows anticipated participating using online tools once (n = 10, 48%) or two or three times (n = 5, 24%) per week. Most respondents (n = 18, 86%) would meaningfully follow one or two discussions per month. The second survey identified themes for improvement such as discussion topics of interest, avoidance of holiday timing and mandatory participation.
Conclusions
An international online distance learning format is an appealing mechanism for improved dissemination and operationalization of the established HPB fellow curricula. Fellows will engage in interactive discussions monthly. Controversial topics or those requiring complex decision making are best suited to this learning format.
Introduction
Hepatopancreatobiliary (HPB) surgery fellowships are offered internationally with curricula developed by international and regional HPB organizations, such as the Americas Hepato-Pancreato-Biliary Association (AHPBA), the International Hepato-Pancreato-Biliary Association (IHPBA), and the Australian and New Zealand Hepatic, Pancreatic and Biliary Association (ANZHPBA).1–3 Despite the presence of common elements within all of these curricula, the varied nature of HPB practices among fellowship training institutions and national health care systems leads to gaps in training across these curricula. Based upon accreditation reviews in the Americas by the Fellowship Council, this variability extends to didactic learning and can result in unwanted inconsistency in the knowledge base of graduating fellows. The Education and Training Committee of the IHPBA has identified significant variation in accreditation processes, examination practices and curricular implementation across HPB training programmes. The AHPBA and ANZHPBA are developing an online interface to facilitate an international, interactive didactic experience to augment HPB fellows' learning, based on the increased use of online educational resources by current trainees. The goal of this initiative is to enable standardization and to broaden implementation of the excellent national and international HPB curricula. This distance learning format provides a forum through which HPB fellows can engage with the HPB fellowship curriculum and establish a community with one another.
Successful distance learning is best achieved when it is guided by the following three principles. Firstly, the course must be accessible and easy to navigate.4 Secondly, students must be able to effectively interact with one another and with instructors or moderators.4,5 Thirdly, learning must be fostered in an active and/or collaborative manner.5 Recently, the IHPBA developed a bespoke virtual learning environment called myHPB which can be accessed via the IHPBA website (http://www.ihpba.org/myhpb), and through which fellows and association members alike can participate in case-of-the-month discussions (CoMDs), as well as virtual journal club (VJC) discussion boards.6 In addition, the AHPBA Education and Training Committee has been developing comprehensive interactive educational modules hosted on the myHPB site, constructed by content experts to cover all aspects of the curriculum. These modules cite primary literature and highlight key learning objectives for each topic within the HPB fellows' curriculum. Moderated interactive discussions are also designed within each topic to support further exploration of the content and its application to patients. In addition to the strict incorporation of the three principles outlined above, obtaining fellow feedback is a critical component of the needs assessment process and guides the construction of an effective online learning environment. The present study represents the findings of a needs assessment survey of North American and Australasian fellows regarding their perceived needs for curricular content, preferences for accessing information, frequency of anticipated interaction, and desires for future content and structure.
Materials and methods
Study participants
Current 2013–2014 academic year HPB fellows in accredited North American HPB fellowships (Fellowship Council and AHPBA) and Australasian HPB fellowships (ANZHPBA) were asked to participate in this study. A total of 38 North American and Australasian fellows were eligible to participate, and were contacted with the survey links via direct e-mail outreach from the authors (SGW and RMM). For both surveys, three additional reminder e-mails were sent. All responses were anonymous.
Online learning format
Selected articles on the topics of biliary tract injury and improving longterm outcomes in hepatocellular carcinoma (HCC) were selected as the first two VJCs. The first CoMD concerned acute pancreatitis. Topics incorporated citations and discussions of primary literature and were chosen by members of the AHPBA Education and Training Committee. The learning objectives for each topic were aligned with the current curriculum to guide fellows through complex clinical scenarios with multiple decision-making branch points. Expert moderators were selected for each of these learning sessions, and included one North American moderator, and one moderator from either Australia and New Zealand or from the UK, where the myHPB platform was developed and extensive experience with distance learning exists. The discussions ran for 4–5 weeks and involved the weekly posting of questions to which fellows were asked to respond; each question was assigned to one to three fellows, but open responses from any of the fellows were also encouraged. Responses were expected to be evidence-based and to include references in support of answers where possible. These discussions ran from September to December 2013.
Survey
To define the current fellows' preferred learning styles, use of study tools, and frequency of anticipated interaction with an online learning interface, a 22-question multiple-choice survey was developed. The survey queried fellow demographics, use of social media and self-reported learning styles and included additional questions intended to identify the tools currently used by fellows to access clinical data and information, and the types of tool they might favour in the future. The majority of questions used a multiple-choice format but provided the opportunity for free-text answers if the trainee could not identify his or her answer amongst the answer choices. The survey was validated by a five-resident cohort for clarity and response time prior to its electronic dissemination to HPB fellows in September 2013. The final survey is provided in Appendix S1 (online).
A second 20-question survey was disseminated electronically during January and February 2014 to the same cohort of fellows following a 3-month initial implementation of the CoMD and VJC. This survey was similarly validated by a five-person cohort of surgical residents and adjusted for clarity and response time. More open-text options were utilized in this survey to solicit guidance as to what might improve the current online learning interface. Additionally, feedback regarding the initial versions of HPB fellow curricular modules on myHPB was evaluated. The second survey can be viewed in Appendix S2 (online).
myHPB web analytics
Online usage of the myHPB CoMD and VJC was recorded. Fellows who chose to subscribe to a discussion and to receive e-mail alerts after each new post were recorded as a measure of the number of fellows participating in the discussions. The number of times a fellow posted a comment in one of the discussions (CoMD or VJC) was also captured. Google analytics data were also used to provide unique page views of each CoMD and VJC, defined as one or multiple page views within the same session by the same user. These data were collected for all users logged into http://www.ihpba.org. Open-text responses on barriers to implementation and desires for future content and functionality were reviewed for emerging themes (by SGW and RMM).
Results
The response rate on the first survey was 55% (21 of 38). The median respondent age was 35 years (range: 30–42 years) and 80% of respondents were male. Most respondents (n = 15, 71%) had completed surgical residency within the previous 3 years and four (19%) trainees had finished ≥5 years previously. The citizenship of surveyed fellows was quite diverse: the sample included trainees from Europe, Australia, New Zealand, and North and South America. Fellows reported studying primarily with focused intent for upcoming operative or clinical scenarios (n = 13, 62%) rather than adhering to a prescribed curriculum (n = 1, 5%). Modes of accessing educational material included the use of computers (n = 18, 86%), textbooks (n = 14, 67%) and smartphones or tablets (n = 9, 43%). Figure 1 delineates anticipated and current tools used by fellows for studying and operating room preparation. Most fellows (n = 14, 67%) claimed familiarity with their required HPB curriculum, but 43% (n = 9) indicated dissatisfaction with current personal study strategies. Fellows indicated they would participate in an interactive discussion board once (n = 10, 48%) or two or three times (n = 5, 24%) per week. Most respondents (n = 18, 86%) would meaningfully follow one or two discussions per month. Table 1 shows that fellows would be open to various educational forums.
Figure 1.
Fellow learning tool preferences. (a) Desired online learning modalities derived from answers to the item: ‘What online resource would you be likely to use in an HPB virtual learning environment? Select all that apply.’ (b) Tools for operating room preparation. (c) Tools for accessing information derived from answers to the item: ‘How do you access information while studying? Select all that apply’
Table 1.
Anticipated use of online hepatopancreatobiliary (HPB) fellowship learning platforms
| Which type of online resource would you be likely to utilize within an HPB fellowship online learning platform? (Select all that apply) | ||
|---|---|---|
| Answer options | Responses, % | Responses, n |
| A self-study module which provides objectives and an overview of content related to a specific topic | 81.0% | 17 |
| Operative video and imaging bank | 81.0% | 17 |
| Online lecture or webinar by an HPB expert | 66.7% | 14 |
| A journal club moderated by international HPB experts | 61.9% | 13 |
| A moderated case-of-the-month discussion | 57.1% | 12 |
| Self-assessment multiple-choice questions | 52.4% | 11 |
| Other (please specify) | 4.8% | 1 |
| Answered question | 21 | |
| Skipped question | 0 | |
Of the 16 (42%) fellows who completed the second survey, 10 reported viewing online modules as requested prior to the survey. Figure 2 indicates that fellows anticipated participating more often than they reported actually logging in. The myHPB database reported that 68% of fellows subscribed to at least one of the discussions and received an e-mail update every time a post was created within either the CoMD or VJC. Figure 3a demonstrates the unique page views by date for each discussion, and Table 2 demonstrates the total number of unique page views and posts to each discussion.
Figure 2.
Anticipated and reported site usage. CoMD, case-of-the-month discussion; VJC, virtual journal club
Figure 3.
Current and future virtual learning environment usage. (a) Actual logins by date and virtual learning environment. (b) Preferences for future module video offerings
Table 2.
Actual myHPB virtual journal club (VJC) and case-of-the-month discussion (CoMD) usage
| Topic | Unique page views, n | Average time on page, min:s | Posts, n |
|---|---|---|---|
| CoMD: acute pancreatitis | 178 | 5:07 | 32 |
| VJC: biliary injury | 341 | 4:17 | 34 |
| VJC: hepatocellular carcinoma | 72 | 0:53 | 15 |
Open-text responses to the second survey were reviewed for emerging themes. Respondents noted the diversity of opinion, landmark article references, evidence-based practice guidance, moderator-led expert discussion, and ability to interact with international colleagues as strengths of the VJC and CoMD. Reported weaknesses of the VJC included disjointed discussion, disparate schedules, the tedium of typing long responses, and falling behind the pace of discussion. Fellows cited clinical and administrative duties as preventing their full participation, and suggested that three rather than five articles would be more ideal for the 4–5-week discussion timeframe. Suggested VJC improvements included mandatory participation, quickened pace, and the avoidance of initiating VJC discussions over the December holiday season. Similar themes emerged in the CoMD evaluation.
Finally, respondents were asked to review newly posted online modules at myHPB. Ten (63%) of the 16 respondents confirmed that they had reviewed the newly created curricular modules. All 10 module viewers agreed the content was easy to view and user-friendly. Fellows were queried on their desires for video content within the modules (Fig. 3b). Most indicated they would definitely watch edited cases, lectures by experts, and techniques and surgical ‘pearls’. Fellows indicated a strong preference for the availability of a video library, links to published guidelines and expert opinion, and succinct summaries of evidence-based guidelines for managing common HPB diseases and conditions.
Discussion
As indicated by the breadth and depth of topics covered within the AHPBA, IHPBA and ANZHPBA HPB fellowship curricula, the mastery of a large fund of knowledge is required in order to become a competent HPB surgeon. Within the confines of a 1–2-year period of study, HPB fellows must develop the expansive, evidence-based knowledge required to enable them to practise multidisciplinary management of benign and malignant HPB disease and develop operative autonomy in the performance of complex HPB surgical procedures. Despite the comprehensive HPB fellowship curricula, only 67% of fellow respondents claimed familiarity with their required HPB curriculum, and 43% indicated dissatisfaction with their current personal study strategies. The development of effective distance learning platforms such as multimedia modules augmented by interactive discussion boards creates a unique opportunity for international collaboration and the standardization of the fellow knowledge base. The AHPBA Education and Training Committee has begun the development of topical modules aligned with the AHPBA HPB fellowship curriculum.1 These modules have specific learning objectives, are centred around patient presentations or specific procedures, utilize multimedia formats to include video and images, contain self-assessment questions, and reference primary landmark journal articles.
In addition to the modules, the myHPB learning platform creates discussions focused on a particular topic with international fellow participation, guided by expert moderators.6 Moderators are selected based upon their expertise and with geographical diversity in mind in order to allow fellows to benefit from the various perspectives that exist at different centres. The goal of this initiative was to create a resource that will allow trainees to feel less overwhelmed by the large volume of available information and more empowered by the development of a strong knowledge base, and to yield a more consistent product at the completion of training. The successful creation of online learning platforms such as those described herein relies on the participation of the targeted learners. The assumption that one can ‘build it and they will come’ is ineffective if the tool does not meet learner needs. Thus, the needs assessment data presented within this study are critical to guide the further development of this HPB fellowship resource. The present study demonstrates that 68% of responding fellows engaged with this learning format and actively participated in the discussions, as evidenced by page views and posts within the discussions. Furthermore, in two of the three discussions, the average time spent on each page exceeded 4 min, which is nearly four times as long as that spent on the ‘average web page’ as calculated by Nielsen.com and suggests that fellows were reading the full discussion rather than skimming the material.7
The evidence for distance learning in surgical education is rapidly growing. Over 90% of US surgical residency programmes have turned to the Surgical Council on Resident Education (SCORE) curriculum developed by the American Board of Surgery to augment resident didactic experiences.8 In the UK, the University of Edinburgh and the Royal College of Surgeons of Edinburgh have developed an online MSc in Surgical Sciences or Edinburgh Surgical Sciences Qualification (ESSQ) and four Masters of Surgery (ChM) programmes. These programmes have jointly recruited over 600 surgical trainees from 48 countries in a 7-year period. These programmes include virtual clinical scenarios, multiple-choice questions, and required discussion board participation in discussions that are maintained by experts and driven by evidence-based medicine.9 Ultimately, students submit a research dissertation and are eligible to obtain a postgraduate degree.9 The UK model attributes some of its success to course design, which adheres to the well-established principles of successful online learning in that the virtual learning environment is specifically tailored to the needs of the institution and the students, rather than representing an ‘off-the-shelf’ learning environment.10 Moreover, the students have clearly defined goals and objectives and are provided with a sense of community and connectivity that can amplify the learning experience.10,11
In the early myHPB AHPBA-sponsored discussions for North American and Australasian HPB fellows, participation was highly variable and depended on the topic and timing of the discussion. It is unclear why the turnout for the second VJC on HCC was not as robust as the first; however, feedback from fellows in the second survey shed some light on possible reasons. The second VJC was initiated during the December holiday season, when many surgeons find themselves burdened with increased operative volumes and seasonal demands that are not present during other seasons. Moreover, the second VJC focused on longterm outcomes for HCC and addressed important concepts that fellows must master, although this topic may not be encountered frequently. By contrast, the bile duct injury journal club focused on a topic with which fellows clearly engaged as a result of the technical considerations discussed and the topic's clear relevance to their practice, both current and future. This journal club was posted in conjunction with a video demonstrating an actual bile duct injury, which was highly valuable to viewers who had not actually seen a bile duct injury previously.12 The narration outlining common errors in visual perception which can lead to this injury was viewed as an excellent addition to this VJC. The web analytics shown in Fig. 3 also demonstrate the perceived value of this VJC as users viewed this discussion and video multiple times following the closure of this discussion.
The analytic data on utilization and feedback from fellows suggest that future VJCs should focus on topics of immediate clinical relevance. Ideally online content will contain supplemental video and/or images, and perhaps a maximum of three or four articles. Fellows found the CoMD to be excellent and the discussion generated spontaneous posts and back-and-forth discussion such as is desired in the use of this type of distance learning format. The expert moderators pushed the fellows to supply evidence to support their posts and recommendations rather than relying on anecdotal experience, which is a key feature of these discussions. In the absence of this effective moderation, the CoMD is at risk for devolving into a series of anecdotes rather than building fellows' knowledge bases in an evidence-based manner.
At present, the chair of the AHPBA Education and Training Committee announces the beginning of the monthly discussion and assigns fellows to respond to specific questions posted by the moderator on a weekly basis. Although this system is effective, it is hoped that spontaneous engagement with the discussions will increase in the future. Interestingly, in the feedback received from fellows, a requirement for some level of participation was recommended to increase participation. This feedback was provided to the North American HPB Fellowship Directors, who agreed to support a protocol of mandatory participation in the discussions and curriculum on myHPB according to a minimum number of posts annually in order to achieve AHPBA HPB certification at the conclusion of training. This is consistent with the practices of the effective ESSQ programme developed in Edinburgh.9
The information collected during this study will help to guide future AHPBA online education endeavours. The needs assessment data and feedback regarding the first iteration of these modules and discussion board designs demonstrate a desire and need for future modifications. Trainees desire a significant component of video material and images to augment teaching modules and to contribute to preparation for procedures. Fellows also indicate a desire for reliable sources of information on ‘getting out of trouble’ in the operating room and the management of perioperative complications because these areas are sparsely addressed in the literature. These topics will be woven into the myHPB online curriculum in the future. As indicated by survey responses, fellows prefer learning about a single topic at a time. It would make sense to provide incoming fellows with a syllabus and to require their monthly mastery of various modules and to sponsor monthly journal clubs or CoMDs centred on certain module topics or controversies. This would also facilitate fellow accountability should full module completion be required for AHPBA certification in the future. In the context of constructing a virtual learning environment, other authors have established that effective e-learning for health professionals depends on the provision of a user-friendly system that facilitates regular student support, both locally and online, for both technical concerns and for the mastery of concepts.13 These principles will be kept in mind as more modules are developed and implemented.
Conclusions
In conclusion, these data provide a needs assessment for an international interactive online HPB fellowship curriculum. As the community of HPB surgeons grows, a standardized knowledge base becomes increasingly important, both to further the field and to provide evidence-based patient care. Hepatopancreatobiliary fellowships have attempted to achieve this using a standardized curriculum, but the implementation of this varies widely across programmes. Online forums provide a common experience for fellows and a venue for current and future collaboration. It is hoped that by offering a common educational experience and inculcating a sense of community in current trainees, fellows will be enabled to deliver a better quality of care to their current and future patients.
Conflicts of interest
None declared.
Supporting Information
Additional supporting information may be found in the online version of this article at the publisher's website:
Hepatopancreatobiliary fellows online learning needs assessment survey.
Follow-up needs assessment survey.
References
- 1.Americas Hepato-Pancreato-Biliary Association. 2008. AHPBA fellows' curriculum. Available at http://www.ahpba.org/assets/fellowship/AHPBA_HPB-Curriculum.pdf (last accessed 11 June 2014)
- 2.Australian New Zealand Hepatic, Pancreatic and Biliary Association. 2009. ANZHPBA Fellows curriculum. Available at http://www.anzhpba.com/pdf/anzhpba-fellowship-hpb-surgery.pdf (last accessed 11 June 2014)
- 3.International Hepato-Pancreato-Biliary Association. 2008. IHPBA training standards. Available at http://www.ahpba.org/assets/fellowship/IHPBA_hpb_training_standards.pdf (last accessed 11 June 2014)
- 4.Muse HE. The Web-based community college student: an examination of factors that lead to success and risk. Internet High Educ. 2003;6:241–261. [Google Scholar]
- 5.Sowan AK, Jenkins LS. Designing, delivering and evaluating a distance learning nursing course responsive to students' needs. Int J Med Inform. 2013;82:553–564. doi: 10.1016/j.ijmedinf.2013.02.004. [DOI] [PubMed] [Google Scholar]
- 6.International Hepato-Pancreato-Biliary Association. myHPB. Available at http://www.ihpba.org/myhpb (last accessed 11 June 2014)
- 7.Nielsen.com. January 2013: Top U.S. Entertainment Sites and Web Brands. Available at http://www.nielsen.com/us/en/newswire/2013/january-2013--top-u-s--entertainment-sites-and-web-brands.html (last accessed 11 June 2014)
- 8. SCORE Curriculum outline for General Surgery Residency. Available at http://www.absurgery.org/default.jsp?scre_booklet (last accessed 11 June 2014)
- 9.Smith PJ, Wigmore SJ, Paisley A, Lamb P, Richards JM, Robson AJ, et al. Distance learning improves attainment of professional milestones in the early years of surgical training. Ann Surg. 2013;258:838–843. doi: 10.1097/SLA.0000000000000211. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Salmon G. Flying not flapping: a strategic framework for e-learning and pedagogical innovation in higher education institutions. ALT-J Res Learn Technol. 2005;13:201–218. [Google Scholar]
- 11.Siemens G. Connectivism: learning theory for the digital age. Int J Instr Technol Distance Learn. 2005;2:3–10. [Google Scholar]
- 12.Connor S. 2011. Preventing bile duct injuries. Available at http://youtu.be/UGhkI6ltAfw (last accessed 11 June 2014)
- 13.Childs S, Blenkinsopp E, Hall A, Walton G. Effective e-learning for health professionals and students – barriers and their solutions. A systematic review of the literature – findings from the HeXL project. Health Info Libr J. 2005;22:20–32. doi: 10.1111/j.1470-3327.2005.00614.x. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Hepatopancreatobiliary fellows online learning needs assessment survey.
Follow-up needs assessment survey.