Abstract
Background
Injuries to the shoulder joint commonly require the attention of an orthopedic surgeon. Shoulder arthroscopy plays an increasingly important role in the diagnosis and repair of shoulder pathology; however, the most effective manner in which to teach orthopedic residents fundamental knowledge of diagnostic shoulder arthroscopy before entering the operating room is unclear. We aimed to compare the existing cadaver-based teaching of diagnostic shoulder arthroscopy knowledge with a method that combines model-and video-based teaching to orthopedic surgery residents in a randomized pilot trial.
Methods
A composite (model/video teaching) method was designed, using prepared teaching videos and the commercially available ALEX shoulder arthroscopy model. First-and second-year orthopedic surgery residents from the University of Calgary were consented, surveyed for their arthroscopy experience and randomized to either cadaver or composite teaching. Subjects wrote a pretest before their teaching session and a posttest afterwards to assess their knowledge of diagnostic arthroscopy. The tests were multiple choice, containing text and pictorial-based questions. The posttest was modified to minimize recall bias. Subjects were also surveyed for their comments regarding the teaching sessions.
Results
Nine of 10 subjects increased their test scores after the teaching sessions, with 4 of 5 in the cadaver-based and 5 of 5 in the composite groups. There were no differences between the teaching groups on their mean pre-or posttest scores. The composite group, but not the cadaver-based group, had a statistically significant increase in posttest scores. When the text-and pictorial-based question sections were analyzed separately, both groups significantly improved their mean text-based score, whereas only the composite group increased their mean pictorial-based questions score. Surveying the residents elicited positive comments regarding both manners of teaching.
Conclusion
This pilot trial suggests that a composite teaching curriculum is at least as effective as a cadaver-based environment for teaching orthopedic surgery residents fundamental knowledge of diagnostic shoulder arthroscopy.
Abstract
Contexte
Les blessures à l'articulation de l'épaule exigent souvent l'intervention d'un chirurgien orthopédiste. L'arthroscopie de l'épaule joue un rôle de plus en plus important dans le diagnostic et la correction des pathologies de l'épaule, mais ne sait pas clairement quelle est la façon la plus efficace d'enseigner aux résidents en orthopédie les bases de l'arthroscopie diagnostique de l'épaule avant d'entrer en salle d'opération. Nous voulions comparer au moyen d'une étude pilote randomisée la façon actuelle d'enseigner à des résidents en chirurgie orthopédique l'arthroscopie diagnostique de l'épaule sur cadavres à une méthode qui combine un enseignement sur modèle et une formation vidéo.
Méthodes
On a conçu une méthode composée (enseignement sur modèle et formation vidéo) fondée sur des vidéos didactiques préparées et sur le modèle commercial d'arthroscopie de l'épaule ALEX. On a obtenu le consentement de résidents de première et de deuxième année en chirurgie orthopédique de l'Université de Calgary, on les a interrogés pour déterminer leur expérience de l'arthroscopie et on les a affectés par randomisation à la méthode de formation sur cadavre ou à la méthode composée. Les sujets se sont soumis à un pré-test avant leur séance de formation et à un post-test par la suite pour évaluer leur connaissance de l'arthroscopie diagnostique. Les tests à choix multiples contenaient des questions à base de texte et d'éléments graphiques. On a modifié le post-test pour réduire le plus possible le biais attribuable à la mémoire. On a aussi demandé aux sujets de commenter les séances de formation.
Résultats
Neuf des 10 sujets ont amélioré leurs résultats de tests après les séances de formation, soit 4 des 5 qui ont reçu une formation sur cadavre et 5 des 5 qui ont reçu la formation composée. Les résultats moyens au pré-test ou au post-test ne différaient pas entre les groupes. Chez les sujets qui ont reçu la formation composée, mais non chez ceux qui ont reçu la formation sur cadavres, les résultats au post-test ont augmenté de façon statistiquement significative. Lorsqu'on a analysé séparément les questions à base de texte et d'éléments graphiques, on a constaté que les deux groupes ont amélioré de façon significative leurs résultats moyens aux questionnaires à base de texte, tandis que seuls les sujets qui ont reçu la formation composée ont amélioré leurs résultats moyens aux questions à base d'éléments graphiques. Le sondage auprès des résidents a suscité des commentaires positifs au sujet des deux méthodes de formation.
Conclusion
Cette étude pilote indique qu'un programme de formation composée est au moins aussi efficace qu'une formation sur cadavres pour inculquer aux résidents en chirurgie orthopédique les bases de l'arthroscopie diagnostique de l'épaule.
Injuries to the shoulder joint rank among the most frequent reasons for a patient to visit an orthopedic surgeon.1 With respect to intraarticular shoulder injuries arthroscopy is considered the gold-standard diagnostic tool2 and, more recently, a mainstay of orthopedic surgical treatment. A thorough knowledge of shoulder anatomy, arthroscopic equipment and anatomic variations all underlie a successful procedure. Indeed, in the hands of a practised surgeon, arthroscopic repair has been suggested to have advantages over open repair for some shoulder pathologies, including glenohumeral instability3 and rotator cuff disorders.4 Proposed advantages of arthroscopic surgical repair over open repair include decreased use of narcotic analgesics postoperatively less disruption of normal anatomy3 and return to a higher level of function for the patient.5
Although evidence suggesting arthroscopy is effective for the diagnosis and treatment of multiple orthopedic disorders is mounting, the most effective manner in which to teach knowledge of diagnostic shoulder arthroscopy to orthopedic residents, before the acquisition of arthroscopic skills, is unclear. Methods that have been tested with varying success in orthopedic teaching include interactive computer simulation,6 visual computer programs7 and verbal and video feedback on procedures performed.8 A recent study showed that a shoulder arthroscopy model can be used to measure arthroscopic knowledge and skill specifically the speed and accuracy in performing specific arthroscopic manoeuvres and the ability to identify anatomic landmarks.9 This work indicates that the use of a surgical simulator can be an effective teaching tool for shoulder arthroscopy. Taken together, this emphasizes that optimizing the most effective training method for specific surgical procedures is of great interest and requires precise determination.
The University of Calgary Orthopaedic Surgery training program currently teaches fundamental knowledge of diagnostic arthroscopy in the laboratory on cadaver specimens and in the operating room on live patients. While cadaver-based learning allows for a flexible environment where time is not an issue, it lacks the real life simulation of performing arthroscopy. Moreover, the use of cadavers is not standardized, requires dedicated space and is associated with supply, cost and legal issues. Although the operating room is a necessary learning environment, the pressures of cost and time10,11 as well as ethical issues12 make it a less than ideal environment, particularly for teaching basic knowledge of surgical equipment and anatomy versus teaching arthroscopic skills. Thus, it is important that a surgical resident be as familiar as possible with the equipment involved and the anatomic structures to be visualized before entering the operating room. To date, the effectiveness of employing standardized, videotaped shoulder arthroscopy in conjunction with tabletop model demonstration in a small group learning environment for the purpose of teaching knowledge has yet to be compared with the current cadaver-based teaching modality. Thus, the goal of this pilot trial was to compare the effectiveness of 2 teaching methods, as measured by the acquisition of fundamental diagnostic shoulder arthroscopy knowledge by orthopedic surgery residents randomized to either a cadaver-based teaching environment, or to a newly designed composite (model and video-based teaching) environment.
Methods
Subject enrollment and study design
This study received institutional approval from the Conjoint Health Research Ethics Board and was approved by the Orthopaedic Resident Training Committee at the University of Calgary.
We evaluated teaching methods in the format of a randomized pilot trial. Eligible subjects were orthopedic surgery residents in the first or second year of training. We excluded subjects lacking informed consent and surgical residents beyond their third year of training, when specific shoulder arthroscopy exposure training is available.
After informed consent was obtained, 10 subjects were enrolled: 5 from each of the first and second years of the Orthopaedic Surgery Residency Program. The subjects also completed a survey to describe their previous experience with shoulder arthroscopy. Information from this survey was used to stratify the subjects based on whether they had previously read about, witnessed or assisted with shoulder arthroscopy surgery. An identical survey was distributed to residents in their third year and above who had undergone a specific shoulder arthroscopy rotation and confirmed that the survey could differentiate between levels of experience. After stratification, a computerized block randomization technique was used to randomize subjects to 1 of 2 groups: cadaver-based teaching or model and video-based teaching (Stata version 8.2).
Before the teaching sessions, each resident wrote the pretest and afterwards wrote the posttest. This 34-question multiple choice test developed for the study was designed to test residents' knowledge of arthroscopy equipment, shoulder anatomy and shoulder pathology rather than their skill at performing arthroscopy. The test was scheduled for 1 hour. Multiple choice questions developed for these tests followed guidelines issued by the Royal College of Physicians and Surgeons of Canada for the purposes of designing certifying examinations,13 and questions were selected based on their relevance to the learning objectives (see below). The pre-and posttests were divided into 2 sections, the first comprised 26 text-based questions, and the second comprised 8 pictorial-based questions. For example, text-based questions tested knowledge of shoulder anatomy, arthroscopic equipment and the order of arthroscopic procedures. The pictorial-based questions required the subjects to identify anatomic structures from intraarticular photographs of joint anatomy and arthroscopic equipment from photographs. The posttest differed from the pretest in that the questions were randomly reordered and the orientation of selected figures was manipulated to avoid direct visual recall bias from the pretest. We did not formally validate the pre-and posttests. The tests were deemed to have face validity, as determined by the investigators and from verbal feedback from the residents involved in the study.
The teaching sessions were conducted by fellowship-trained orthopedic shoulder surgeons from the University of Calgary Sport Medicine Centre (1 preceptor per group). Two hours were allocated to independently teach their respective sessions. The learning objectives were identical for both the cadaver-based and the composite teaching groups, as follows: 1) To become familiarized with shoulder arthroscopy equipment; 2) learn the sequence and steps of an arthroscopic procedure of the shoulder; 3) identify bony landmarks of the shoulder and standard arthroscopy portals; 4) understand the orientation of the arthroscope, sheath and camera; and 5) identify normal and pathological shoulder anatomy.
For the composite environment, 2 videos were created by filming shoulder arthroscopic procedures on live patients, in both the lateral decubitus and beach chair positions. The videos were edited and archived. The teaching model used was a commercially available arthroscopy model (ALEX shoulder arthroscopy model,14,15 Sawbones, Linvatec Corporation; Fig. 1). After watching and discussing the videos, 2 residents per model worked together. Similarly, 2 residents per shoulder worked together in the cadaver-based teaching environment.
Arthroscopy equipment
The arthroscope used in the teaching sessions was a standard 30° arthroscopy set-up used for shoulder surgery by the Division of Orthopaedic Surgery at the University of Calgary. The set-up included a 2.9-mm QuickLatch arthroscope with a high-flow sheath (outside diameter 5.2 mm, length 120 mm) (Linvatec Corporation), and intraarticular images were displayed on a monitor.
Statistical analysis
We performed statistical analyses on the pre-and posttest mean scores. We used the 2-tailed, unpaired Student's t test to compare mean scores between the teaching groups, whereas the 2-tailed, paired Student's t test was used to compare within the groups. Statistical significance was considered to be reached if p < 0.05. In addition, we calculated 95% confidence intervals (CIs) for the difference between means.
Results
We recruited 10 first-and second-year orthopedic residents into this study, 5 residents from each year of training. The stratification and randomization process resulted in comparable groups with respect to experience with shoulder arthroscopy and level of training (Table 1).
Overall, 9 of 10 subjects improved their scores on the posttest compared with the pretest. In the cadaver-based teaching group, 4 of 5 subjects improved their test score on the posttest while 1 of 5 decreased by 1 point in the posttest (Fig. 2 left). In the composite teaching group, all subjects improved their test score on the posttest (Fig. 2, right). These results illustrate that the students learned from both teaching methods.
The mean pre-and posttest scores from both groups are illustrated in Figure 3 and are listed in Table 2. Neither the pretest mean score nor the posttest mean score between the groups were statistically significantly different.
In the cadaver-based teaching group, the pre-to posttest score difference showed a trend toward improvement but was not statistically significant. However, the composite group did show a statistically significant improvement in their score from the pre-to the posttest (Fig. 3). Differences were calculated between the pre-and posttest scores within each group. The average within-group difference is a reflection of the improvement seen based on the teaching method. The cadaver group showed an improvement of 13.5%, compared with 19.4% in the composite group.
Only subject 6 in the cadaver-based teaching group showed a posttest mark lower than their pretest mark. If this subject's score is omitted from the analysis, the cadaver-based group's increase in mean pre-versus posttest scores still does not reach statistical significance (pretest 20.0 [standard deviation {SD} 2.35], posttest 26.00 [SD 1.22]; p = 0.0777, 95% CI –13.23 – 1.23).
Both the pre-and posttests were divided into 2 sections: 26 text-based questions and 8 pictorial-based questions. To determine subjects' specific areas of improvement, we evaluated the mean scores from each section of the test within and between the 2 groups.
Both the cadaver-based group (16.2 [SD 1.39] to 20.4 [SD 0.93]; p = 0.0277; 95% CI –7.65 to –0.75) and the composite group (15.8 [SD 0.58] to 19.8 [SD 1.11]; p = 0.0189, 95% CI –6.91 to –1.09) showed a statistically significant increase in mean score on the 26 questions of the text-based section of the test (Fig. 4). However only the composite group (2.4 [SD 0.51] to 5.0 [SD 0.45]; p = 0.0123, 95% CI –4.27 to –0.93), and not the cadaver-based group (4.6 [SD 0.75] to 5.0 [SD 0.55]; p = 0.7174, 95% CI –3.26 to –2.46), increased their score on the 8 pictorial-based questions in a statistically significant manner (Fig. 4).
Discussion
In this pilot study, we compared 2 methods by which to teach the basics of diagnostic shoulder arthroscopy to novice orthopedic surgery residents in cadaver-based versus composite learning environments. We found the group that received teaching with models and videos showed a statistically significant improvement in their mean pre-versus posttest scores.
The subject population in this study comprised all the first-and second-year orthopedic surgery residents at the University of Calgary. Subjects were stratified based on previous experience with shoulder arthroscopy and randomized to the teaching environments. The goal of each group was to learn fundamental knowledge of diagnostic shoulder arthroscopy before experiencing such a procedure in the operating room later in the training program. The stratified randomization process resulted in comparable groups at baseline, as measured by statistically similar mean pretest scores.
The small sample size of this pilot study was not intended for specific hypothesis testing that would definitively determine a difference between teaching methods. However, the statistically significant improvement observed in the group receiving composite teaching can be explained in the following ways: 1) there were differences between the 2 groups of subjects at baseline; 2) of the 2 teaching methods, the composite session was more effective; or 3) the test was biased toward the composite teaching method.
First among these possibilities, although stratification and randomization yielded comparable groups, there were differences between the 2 groups. For example, although not statistically different, the composite group started from a lower mean score (18.2 [standard error of mean (SEM) 0.66]) than the cadaver group (20.8 [SEM 1.98]). The lower mean score left more room for improvement in the composite group and might have favourably biased that group's ability to show a difference in test scores.
Second, it is possible that the composite teaching environment was a more favourable style than the cadaver environment. The 2 groups were taught by fellowship-trained staff orthopedic surgeons working from identical teaching objectives; however, different teaching styles might have influenced the results. Also, the opportunity to learn in a novel environment might have influenced the composite group. This possibility was also realized as a potential factor in teaching shoulder anatomy knowledge with a virtual reality simulator versus a textbook.16 Regardless a generalized role for models in surgical education has been established in the literature. A low-fidelity surgical tubing model was found equally effective to a high-fidelity rat vas deferens model, as evidenced in a study of microsurgical anastomosis by surgical trainees.17 Further, a study of saphenofemoral dissection surgery showed that technical ability on a model significantly correlated to performance in the operating room.18 Given that arthroscopy experts feel a substantial number of procedures must be performed before this skill is mastered19 it is critical that fundamental knowledge of arthroscopy first be taught effectively. For example, this includes knowledge of the surgical equipment, how to orient the arthroscope and an appreciation for surgical anatomy. Practice with bench models may better prepare surgical trainees to learn skills using cadaveric specimens20 and help build their confidence before entering the operating room.12 Taken together with our results, this implies that the use of videos and models will allow for improved knowledge acquisition in the early stages of learning diagnostic shoulder arthroscopy, thus providing a solid base for skills development.
The last explanation for the test score difference is the possibility that the test questions were biased. A consensus of the authors determined that 6 questions were likely biased toward the composite group and 2 toward the cadaver group; 24 were considered to be neutral. We performed a sensitivity analysis by removing all of the biased questions. The results of this analysis showed that 3 of 5 residents in the cadaver-based group improved their scores, and 2 residents showed no change. In the composite group, all residents improved their scores as a result of the teaching. Therefore, it is unlikely that the test was biased toward either group.
Acknowledging the limitations of sample size and potential biases, these study data support the conclusion that combined teaching is at least equally as effective as the traditional cadaver-based teaching of fundamental shoulder arthroscopy knowledge.
The real-world significance of these observations lies in their implications in training surgical residents. The use of cadavers in surgical teaching is ethically challenging, costly, not standardized, requires specialized facilities and is associated with concerns of supply and legality in some countries. Teaching orthopedic surgery residents in the operating room leads to longer procedure times and higher costs.10,11 Thus, preparing trainees in an optimal environment, before they enter the operating room to assist with or perform a given procedure, is an important step. This study suggests that the use of videos and models to teach fundamental knowledge of shoulder arthroscopy can bypass the more involved procedure of acquiring, storing and disposing of cadaver-based teaching equipment. This may be advantageous, since models and videos are easily archived, standardized and do not require specialized facilities. This is not to discount a role for cadaver-based teaching; however, as a method for teaching basic knowledge in this particular surgical area, a model and video approach can be an appropriate introduction. A cadaver-based approach may be a more effective method for skills acquisition than knowledge. Performance in the operating room needs to be assessed as the final outcome of a surgical training program. Such a curriculum would target junior residents, because more experienced residents and surgeons may require a higher fidelity model than the ALEX. Further, the timing of composite sessions in relation to rotations in the residency curriculum, and whether models and videos facilitate knowledge retention, remains to be determined.
In a training environment, students will ideally be taught in a manner that is enjoyable for them and effective in fulfilling their learning objectives. In an electronic survey completed after our teaching sessions, 4 of 4 respondents from the composite group and 5 of 5 respondents from the cadaver-based teaching group expressed positive comments regarding their respective teaching environments. This favourable response suggests that both the established and the novel teaching approach are accepted by the trainees. These positive comments also support a teaching environment in which trainees could first establish knowledge with models and videos and then proceed to learn more arthroscopic skills on cadaveric shoulders.
Overall, this work supports the notion that videos and models are relevant to surgical education. Further studies will optimize this teaching tool and determine its use for arthroscopic skills acquisition, ensuring surgical residents both an enjoyable and effective training program.
Conclusion
This pilot trial suggests that a model-and video-based teaching curriculum is at least as effective as a cadaver-based environment for teaching orthopedic surgery residents fundamental knowledge of diagnostic shoulder arthroscopy.
Acknowledgments
We thank Dr. Robert Hollinshead for assistance in video creation and Dr. Jacques Bouchard for assistance incorporating this project into the orthopedic surgery resident training program. We also acknowledge Catharine Hawn for help creating the videos and in facilitating the teaching and Shane Gormley and the Linvatec Corporation for donating equipment used in this study. Dr. Ceponis was supported by a Summer Research Scholarship from the Faculty of Medicine at the University of Toronto.
Competing interests: None declared.
Correspondence to: Dr. Nicholas G.H. Mohtadi, University of Calgary Sport Medicine Centre, 2500 University Dr. N.W., Calgary AB T2N 1N4; fax 403 220-9489; mohtadi@ucalgary.ca
References
- 1.Schappert SM. Office visits to orthopedic surgeons: United States, 1995-1996. Adv Data 1998;302:1-32. [PubMed]
- 2.Mohtadi NG, Vellet AD, Clark ML, et al. A prospective, double-blind comparison of magnetic resonance imaging and arthro-scopy in the evaluation of patients presenting with shoulder pain. J Shoulder Elbow Surg 2004;13:258-65. [DOI] [PubMed]
- 3.Angelo RL. Controversies in arthroscopic shoulder surgery: arthroscopic repair versus open Bankart repair, thermal treatment of capsular tissue, acromioplasties —are they necessary? Arthroscopy 2003;19(Suppl 2):224-32. [DOI] [PubMed]
- 4.Millstein ES, Snyder SJ. Arthroscopic management of partial, full-thickness, and complex rotator cuff tears: indication, techniques and complications. Arthroscopy 2003;19(Suppl 1):189-99. [DOI] [PubMed]
- 5.Fabbriciani C, Milano G, Demontis A, et al. Arthroscopic versus open treatment of Bankart lesion of the shoulder: a prospective randomized study. Arthroscopy 2004;20:456-62. [DOI] [PubMed]
- 6.Pedowitz RA, Esch J, Snyder S. Evaluation of a virtual reality simulator for arthroscopy skills development. Arthroscopy 2002;18:E29. [DOI] [PubMed]
- 7.Thomas RL, Allen RM. Use of computer-assisted learning module to achieve ACGME competencies in orthopaedic foot and ankle surgery. Foot Ankle Int 2003;24:938-41. [DOI] [PubMed]
- 8.Backstein D, Agnidis Z, Regehr G, et al. 2004. The effectiveness of video feedback in the acquisition of orthopedic technical skills. Am J Surg 2004;187:427-32. [DOI] [PubMed]
- 9.Srivastava S, Youngblood PL, Rawn C, et al. Initial evaluation of a shoulder arthroscopy simulator: establishing construct validity. J Shoulder Elbow Surg 2004;13:196-205. [DOI] [PubMed]
- 10.Farnworth LR, Lemay DE, Wooldridge T, et al. A comparison of operative times in arthroscopic ACL reconstruction between orthopaedic faculty and residents: the financial impact of orthopaedic surgical training in the operating room. Iowa Orthop J 2001;21:31-5. [PMC free article] [PubMed]
- 11.Lavernia CJ, Sierra RJ, Hernandez RA. The cost of teaching total knee arthroplasty surgery to orthopaedic residents. Clin Orthop Relat Res 2000;380:99-107. [DOI] [PubMed]
- 12.Raja AJ, Levin AV. Challenges of teaching surgery: ethical framework. World J Surg 2003; 27:948-51. [DOI] [PubMed]
- 13.Royal College of Physicians and Surgeons of Canada. Developing multiple-choice questions for the RCPSC certificationexaminations [Web site of the Royal College of Physicians and Surgeons of Canada]. Available: http://rcpsc.medical.org/residency/certification/examinerguide/multiplechoice_e.php#preface (accessed 2005 June 17).
- 14.Snyder SJ. Learning shoulder arthroscopy. In: Shoulder arthroscopy. 2nd ed. Philadelphia: Lippincott Williams & Wilkins; 2003. p. 1-11.
- 15.On-line image. Available: https://secure.sawbones.com/products/product.asp?c=21|26|1&p=383 (accessed 2005 July 20).
- 16.Hariri S, Rawn C, Srivastava S, et al. Evaluation of a surgical simulator for learning clinical anatomy. Med Educ 2004;38:896-902. [DOI] [PubMed]
- 17.Grober ED, Hamstra SJ, Wanzel KR, et al. The educational impact of bench model fidelity on the acquisition of technical skill: The use of clinically relevant outcome measures. Ann Surg 2004;240:374-81. [DOI] [PMC free article] [PubMed]
- 18.Datta V, Bann S, Beard J, et al. Comparison of bench test evaluations of surgical skill with live operating performance assessments. J Am Coll Surg 2004;199:603-6. [DOI] [PubMed]
- 19.O'Neill PJ, Cosgarea AJ, Freedman JA, et al. Arthroscopic proficiency: A survery of orthopaedic sports medicine fellowship directors and orthopaedic surgery department chairs. Arthroscopy 2002;18:795-800. [DOI] [PubMed]
- 20.Anastakis DJ, Regehr G, Reznick RK, et al. Assessment of technical skills transfer from the bench training model to the human model. Am J Surg 1999;177:167-70. [DOI] [PubMed]