Abstract
Use of mobile tablet computers (MTCs) in residency education has grown. The objective of this study was to investigate the impact of MTCs on multiple specialties' residency training and identify MTC adoption impediments. To our knowledge, this current project is one of the first multispecialty studies of MTC implementation. A prospective cohort study was formulated. In June 2012 iPad2s were issued to all residents after completion of privacy/confidentiality agreements and a mandatory hard-copy pre-survey regarding four domains of usage (general, self-directed learning, clinical duties, and patient education). Residents who received iPads previously were excluded. A voluntary post-survey was conducted online in June 2013. One-hundred eighty-five subjects completed pre-survey and 107 completed post-survey (58% overall response rate). Eighty-six pre- and post-surveys were linked (response rate of 46%). There was a significant increase in residents accessing patient information/records and charting electronically (26.9% to 79.1%; P<.001), but a significant decrease in looking up drug and treatment reference material (97.0% to 82.1%; P=.0039). There was a significant increase in MTC use as a primary means of charting when conducting rounds (4.9% to 39.5%; P<.001) and a significant decrease in using paper charts (30.1% to 15.7%; P=.0073). There was also a significant increase in MTC use as a primary means for explaining a diagnosis (7.7% to 57.7%; P<.001). The use of MTC has an impact on how residents approach medical education, clinical practice, and patient education. The survey tool may be useful in collecting data on MTC use by other graduate medical education programs.
Keywords: mobile tablet computers, graduate medical training
Introduction
The adoption of mobile tablet computers (MTCs) among physicians has become increasingly common, particularly in the clinical setting. In a recent survey of 2,950 physicians, 72% reported owning a tablet computer and over half using these devices at point of care.1 Recent studies described the benefits of MTC implementation in a variety of settings, including the emergency department and in rural locations.2–8 Interest regarding the utility of MTCs in graduate medical education (GME) has been noted from a variety of medical and surgical specialties.9–19
A number of specialties have evaluated the impact of MTC implementation on residents. The adoption of MTCs has been associated with increased perceived and actual resident efficiency in an internal medicine residency program, although enthusiasm was slightly diminished in a follow-up study.9,10 Other internal medicine programs have also found high integration and clinical use of MTCs following distribution.11 Radiology residents have indicated that MTCs would be beneficial to them, particularly in studying, in addition to a change from printed to electronic educational materials in this specialty after MTC distribution.12–15 Likewise, 81.6% of residents in an anesthesia program either agreed or strongly agreed that use of MTCs would improve their ability to learn their specialty.16 In a study of orthopaedic residents on an anesthesia rotation, it was similarly found that the residents' perception of the quality of the instruction was improved significantly following distribution of a syllabus, educational materials, and schedule on an iPad instead of a printed format.17 In neurosurgical training, the easy access to information through the MTC and its portability provided more opportunities and time for studying for 92% of residents polled and an improvement in both global scoring and on 16 of 18 individual scoring areas of the Congress of Neurological Surgeons Self-Assessment examination.18
More recently, a small multispecialty, resident pilot study found that there was a self-reported increase in clinical efficiency in addition to reported feelings that the universal adoption of iPads would have benefits in coordination of care and educational activities.19 To our knowledge, there are no studies on the impact of the implementation of mobile tablet computing on all residents in a multispecialty GME institution. This study seeks to delineate the impact of MTCs on various resident specialties regarding patient education: general, didactic, and clinical use, and to identify impediments to MTC adoption.
Methods
To evaluate the effect of MTCs on residents, a prospective cohort study was formulated. All residents were issued iPad 2s20 at the start of the 2012–2013 academic year to support clinical and educational duties related to residency training. A mandatory completion of a hard-copy pre-survey and privacy/confidentiality agreement was required. The University of Hawai‘i residency training programs total approximately 220 institutional residents and fellows from nine core programs and six subspecialty fellowship programs. The pre- and post-surveys were done upon the request of the residency program administration because they wanted to determine whether the introduction of the iPads had any impact on the residents' educational experiences/training. The overall hypothesis was: Use of the iPad would make residency training more efficient and effective by making educational materials more readily available to the residents.
The survey instrument was based on a previous assessment of residents from two programs (Orthopaedic Surgery and Pathology) who had received iPads at the start of the prior academic year (2011–2012) as part of a pilot study. The survey items were developed largely based on the researchers' past experiences and perceptions of tablet use. In addition to the collection of demographic information, there were four primary domains of interest that were addressed: general computer usage, self-directed learning, usage in clinical duties, and usage in patient education activities. The voluntary post-survey was administered to all residents online (via SurveyMonkey®) immediately prior to the conclusion of the 2012–2013 academic year. The pre- and post-survey questions were identical, with the exception of utilizing past tense for the post-test survey.
A total of 185 residents were included in this study. Participants were allowed to install additional applications, and several specialties reimbursed residents for applications purchased during the study. Continual remote access to electronic medical libraries was provided in addition to wireless network access across the various training locations.
For questions with only a single selection out of a group, we dichotomized each variable based on the frequency distribution. For example, “time spent on study weekly” was coded into over 10 hours (10–13 hours, 14+ hours) vs less than 10 hours (0–3 hours, 4–7 hours, 7–10 hours). For questions where one could select multiple choices, we coded each choice into a binary variable: chosen vs not chosen. The University of Hawai‘i's Institutional Review Board deemed this study as exempt (UH CHS #19371).
The survey data were summarized by descriptive statistics: frequencies, for categorical variables and for ordinal variables; means (standard deviations) of ranks of the data. Pre- and post-survey responses were compared using matched-pair McNemar's test for binary variables and non-parametric Wilcoxon signed rank test for ordinal variables, based on ranks of the data. All analyses were performed in SAS 9.3 21 and a two-tailed P-value <.05 was considered statistically significant.
Results
One hundred eighty-five subjects completed the pre-survey and 107 completed the post-survey for a 58% overall response rate. Respondents had a mean age of 31±6 years and 52% were female (Table 1). Eighty-six pre- and post-surveys could be linked and merged for a response rate of 46%.
Table 1.
Summary of Demographics Information of Those Who Completed the Pre-survey
| Age (year), Mean ± SD | 31 ± 6 | |
| Female, n (%) | 96 (52%) | |
| Residency Programs | Sub-Specialty Fellowships | Pre-Survey Distribution by Program (n) |
| Family medicine | 18 | |
| Internal medicine | 40 | |
| Cardiovascular disease | 4 | |
| Geriatric medicine | 4 | |
| Obstetrics and gynecology | 25 | |
| Orthopaedic surgery | 2 | |
| Pathology-anatomic and clinical | 4 | |
| Pediatrics | 23 | |
| Neonatal-perinatal medicine | 1 | |
| Psychiatry | 26 | |
| Child and adolescent psychiatry | 1 | |
| Geriatric psychiatry | 3 | |
| Surgery | 21 | |
| Surgical critical care | 2 | |
| Transitional year | 9 | |
| Unidentified | 2 | |
| Total | 185 |
SD = standard deviation
Comparing pre- and post-survey responses, a significantly lower proportion of residents reported using their MTC to look up drug and treatment reference material (97.0% to 82.1%; P=.0039) and a significantly higher proportion to access patient information and records (26.9% to 79.1%; P<.001) following implementation (Table 2). There was no significant difference in the other activities between the pre- and post-surveys. For types of MTC use, residents ranked patient education higher (rank difference: post-pre=−0.47, P=.0053) and studying lower in the post-survey (rank difference: post-pre=0.39, P=.039), while there was no difference in clinical use or entertainment (Table 2).
Table 2.
Summary of Resident Use of Mobile Table Computer
| Survey Question | # Completed Surveys | Pre- / Post-Survey Status | P-value | |||||
| Yes / Yes: n |
Yes / No: n |
No / Yes: n |
No / No: n |
Pre-Survey Yes: n (%) |
Post-Survey Yes: n (%) |
|||
| Activity with tablet computer | ||||||||
| Look up drug and treatment reference material | 67 | 54 | 11 | 1 | 1 | 65 (97.0%) | 55 (82.1%) | .0039 |
| Access patient information/records | 67 | 14 | 4 | 39 | 10 | 18 (26.9%) | 53 (79.1%) | <.001 |
| # Completed Surveys | Pre-survey rank mean (SD) | Post-survey rank mean (SD) | Rank Difference (Post − Pre) mean (SD) | P-value | ||||
| Type of use | ||||||||
| Studying | 73 | 1.62 (0.79) | 2.01 (1.34) | 0.39 (1.60) | .039 | |||
| Clinical use | 73 | 1.88 (0.81) | 2.12 (1.02) | 0.24 (1.31) | .12 | |||
| Patient education | 73 | 3.15 (0.97) | 2.68 (0.99) | −0.47 (1.42) | .0053 | |||
| Personal entertainment | 73 | 3.69 (1.02) | 3.68 (0.92) | −0.01 (1.32) | .93 | |||
| Other | 73 | 5.59 (0.94) | 4.51 (0.90) | −1.08 (1.37) | <.001 | |||
Note: Each respondent's answers to the pre- and post-distribution surveys were matched for McNemar Chi-square analysis for categorical variables and Wilcoxon signed rank test for ordinal variables. “# of completed surveys” is the number of respondents who provided responses to the same question in both the pre- and post- surveys. SD = standard deviation.
Impediments to MTC Adoption
There were no significant differences between the pre- and post-surveys in terms of residents' concerns about using mobile device to communicate between physicians (results not shown). For patient care activities, there was a significant increase in the concern that patients did not have the technology (P=.018) (results not shown).
Educational Use
Table 3 summarizes educational usage, clinical usage, and use in patient education of the MTCs. With regard to educational usage, there was a significant decrease in the percent of time spent studying from textbooks (P<.001). Similarly there was a significant decrease in the proportion of respondents studying over 10 hours weekly (P=.033). There was no difference in time spent studying from a computer. There was a significant decrease in use of print articles (59.0% to 42.2%; P=.013) as well as using a computer for assigned reading (47.0% to 31.3%; P=.042); conversely, there was a significant increase in the use of a MTC for assigned reading (14.5% to 66.3%; P<.001). Use of a phone for reading was rare in the pre and post-distribution analysis. Following distribution of the tablets, residents were significantly more likely to study at a hospital (rank difference: post-pre = −1.11, P<.001) or library (rank difference: post-pre=−1.90, P<.001) and less likely to study at home (rank difference: post-pre = 1.60, P<.001).
Table 3.
Medical Education, Clinical Practice and Patient Education Use of Mobile Tablet Computers
| Medical Education | ||||||||
| # Completed Surveys | Pre-survey rank mean (SD) | Post-survey rank mean (SD) | Rank Difference (Post − Pre) mean (SD) | P-value | ||||
| Preferred location to study | ||||||||
| Hospital study area | 71 | 3.24 (1.51) | 3.38 (1.24) | 0.14 (1.75) | .50 | |||
| Hospital work area | 71 | 3.17 (1.46) | 2.06 (1.16) | −1.11 (1.74) | <.001 | |||
| Library | 71 | 4.35 (1.43) | 2.44 (1.34) | −1.90 (2.05) | <.001 | |||
| Home | 71 | 1.63 (0.98) | 3.26 (1.28) | 1.60 (1.61) | <.001 | |||
| Coffee shop | 71 | 3.75 (1.68) | 3.85 (1.27) | 0.10 (1.65) | .62 | |||
| Survey Question | # Completed Surveys | Pre / Post Survey Status | P-value | |||||
|
Yes / Yes: n |
Yes / No: n |
No / Yes: n |
No / No: n |
Pre-Survey Yes: n (%) |
Post-Survey Yes: n (%) |
|||
| Percentage of time studying with tool | ||||||||
| Studying from textbooks: Yes: “>=50%” No: “<50%” |
84 | 25 | 30 | 2 | 27 | 55 (65.5%) | 27 (32.1%) | <.001 |
| Weekly studying time: Yes: “10+ hrs” No: “<10 hrs” |
83 | 6 | 16 | 6 | 54 | 22 (26.5%) | 12 (14.5%) | .033 |
| What was used to read assigned article | ||||||||
| Hard copy | 83 | 26 | 23 | 9 | 25 | 49 (59.0%) | 35 (42.2%) | .013 |
| Computer | 83 | 12 | 27 | 14 | 30 | 39 (47.0%) | 26 (31.3%) | .042 |
| Tablet Computer | 83 | 10 | 2 | 45 | 26 | 12 (14.5%) | 55 (66.3%) | <.001 |
| Clinical Practice | ||||||||
| How were procedures/cases logged | ||||||||
| Tablet computer in addition to ACGME log: | 81 | 3 | 1 | 29 | 48 | 4 (4.9%) | 32 (39.5%) | <.001 |
| How did you chart when conducting rounds? | ||||||||
| Paper Charts | 83 | 9 | 16 | 4 | 54 | 25 (30.1%) | 13 (15.7%) | .0073 |
| Tablet Computer | 83 | 1 | 4 | 22 | 56 | 5 (6.0%) | 23 (27.7%) | <.001 |
| Primary means by which you viewed radiographic images | ||||||||
| Dedicated PACS computer system (hospital) | 82 | 34 | 25 | 13 | 10 | 59 (72.0%) | 47 (57.3%) | .052 |
| Tablet Computer | 82 | 1 | 1 | 6 | 74 | 2 (2.4%) | 7 (8.5%) | .059 |
| Patient Education | ||||||||
| Print-out/hand-out | 26 | 13 | 6 | 1 | 6 | 19 (73.1%) | 14 (53.8%) | .059 |
| Tablet Computer | 26 | 2 | 0 | 13 | 11 | 2 (7.7%) | 15 (57.7%) | <.001 |
| Will the use of the tablet computer improve patient satisfaction? Yes: “Improve” No: “No effect” |
82 | 31 | 19 | 10 | 22 | 50 (61.0%) | 41 (50.0%) | .095 |
Note: Each respondent's answers to the pre- and post-distribution surveys were matched for McNemar Chi-square analysis for categorical variables and Wilcoxon signed rank test for ordinal variables. “# of completed surveys” is the number of respondents who provided responses to the same question in both the pre- and post- surveys. ACGME = Accreditation Council for Graduate Medical Education; HR = hour; PACS = Picture Archive and Communication System; SD = standard deviation.
Clinical Use
There was a significant increase in the proportion of tablet use for logging procedures and cases (4.9% to 39.5%; P<.001) (Table 3). There was no difference in the use of a traditional computer for charting, however, there was a significant decrease in use of paper charts (30.1% to 15.7%; P=.0073) concomitant with a significant increase in use of a MTC for charting (6.0% to 27.7%; P<.001). There was a non-significant trend towards decreased use of Picture Archive and Communication System (PACS) (72.0% to 57.3%; P=.052) and a non-significant increased use of a tablet as the primary means for radiographic image viewing (2.4% to 8.5%; P=.059) (Table 3).
Patient Education
Regarding patient education, there were no significant differences in the use of media or an internet page and printouts or handouts as a primary means to explain a diagnosis in the pre and post-distribution survey. However, there was a significant increase in the use of a MTC as a primary means for explaining a diagnosis (7.7% to 57.7%; P<.001) (Table 3). There was no significant difference in the proportion of residents who felt that MTCs would reduce the patient's length of stay in the hospital or affect patient satisfaction.
Discussion
There has been recent interest in the literature in utilizing MTCs in GME. We sought to determine this effect across five domains: general use, impediments to use, educational use, clinical use, and use for patient education. There was a significant increase in the proportion of residents that used their iPad to access patient information and records with a small but significant decrease in use to find drug and treatment reference material in “general use.” These findings are consistent with other studies.9,11,19 Interestingly, the only significant change in pre- and post-distribution response concerning impediments to utilization of the device with physicians or patients was an increase in concern that patients did not have the technology.
In “educational use,” residents identified a significant increase in the use of hospital work areas and libraries as study locations with a commensurate decrease in the preference for studying at home. There was a significant shift from the use of print materials to the use of the tablet computer for access to textbooks and journal articles, as noted previously.13–15 There was also a significant decrease in the proportion of residents indicating that they studied for 10+ hours. This is in contrast to what was observed in other studies.13,14 However, this difference in findings may be a result of increased studying efficiency similar to the increased efficiency observed in internal medicine residents with iPads.9 Future studies could help determine if increased studying efficiency is present and if there is an improvement on objective outcomes such as in-training examination scores, as was demonstrated among orthopaedic residents in a previous pilot study.22
Similar to the findings of MTC use in the general domain analysis, “clinical usage” residents indicated a significant increase in the use of their MTC for charting while conducting rounds. Over the same period there was a significant decrease in the proportion of residents reporting use of paper charting. This may highlight the advantages of access and portability of the MTC, but may also reflect a hospital/system requirement. This finding is similar to other studies where it was noted that iPads were heavily used for clinical work and were noted to have improvements on workflow.9,11
Residents indicated an increase in their usage of their iPad as a primary means for explaining a diagnosis to a patient. There was also an increase in the proportion of residents who indicated that they felt use of iPads by residents would lead to improved patient satisfaction. This is similar in theme to the positive patient comments noted in studies of internal medicine residents.10
This study has a number of limitations. This is a single-institution study with a limited number of specialty residency programs, which may limit its generalizability. In both the pre- and post-distribution surveys the linked response rate was 46%, and indications of usage on the survey may have been affected by recall or participation bias. Also, the overall sample size does not allow for comparisons between specialties. Additionally, a few weeks prior to distribution of the second survey, policy was changed such that the iPads were available for purchase at the end of residency at full price rather than a nominal fee. This may have led to negative associations with the study, and may have resulted in the lower response rate of the secondary survey. The lack of qualitative data limited our ability to explain the quantitative results. No educational outcome data, eg, in-training assessment data, were linked with the current study to formally evaluate the effectiveness of the MTC use. Lastly, this study did not specifically address potential concerns of patient privacy or data security. While a small majority indicated that concerns of patient privacy did not hold them back from pursuing physician-to-physician or physician-to-patient activities with their device, this was not explored further. However, the devices were secured with a password, and access to the EHR was through a password-protected secure server.
Conclusion
Since this study was initiated, the use of MTCs in GME has grown significantly. The use of MTC has a significant impact on how residents approach medical education, clinical practice, and patient education. The survey tool can be useful in collecting data on MTC use by other graduate medical education programs. However, formal evaluations of the effectiveness in improving educational outcomes are still needed. A more thorough assessment (quantitative and qualitative) needs to be conducted to determine the long-term impact of iPad use in residency education/training. A follow up study could be conducted across all specialties to identify commonalities in experience versus specialty-specific issues. If needed, this could aid efforts to better tailor MTC use, especially with regard to learning how to optimally care for patients.
Acknowledgement
The authors would like to thank Stephanie Nishimura PhD and Jamie Castelo BA for their assistance with reviewing parts of the manuscript, and Jewellien Robinson for her technical assistance.
Conflict of Interest
None of the authors identify any conflicts of interest.
Funding: JJC and YC were partially supported by grants U54MD007584 and G12MD007601 from the National Institute on Minority Health and Health Disparities and P20GM103466 from the National Institute of General Medical Sciences from the National Institutes of Health. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.
References
- 1.Research M. Taking the pulse: U.S. 2013. New York, New York: Manhattan Research; 2013. [Google Scholar]
- 2.Horng S, Goss FR, Chen RS, Nathanson LA. Prospective pilot study of a tablet computer in an Emergency Department. International Journal of Medical Informatics. 2012 May;81(5):314–319. doi: 10.1016/j.ijmedinf.2011.12.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Rao AS, Adam TJ, Gensinger R, Westra BL. Study of the factors that promoted the implementation of electronic medical record on iPads at two emergency departments. AMIA … Annual Symposium proceedings / AMIA Symposium. AMIA Symposium. 2012;2012:744–752. [PMC free article] [PubMed] [Google Scholar]
- 4.Anderson C, Henner T, Burkey J. Tablet computers in support of rural and frontier clinical practice. International Journal of Medical Informatics. 2013 Nov;82(11):1046–1058. doi: 10.1016/j.ijmedinf.2013.08.006. [DOI] [PubMed] [Google Scholar]
- 5.Murphy AD, Belcher HJ. A novel method for sterile intra-operative iPad use. Journal of Plastic, Reconstructive & Aesthetic Surgery: JPRAS. 2012 Mar;65(3):403–404. doi: 10.1016/j.bjps.2011.08.037. [DOI] [PubMed] [Google Scholar]
- 6.Mashman W. The iPad in cardiology: tool or toy? JACC. Cardiovascular Interventions. 2011 Feb;4(2):258–259. doi: 10.1016/j.jcin.2011.01.001. [DOI] [PubMed] [Google Scholar]
- 7.Szekely A, Talanow R, Bagyi P. Smartphones, tablets and mobile applications for radiology. European Journal of Radiology. 2013;82:829–836. doi: 10.1016/j.ejrad.2012.11.034. [DOI] [PubMed] [Google Scholar]
- 8.Berger E. The iPad: gadget or medical godsend? Annals of Emergency Medicine. 2010 Jul;56(1):A21–A22. doi: 10.1016/j.annemergmed.2010.05.010. [DOI] [PubMed] [Google Scholar]
- 9.Patel BK, Chapman CG, Luo N, Woodruff JN, Arora VM. Impact of mobile tablet computers on internal medicine resident efficiency. Archives of Internal Medicine. 2012 Mar 12;172(5):436–438. doi: 10.1001/archinternmed.2012.45. [DOI] [PubMed] [Google Scholar]
- 10.Luo N, Chapman CG, Patel BK, Woodruff JN, Arora VM. Expectations of iPad use in an internal medicine residency program: is it worth the “hype”? Journal of Medical Internet Research. 2013;15(5):e88. doi: 10.2196/jmir.2524. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Walsh C, Stetson P. EHR on the move: resident physician perceptions of iPads and the clinical workflow. AMIA … Annual Symposium proceedings / AMIA Symposium. AMIA Symposium. 2012;2012:1422–1430. [PMC free article] [PubMed] [Google Scholar]
- 12.Korbage AC, Bedi HS. Mobile technology in radiology resident education. Journal of the American College of Radiology : JACR. 2012 Jun;9(6):426–429. doi: 10.1016/j.jacr.2012.02.008. [DOI] [PubMed] [Google Scholar]
- 13.Korbage AC, Bedi HS. The iPad in radiology resident education. Journal of the American College of Radiology : JACR. 2012 Oct;9(10):759–760. doi: 10.1016/j.jacr.2012.05.006. [DOI] [PubMed] [Google Scholar]
- 14.Sharpe EE, 3rd, Kendrick M, Strickland C, Dodd GD., 3rd The Radiology Resident iPad Toolbox: an educational and clinical tool for radiology residents. Journal of the American College of Radiology : JACR. 2013 Jul;10(7):527–532. doi: 10.1016/j.jacr.2013.02.007. [DOI] [PubMed] [Google Scholar]
- 15.Berkowitz SJ, Kung JW, Eisenberg RL, Donohoe K, Tsai LL, Slanetz PJ. Resident iPad Use: Has It Really Changed the Game? Journal of the American College of Radiology : JACR. 2013 Jun 25; doi: 10.1016/j.jacr.2013.04.017. [DOI] [PubMed] [Google Scholar]
- 16.Chu LF, Erlendson MJ, Sun JS, Alva HL, Clemenson AM. Mobile computing in medical education: opportunities and challenges. Current Opinion in Anaesthesiology. 2012 Dec;25(6):699–718. doi: 10.1097/ACO.0b013e32835a25f1. [DOI] [PubMed] [Google Scholar]
- 17.Tanaka PP, Hawrylyshyn KA, Macario A. Use of tablet (iPad(R)) as a tool for teaching anesthesiology in an orthopedic rotation. Revista Brasileira de Anestesiologia. 2012 Mar-Apr;62(2):214–222. doi: 10.1016/S0034-7094(12)70119-8. [DOI] [PubMed] [Google Scholar]
- 18.Gonzalez NR, Dusick JR, Martin NA. Effects of mobile and digital support for a structured, competency-based curriculum in neurosurgery residency education. Neurosurgery. 2012 Jul;71(1):164–172. doi: 10.1227/NEU.0b013e318253571b. [DOI] [PubMed] [Google Scholar]
- 19.Lobo MJ, Crandley EF, Rumph JS, et al. Pilot study of iPad incorporation into graduate medical education. Journal of Graduate Medical Education. 2013 Mar;5(1):142–144. doi: 10.4300/JGME-D-12-00007.1. 2013. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.iPad. Cupertino, CA: Apple Inc.; [Google Scholar]
- 21.SAS Version 9.3. Cary N: SAS Institute Inc.; [Google Scholar]
- 22.Varcadipane JC, Dupaix JP, Nishimura S, et al. Mobile computing as a tool in orthopaedic surgery residency training: a pilot study. Current Orthopaedic Practice. 2015;26(4):376–381. [Google Scholar]