Abstract
For clinical simulation to be of maximum benefit, obtaining timely feedback from participants is vital in ensuring suitable improvements are made in the content and delivery of teaching in this setting. This report reviews the literature and describes the use of quick response (QR) codes instead of paper feedback forms following simulation-based learning sessions for fourth-year medical students. This newly implemented electronic method of collecting feedback has resulted in an increase in feedback response rate, reduction in administrative workload and a reduced carbon footprint. We also discuss other QR code-based innovations currently being implemented in this setting.
Keywords: feedback, Qr codes, simulation-based medical education
Introduction
The concept of service providers requesting feedback from end users has evolved over the years. In medical education, the main purpose of feedback from trainees and students is to enhance quality of learning and teaching.1 Timely delivery and analysis of feedback generated is important.2 3 Unfortunately, this remains an issue that exasperates both the service provider as well as the feedback provider.4
Feedback can be obtained by questionnaire in the form of paper or electronically; via links sent across email.4 Paper format notably has issues with a poor return rate and in cases where handwriting is illegible; interpreting can become a major problem. Also, the need for conversion into an electronic format for analysis and storage can be particularly taxing for the technology naive. Written feedback does not guarantee participants anonymity due to identification of handwriting.5 In the present day when sustainability and minimising harm to the environment is topical, the reliance and use of paper for feedback is notably not eco-friendly.6 The introduction of quick response (QR) codes has presented a method of overcoming these hurdles. In this report, we discuss the use of QR codes for feedback from medical students in the context of simulation-based clinical education.
Background
Invented in 1994, QR codes were created for the purpose of assisting manufacturers keep track of vehicle parts. They represent a two-dimensional barcode capable of holding thousands of numeric, alphanumeric, binary data and kana.7
QR codes can be generated free of charge via the internet and read using any mobile phone with a preinstalled QR code reader, obviating the need of text to transfer information.7 8 Most of the studies reviewing the use of QR codes have mostly been carried out in retail. Ertekein and Pelton noted that unlocking promotional deals served as incentive for consumers who used their smart phones to scan QR codes in magazines. In return, this provided feedback to marketers, enabling better target of appropriate audience for various advertisements.9
Methods
We performed our database search and reviewed the literature using the National Institute for Health and Care Excellence -Healthcare Databases Advanced Search; this is summarised in table 1. Our database search yielded 383 hits for QR Codes. When ‘QR codes and medical students and feedback’, ‘QR codes and simulation feedback’ were searched, we received four hits and one hit respectively.
Table 1.
Summary of the database and literature search performed using National Institute for Health and Care Excellence-Healthcare Databases Advanced Search
| Databases searched | Date of search | Search term | Hits after removing duplicates | Comments |
| BNI, CINAHL, EMBASE, MEDLINE, PubMed, PsycINFO | 7 November 2018 | QR codes | 383 | Innovative uses of quick response (QR) codes
|
| QR codes and feedback and medical students | 4 |
*In a randomised trial consisting 29 obstetrics and gynaecology clerkship sites (University of Washington), instructors found QR code evaluations superior to paper evaluations. |
||
| QR codes and feedback and simulation | 3 |
|
Our simulation education centre routinely used paper feedback forms after simulation teaching. In April 2018, we trialled the use of online surveys accessed via QR codes to obtain feedback. QR codes were generated by faculty and displayed at a convenient place in the simulation suite. To leave feedback, the fourth-year medical students were directed to scan the QR code. Scanning the QR code with their phones or other suitable electronic devices with QR code readers, automatically directed them to a survey page where they could fill in their feedback. We then converted feedback collection across all the other clinical simulation-based training to QR codes. Subsequently, we directly compared paper-based feedback to the use of online surveys via QR codes in terms of speed of response, cost, time invested (from creation to analysis), response rates, quality of feedback received as well as usability by members of the faculty.
Results
Between 2018 and 2019, on average 250 sessions of simulation-based learning took place at our trust. The mean participant size per session was 10. Minimum number of individual feedback requested through the year was 2500.
Paper feedback took an estimated administrative time of 208 hours a year (figure 1 shows distribution of this time spent). This included time to translate, manually distribute, collect, collate and produce results for all faculty to access in a readable form. With use of QR codes for feedback, time spent reduced to 12 hours in the last 12 months (see figure 2). Feedback rate rose to 100% from 70%. The quality of feedback remarkably improved, demonstrating more candid opinions of the participants as well as positive criticism. Members of the simulation faculty found it very user-friendly and became more optimistic about receiving feedback especially as electronical analysis could be carried out on the spot.
Figure 1.
Chart showing total time (per year) invested in obtaining paper-based feedback.
Figure 2.
Chart showing total time (per year) spent in obtaining feedback using online survey accessed via QR Code. QR, quick response.
Regarding paper-based feedback (for 250 simulation sessions per year), 208 working hours cost £2007.20 (at a band 3 pay rate of £9.65 per hour). Calculated cost per year of procuring 2500 sheets of paper based on the mean participant attendance of 10 per session was £8.15. Equitrac estimated cost per black and white print was £100 (£0.04 per sheet). The total cost for paper feedback was £2115.35. Annual subscription to an online survey site costs the trust £350, but QR codes are generated free over the internet. The 12 work hours spent producing the feedback electronically cost £115.8 (£9.65 per hour). Therefore, the use of online survey via QR codes for feedback worked out cheaper in comparison to paper-based feedback.
Storage and disposal of paper feedback forms ceased to be of concern. Using QR codes to direct learners to an online survey to provide feedback, as opposed to paper forms, would save one tree every 4 years (as per the online tree calculator), implying the use of an online survey was more eco-friendly.6
Discussion
‘BeeHive’ is a simulation faculty at Medway National Health Service Foundation trust with about 250 sessions of multidisciplinary team simulation based training annually. Previously, we issued structured paper-based feedback questionnaire after simulation sessions. It was a herculean administrative task to distribute, collect, analyse and act on the feedback.
Using paper feedback, we had a 60%–70% return rate; often poor quality. Google forms were tried with no improvement in response rates. Our next approach was to use a commercial electronic survey method emailing the intended respondents. As this records internet protocol address, there are issues with anonymity. We then looked at QR codes and bit.ly URL to access feedback. Both bit.ly and QR codes were good options as they allowed for the analysis of feedback in real time. Improvements, if necessary, could be implemented during the current session and other sessions held that day. However, to access bit.ly, a shortened web link has to be typed in, and there is the possibility of input errors. QR codes just by being scanned, automatically link to the online feedback page, and this is far less likely to result in errors. Second bit.ly is useful for candidates to access material multiple times, but for the purpose of one-off feedback, there is not a requirement for this.
In medicine, smart phone applications have increasingly become recognised as methods of disseminating information. A systematic review by Mosa et al,10 which reviewed 55 articles, over 80 different medical smart phone applications had been mentioned. Of these, more than 66% were related to disease diagnosis and drug formularies. Only one-eighth of these applications were designed to cater for medical or nursing students.10 There was no mention of the use of quick reader codes specifically for feedback for clinical-based simulation learning.
Jamu et al 8 evaluated the use of QR codes in enhancing ‘Just in Time Learning’. Their report highlighted that the uptake, though high was not consistent and that further training would be required for users less acquainted with technology.8 In a randomised control trial comparing paper and QR feedback, Sobhani et al 11 assessed the experiences of both instructors and their obstetrics and gynaecology Students. Their results revealed that indeed instructors were more likely than students to agree that feedback evaluation using QR codes was superior to paper-based evaluation.11 Conversely, according to Snyder et al,12 QR code feedback was deemed potable and easy to access, it also resulted in timely delivery of feedback by the participants.12
Future innovation
In addition to immediate feedback, we have now developed a method of receiving delayed feedback linked with QR codes for our foundation year 1 trainees following their transition to foundation training simulation training. This affords us the opportunity to gauge which skills and knowledge garnered from simulation training are still kept up in ongoing clinical practice even a few weeks or months down the line.
We have gone on to develop screen-based simulation using QR codes. Information is stored on a cloud drive (laboratory results, temperature, X-rays) and then linked to a QR code which the participants then scan into as the scenario progresses. Screen-based real time simulations reduce the constant need for such computer-controlled manikins; maybe even simulation suites in the near future and would hopefully become a cost-effective alternative.
For the year 4 mid-block medical students, knowledge surveys linked to QR codes were provided to give them an opportunity to offer feedback highlighting areas of satisfaction or areas requiring improvement. This procedure will provide an objective method of ascertaining their progress while there is still time to implement changes rather than at the very end of the block.
We aim to carry on monitoring our feedback rate and quality, as well as optimise the use of QR codes.
Conclusion
The importance of effective feedback cannot be over emphasised. In this setting of simulation-based education for medical students, as compared with paper forms, use of an online survey platform accessed via QR codes for feedback was less expensive, yielded higher response rates, timelier response, content of the feedback was more candid, and members of the faculty found it availed them quicker analysis and establishment of the improvements required. Although the uptake of this technology in the medical terrain appears tardy, there remains ample opportunity to explore further its use in this area.
Footnotes
Contributors: All the authors in different but substantial capacities contributed to the conception, design, data collection, analysis, drafting and revising of this work.
Funding: The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests: None declared.
Provenance and peer review: Not commissioned; externally peer reviewed.
References
- 1. Brennan J, William R. Collecting and using student feedback on quality and standards of learning and teaching in higher education, 2003. [Google Scholar]
- 2. Singhapricha T, Meloy P, Shah B, et al. The Use of Quick Response (QR) codes to improve resident compliance and assessment western. J Emerg Med 2018;19. [Google Scholar]
- 3. Barnes N, Nokowaskowski A. The Use of QR Codes to Promote Timely Feedback Family Medicine. 2018;50:635. [DOI] [PubMed] [Google Scholar]
- 4. Taylor-Powell E, Renner M. Collecting Evaluation data: End-of-session Questionnaires (No. 11). University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation 2009 p3-4. http://www.cefe.illinois.edu/tools/Evaluation%20Training%20Resources/UW%20Extension%20Collecting%20Evaluation%20Data_1108.pdf.
- 5. Heath NM, Lawyer SR, Rasmussen EB. Web-Based versus Paper-and-Pencil Course Evaluations. Teach Psychol 2007;34:259–61. 10.1080/00986280701700433 [DOI] [Google Scholar]
- 6. Tree House. Tree usage calculator. http://treehouse.i-hmc.co.uk/calculator/index.php (Accessed 24 Dec 2018).
- 7. Rivers JD. Utilizing the Quick Response (QR) Code within the Japanese EPL Environment. Jalt Call Journal 2009;5:p15–28. [Google Scholar]
- 8. Jamu J, Lowi-Jones H, Mitchell C. Nurse education in practice. Science 2016;19:107–12. [DOI] [PubMed] [Google Scholar]
- 9. Ertekein S, Pelton LE. An empirical study of consumer motivations to use QR codes on magazine ads. American Journal of Contemporary Research 2014;5. [Google Scholar]
- 10. Mosa AS, Yoo I, Sheets L. A systematic review of healthcare applications for smartphones. BMC Med Inform Decis Mak 2012;12:67. 10.1186/1472-6947-12-67 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Sobhani NC, Fay EE, Schiff MA, et al. Randomized trial of smartphone-based evaluation for an obstetrics and gynecology clerkship. J Surg Educ 2018;75:1006–13. 10.1016/j.jsurg.2017.11.009 [DOI] [PubMed] [Google Scholar]
- 12. Snyder MJ, Nguyen DR, Womack JJ, et al. Testing Quick Response (QR) codes as an innovation to improve feedback among geographically-separated clerkship sites. Fam Med 2018;50:188–94. 10.22454/FamMed.2018.936023 [DOI] [PubMed] [Google Scholar]