Abstract
Objectives
Conduct a scoping review to identify the approaches used to integrate digital literacy into undergraduate pharmacy programs across different countries, focusing on methods for education, training, and assessment.
Materials and methods
Following the Joanna Briggs Institute methodology, we searched 5 electronic databases in June 2022: MEDLINE (Ovid), PubMed, Embase, Scopus, and CINAHL. Three independent reviewers screened all articles; data extraction was conducted by 2 reviewers. Any discrepancies were arbitrated by 2 additional reviewers.
Results
Out of 624 articles, 57 were included in this review. Educational and training approaches for digital literacy in undergraduate pharmacy programs encompassed a theoretical understanding of health informatics, familiarization with diverse digital technologies, and applied informatics in 2 domains: patient-centric care through digital technologies, and the utilization of digital technologies in interprofessional collaboration. Blended pedagogical strategies were commonly employed. Assessment approaches included patient plan development requiring digital information retrieval, critical appraisal of digital tools, live evaluations of telehealth skills, and quizzes and exams on health informatics concepts. External engagement with system developers, suppliers, and other institutes supported successful digital literacy education.
Discussion and conclusion
This scoping review identifies various learning objectives, teaching, and assessment strategies to incorporate digital literacy in undergraduate pharmacy curricula. Recommendations include acknowledging the evolving digital health landscape, ensuring constructive alignment between learning objectives, teaching approach and assessments, co-development of digital literacy courses with stakeholders, and using standardized guidelines for reporting educational interventions. This study provides practical suggestions for enhancing digital literacy education in undergraduate pharmacy programs.
Keywords: digital health, digital literacy, education, pharmacy students, electronic health records
Introduction
Digital health is an umbrella term that refers to “the use of technologies for healthcare”.1 It encompasses a wide range of fields, including electronic health (eHealth), wearable and mobile health technology (mHealth), and emerging areas, such as big data, genomics, and artificial intelligence (AI).1–3 Digital health technologies (DHTs) have been increasingly important and beneficial in healthcare, with the coronavirus disease 2019 pandemic further accelerating their roles particularly in areas of telemedicine and remote care.4–6
DHTs have been increasingly adopted in pharmacy practice, including electronic patient records (EPRs), clinical decision support (CDS) systems, e-prescribing, and robotic medicine dispensing, to support delivery of patient care.7,8 Pharmacists are using DHT products and services to optimize patient care, for example, to document interventions, perform assessments, patient education, and monitor patients.9 Although the shift toward digital health promises substantial benefits, achieving these will require a digitally literate healthcare workforce.5,10 Furthermore, due to the rapid advancement in the use of technology around the world, the pharmacy profession cannot wait for the gradual incorporation of digital technologies into practice and education.7
In the United States, the Digital Nation initiative was launched in 2010, to promote digital literacy and inclusion, with the Accreditation Council for Pharmacy Education (ACPE) developing standards for Digital Health in 2016.11 These standards were designed to ensure that future pharmacists were equipped to utilize DHTs effectively while providing patient care.12 In the United Kingdom, NHS England supports workforce transformation and defines digital literacy as “those capabilities that fit someone for living, learning, working, participating and thriving in a digital society”.13 In 2021, Health Education England (HEE) (now part of NHS England) published a guidance document on digital literacy for the pharmacy workforce. The report focused on major aspects of digital transformation, including how to improve the pharmacy workforce’s digital literacy and align to the National Health Service’s (NHS) Long-Term Plan and the NHS People’s Plan.13 Despite these efforts, when the International Pharmaceutical Federation reviewed digital health in pharmacy education, they found that a large proportion of pharmacy schools do not incorporate digital health education in their curricula.14
Evidence suggests that improving the digital health literacy of the pharmacy workforce should be comprehensive, starting with undergraduate education and continuing through postgraduate training and beyond.5 As digital literacy is still a relatively new concept in pharmacy education, there is a need to systematically explore the current evidence to identify successful approaches and recommendations for integrating digital literacy into education. A preliminary search of MEDLINE, the Cochrane Database of Systematic Reviews, and Joanna Briggs Institute (JBI) Evidence Synthesis was conducted, and no current systematic reviews on the topic were identified.
Through a scoping review, we aim to address the research question: “What strategies for education, training, and assessing digital literacy are used for undergraduate pharmacy students and trainees globally?” The objectives of the review are as follows: (1) explore educational and training approaches for equipping pharmacy students and trainees with digital literacy skills, (2) identify digital competencies/capabilities included in the curricula, (3) identify the learning outcomes integrated into the curricula, (4) investigate pedagogical methods used to deliver digital literacy content, (5) explore methods for assessing digital literacy in pharmacy students and trainees, and (6) identify factors that contribute to or hinder the effectiveness of implementing digital literacy in pharmacy programs. We further hope to identify areas for improvement and future research.
Methods
The scoping review was conducted in accordance with the JBI methodology for scoping reviews.15 A protocol detailing the proposed method was registered in Open Science Framework: 10.17605/OSF.IO/T6BP5, which is summarized in the following sections.
Eligibility criteria
Table 1 outlines the eligibility criteria for the review which are framed around participants, concept, context (PCC) and types of sources.
Table 1.
PCC | Description |
---|---|
Participants | Studies are selected if they include undergraduate pharmacy students or foundation trainee/preregistration pharmacists. Qualified pharmacists will be excluded. |
Concept | Studies will be considered if they include digital literacy in respect to
|
Context | Worldwide studies. |
Types of sources | All study types and methods were considered. Given that the concept of health informatics (HI) was broadly adopted from 1995, papers prior to this date will be excluded.16 |
Search strategy
The search strategy commenced in June 2022, an initial limited search of MEDLINE (Ovid) to identify keywords contained in the titles and abstracts or index terms of relevant articles. A comprehensive search was then conducted in EMBASE, PubMed, Scopus, CINAHL, MEDLINE (Ovid), and key journals in the research area, for example, American Journal of Pharmaceutical Education, BMC Medical Education, Medical Education, and Medical Teacher with keywords and index terms on concepts of digital literacy, education, training, assessment, and undergraduate pharmacy students and trainees, that were slightly modified to fit each database’s requirements (Figure 1). No search limitations (eg, dates) were applied to the search. The detailed search strategy used for MEDLINE (Ovid), which was the basis for all other searches is included in Table S1. Details of the search strategies used for the other databases are presented in Table S2. Finally, the reference lists of all included sources were screened to identify additional studies. Studies published in English, prior to 1995 were excluded.
Study selection
All identified studies were collated and uploaded into EndNote 20.2 (Clarivate Analytics, PA, USA), where duplicates were removed using the “remove duplicates” function. A secondary manual check was also conducted to ensure complete removal of all duplicates. The study selection was performed in 2 phases and was conducted by 3 reviewers (M.A., G.R., V.B.). Initially, titles and abstracts of the identified studies were screened independently by the reviewers, based on the predefined inclusion and exclusion criteria. Included studies then progressed to the full-text review phase and were further independently screened by all reviewers. The screening process was facilitated using Microsoft Excel, reviewers documented their decisions noting any reasons for exclusion where relevant. Throughout the evidence selection process, periodic meetings were held to discuss divergent opinions and reach collective decisions. Any disagreements that arose were resolved through discussion, or with additional reviewers (C.T., H.N.).
Data extraction
A pilot was conducted using a select sample of approximately 10 studies to test the preliminary data extraction form. Based on the insights from this pilot, modifications were made to the form, now provided in Table S3. The data extraction table included detailed criteria such as PCC, study methods, and key findings in line with the review’s aim. Two independent reviewers (M.A. and G.R.) conducted the data extraction on half the included studies. The lead author further reviewed and double-checked data extraction across all included studies. Where discrepancies in extraction arose, they were discussed between both reviewers (M.A. and G.R.), with further input from C.T. where necessary to reach consensus.
Data analysis
A thematic analysis was conducted to identify common patterns and themes across the studies using the HEE capability framework as our initial thematic framework to code the data, whilst also using an inductive approach to allow for new themes and concepts to be detected.13 This involved reading the full texts and data extraction table in detail and applying codes to the data, which were grouped into themes. We then conducted a narrative synthesis, which involved summarizing the themes in a narrative summary. Our analytical focus was primarily on the competencies and educational interventions surrounding digital literacy education. This included the aim, structure, teaching methods, delivery modes, targeted skills, assessment strategies, and resultant outcomes of these interventions. Constructive alignment theory was further used to assess the presence of alignment of the taught material and assessment approaches within the reviewed studies, only including studies with detailed description of educational interventions.17 To ensure the integrity and reliability of our findings, themes and sub-themes were iteratively reviewed, refined, and validated through multiple rounds of discussions among the research team until a consensus was achieved.
Results
Descriptive summary of the included studies
We identified 624 studies; 557 from database searches and 67 from other sources. Duplicates were removed (n = 273), leaving 351 studies for screening of title and abstract screening. We retrieved 93 studies for full-text review and 51 met the inclusion criteria; a further 6 were included through manual searching of reference lists. A total of 57 studies were included in this scoping review (Figure 2). Most of the studies (n = 36) were published between 2016 and 2022.14,18–64 The majority of the studies (n = 49) were conducted in the United States,14,18–69 with the other studies being conducted in Canada (n = 2),29,70 Romania (n = 1),71 Malaysia (n = 1),72 Singapore (n = 1),73 the United Kingdom (n = 1),40 Poland (n = 1),48 and 1 multi-country study (n = 1.).14 Out of the 57 studies, 11 focused on digital literacy competency and skills.14,24–28,39,40,47,58,62,68 The remaining studies covered various aspects of digital literacy education, including 23 studies on Electronic Health Record (EHR) skills,22,30,32,33,35–38,41,42,44–46,49–51,54,55,60,63–65,73 14 studies on pharmacy informatics courses,19,21,29,31,34,52,59,61,66,67,69–72 4 studies discussed tele-pharmacy,20,43,48,53 3 studies on mobile health (mHealth),18,23,27 and 2 studies covered Prescription Drug Insurance tools.56,57 Additional details about the characteristics of the included studies can be found in Table S4. Table S5 provides an outline for the educational interventions, skills covered, and outcomes of each study.
We identified 4 main themes, some of which had subthemes: (1) competencies, skills, and learning objectives, (2) mode of delivery, (3) assessment methods, and (4) course development (Figure 3). These themes, along with supporting evidence from relevant studies, are discussed in the following sections.
Competencies, skills, and learning objectives
Digital literacy learning objectives
The skills and learning objectives can be classified into 3 major categories: theoretical understanding of health informatics (HI), familiarity with diverse digital technologies, and applied informatics, which centered on 2 specific domains: the delivery of patient-centered care through DHTs and the utilization of DHTs in interprofessional collaboration (Figure 4).
Training aimed to provide students with a theoretical understanding of fundamental concepts and terminology in HI. Furthermore, the learning objectives included student understanding of the application of HI principles in clinical practice and the benefits of HI for patient care and pharmacy services.21,69,71,72 Moreover, the familiarization with various digital technologies intended to expose students to a broad range of technological tools employed in healthcare, thus enhancing their readiness to employ systems such as EHRs, automated technologies, telecommunication tools, and remote dispensing.31,34,44,69,72
Lastly, studies highlighted the application of informatics principles and DHTs in delivering patient-centered care and promoting interprofessional collaboration. This included enhancing documentation skills,22,37,54,55,60 analysis and interpretation of data using digital technology,18,20,22,23,30,32,35,38,49,51,56,57,60,63,73 and skills related to using digital systems for prescription verification, and medication safety monitoring.41,42,49 Interprofessional collaboration was developed through telehealth consultations, which required communication and collaboration within interprofessional teams for medication management.56,57 Multi-disciplinary projects involved developing an mHealth app prototype for smartphones that facilitated engagement with software engineers to address mobile technology requirements and find innovative solutions.27
Digital literacy competencies and skills
Numerous studies examined digital literacy competencies for pharmacy students, focusing on developing skills and knowledge related to informatics terminology, concepts, and building effective communication skills whilst using DHTs.25,39,58,62 Some studies specifically concentrated on competencies related to EHRs40,47 and mHealth.68
Most studies recommended the early introduction of training in undergraduate education, to provide a strong foundation in core competencies aligned with the ACPE standards.25,39,58,62 Fox et al58 proposed core competencies for pharmacy informatics education, including knowledge of basic terminology around technology, informatics, and healthcare; understanding reasons for the systematic processing of healthcare data; and recognizing the benefits and constraints of DHT, to develop practical informatics skills. Other approaches focused on introducing students to technologies used across each stage of the medication use process (ie, procurement, prescribing, order verification, compounding, dispensing, administration, and monitoring).25,62 Martin et al39 meanwhile outlined foundational and applied competencies in HI for pharmacy students. The foundational competencies covered HI concepts, ethical principles, health data management, project management, and communication skills among others. The applied competencies included incorporating HI into patient care, using health information technology for CDS systems, managing medication therapy, support population health management and supporting research.39
Studies also emphasized the importance of training pharmacy students on interoperability standards, biomedical informatics, emerging technologies, legal and regulatory aspects, patient outcomes and safety, and professional development.25,39,62 Flynn et al26 suggested that advanced informatics training should include domains like the design and modeling of information systems, development of prototypes, implementation of health IT, and evaluation of health information resources.
Some studies focused on providing students with opportunities to use and develop skills in using specific DHTs, such as EHRs and mHealth, to prepare them for real-world practice.26 In the United Kingdom, a National Working Group was formed to integrate EPRs into the undergraduate curriculum for healthcare students.40 They defined 6 domains of competence and 29 learning outcomes, including digital health work, accessing data, communicating, generating data, interdisciplinary collaboration, and monitoring and auditing.40
Modes of delivery
Blended learning approach
The majority of the studies employed a blended learning approach.18,20–23,27,30–35,37,38,41–44,46,48,49,51,53–57,60,61,63–65,70–73 Blended learning approaches combined traditional face-to-face instruction with technology-mediated instruction, such as using digital technologies in the classroom,20,22,23,27,30–32,34,35,37,38,41–44,46,48,49,51,53,54,56,60,61,63–65,70–73 providing remote access to DHTs,33,55,57 and giving access to prerecorded online material.18,21,42,74
Practical exposure to digital health technologies
Wasynczuk and Sheehan51 integrated a teaching EHR system within a pharmacy school, giving students access to a simulated patient’s medication history and medical record. Students analyzed the data to produce an accurate response to a patient-specific medication query.51 Students’ perceptions of their knowledge and confidence in using EHR were reported to have improved (from 61.4% to 89.3% and 20.5% to 82.8%, respectively).51 Similarly, Neumann-Podczaska et al48 described an interprofessional telemedicine project, where pharmacy students engaged with medical students in teleconsultations and evidence-based recommendation. These authors reported increased student confidence and understanding of patient care post training.48 Three studies discussed remote access to DHT.33,55,57 Brown et al55 introduced an Internet-based medical chart in a pharmacotherapy course that students accessed from home. The web-based nature of the EHR enabled students to access patient information, develop clinical notes, and view progress notes remotely, however, the study reported connectivity and technical issues, which are important limitations to consider.55
Online training and e-learning
Online training material was used in some studies, alongside traditional learning methods.18,21,42 Hincapie et al21 integrated the online pharmacy informatics module “Partners in E (PinE),” covering fundamental concepts like, interoperability, data management, and DHT. A team-based learning (TBL) approach was used for preclass preparation, individual and team readiness assurance, problem-solving activities, and immediate feedback.21 Pre- and post-survey findings from 83 students revealed a considerable improvement in their knowledge and confidence.21
Online delivery approach
Learning content was delivered solely via online methods in 2 studies.50,69 One study used online modules covering digital literacy topics, including EHRs, telemedicine, and CDSSs.69 The online course significantly improved students’ knowledge, attitudes, and confidence in HI and student experience was positive.69 Another study integrated an EHR system into an online pharmacotherapy course where students practiced the pharmacist’s patient care process.50 The majority (60% [n = 15]) of students reported positive perceptions of this approach, recognizing its value in data collection, therapy assessment, plan development, monitoring/follow-up, and better preparation for experiential rotations involving EHR systems.50
Modes of assessment
Strategies used to assess student proficiency and knowledge for the intended learning outcomes were grouped into 4 broad themes:
Presentation of a patient plan based on information retrieved from a digital system. This included several components such as completing Subjective, Objective, Assessment, and Plan (SOAP) notes,22,37,38,54,55,60,64 submitting handwritten progress notes of simulated cases,33,35,41,63 using EHRs to respond to patient-specific questions,51,73 presenting a simulated patient case to other students in a grand round format,49 and comparing prescription drug plans on an EHR system before choosing the most option.56,57
Live evaluations of digital consultation skills, for example, assessing student performance during telehealth activities.53
Critical appraisal of DHTs, where students completed a worksheet, and outlined their evaluation of different mHealth apps, to determine which one would be most effective for a specific healthcare need.18,23
Quiz and written exams of knowledge and understanding of fundamental HI concepts.21,67,69
Many of the included studies lacked a comprehensive and standardized description of their educational interventions, potentially affecting their effectiveness and reproducibility. However, in studies with detailed descriptions, constructive alignment theory was employed to link the learning objectives, educational activities, and assessments (see Table 2 and Table S6). Learning objectives were also mapped to the HEE capability framework to highlight areas that the education/training addresses.
Table 2.
HEE domain | Learning objective | Related teaching approach | Related assessment mode |
---|---|---|---|
Information, data, and content: able to gather and evaluate relevant health information and data using DHT. | Ability to gather and evaluate relevant patient information and data using EHRs.22,33,35,37,41,42,46,49,51,54,55,60,73 | A hands-on approach to review simulated patient cases using EHRs and develop SOAP notes.22,33,35,37,42,54,55,60 |
|
Incorporation of EHR systems in various courses.46,49 | The study lacks clear information regarding the specific method used for assessment.46,49 | ||
Case-based approach to use EHR for drug information inquiry.51 | Written assignment: Gathering patient-specific information and writing a response to a drug-related question.51 | ||
Simulated EHR using Mobile app.71,73 | The study lacks clear information regarding the specific method used for assessment.73 | ||
Ability to handle drug information.72 | Practical sessions.72 | Evaluation of pharmacy-related Internet sites and establishment of websites.72 | |
Ability to gather, evaluate patients’ information using telehealth.20 | Simulated IP telehealth visit.20 | The study lacks clear information regarding the specific method used for assessment.20 | |
Ability to find, evaluate, and effectively use mHealth technologies.18,23 | |||
Ability to evaluate patients’ information using Medicare Part D plans.56,57 | Lectures on the Medicare Part D program and its basic concepts.56,57 | ||
Communication, collaboration, and participation: Able to effectively communicate and collaborate with other healthcare professionals using digital platforms and tools to improve patient outcomes. | Ability to discuss health technologies with other health care professionals for the common benefit of safe, quality patient care.69 | Online course on HI.69 | Participation in online discussions.69 |
Ability to provide healthcare services to rural area through telehealth.53 | Combination of didactic coursework and experiential learning in tele pharmacy.53 | ||
Ability to communicate and collaborate effectively as an IP team.20,48 | Simulated IP telehealth visit.20,48 | The study lacks clear information regarding the specific method used for assessment.20,48 | |
Ability to communicate mobile technology needs and solutions with software engineers.27 | Course in mHealth for pharmacy and computer science students.27 | ||
Technical proficiency: Able to navigate and troubleshoot digital health tools and software with confidence and efficiency. | Proficiency in using HI systems and software and accessing drug information sources.70–72 | Hands-on experience with software applications in pharmacy.71 | The study lacks clear information regarding the specific method used for assessment.71 |
Practical session on drug information handling skills.72 | Evaluation of pharmacy-related Internet sites and the establishment of websites.72 | ||
Incorporation of hospital information system in HI course.70 | Practical exams on clinical workflows: Complete clinical and administrative tasks related to pharmacy management.70 | ||
Ability to navigate and use of EHRs.22,33,35,37,38,41,42,46,49,51,54,55,60,63,73 | Simulated EHR experience.22,33,35,37,42,54,55,60,63 |
|
|
Simulated EHR using Microsoft PowerPoint slides.38 | The studies lack clear information regarding the specific method used for assessment.38,49,73 | ||
Simulated EHR using mobile app.73 | |||
Incorporation of EHR systems in various courses.49 | |||
Case-based approach to use EHR for drug information inquiry.51 | Assignment: Gathering patient-specific information and writing a response to a drug-related question.51 | ||
Remote accessing and reviewing EHR of real patients.41 | A competency-based assessment to evaluate pharmacy students’ medication safety skills.41 | ||
EHR training approaches in classroom and during APPEs.46 | The study lacks clear information regarding the specific method used for assessment.46 | ||
Ability to use online Medicare tools in evaluating and enrolling in Medicare Part D plans using online tools.56,57 | Introducing a teaching module on Medicare prescription drug.56 | ||
A PBL session using Medicare prescription drug plan finding tool.57 | Case-based exercise to identify the 3 least expensive plans and corresponding costs.57 | ||
Ability to navigate and use telehealth tools.20,48,53 | Combination of didactic coursework and experiential learning in tele pharmacy.53 | Evaluation of student performance: Quality of documentation of a daily log sheet of their activity.53 Live evaluations of student performance in telecommunication stations by the faculty mentor.53 | |
Simulated IP telehealth visit.20,48 | The study lacks clear information regarding the specific method used for assessment.20,48 | ||
Creation, innovation, and research: Able to create and innovate digital health solutions to improve patient care and outcomes and conduct research using DHT. | Ability to design and implement mHealth interventions.27 | Project-based learning and PBL was used mHealth course.27 | |
Objectives not covered by the HEE capabilities framework. | Demonstrate understanding of basic concepts of HI technology.21,34,57,72 | Introducing a pharmacy informatics course using TBL approach.21 | Individual readiness assurance test, team readiness assurance test, and application exercises to evaluate and analyze case scenarios related to HIT and informatics encountered in pharmacy.21 |
Implementation of new course “Pharmacoinformatic” involving lectures on drug information basics.72 | Coursework (40%) essay final examinations (60%).72 | ||
Online lectures on health technologies and informatics integration in pharmacy.69 | Multiple-choice, true-false, and short-answer questions administered at regular intervals.69 | ||
Facilitated discussion on pharmacist’s role in pharmaceutical care.63 | Short quiz assessing understanding of concepts related to readings and laboratory activities.63 | ||
Lectures on a specific digital technology for example, Medicare Part D.57 | Case-based exercise to identify the 3 least expensive plans and corresponding costs.57 | ||
Informatics skills laboratory, covering DHT such as types of automation equipment and the role of technology in the medication use process.34 | Studies lacks clear information regarding the specific method used for assessment.34 |
Course development
Strategies for developing courses
Various strategies have been utilized in developing digital health courses at undergraduate pharmacy schools. These approaches include academic processes such as literature reviews,69 evaluations of existing courses,29,71 and course piloting.34,49 Also, collaborative working with organizations,29,66 experts,53,57 commercial companies,31,34 and other schools.18,20
Fuji et al69 developed an HI elective course based on a literature review with topics selected through an evidence-based approach, stakeholder suggestions, and reviewing informatics in pharmacy practice. Rocchi et al,29 updated an existing e-resource prepared by the Association of Colleges of Pharmacy in Canada, addressing gaps in informatics content in undergraduate curricula.29 Rigorous editorial advisory group and peer reviews were undertaken to ensure the resource was relevant and up-to-date.
Key expert stakeholder involvement was important in several studies, that obtained feedback from clinicians, faculty members, and health policy experts on the developed courses.53,57 Seifert et al53 reported developing a tele-pharmacy teaching model through discussions with members from the Office of Rural and Community Health and the Texas Tech University Health Sciences Centre Telemedicine program, as well as a rural telehealth practitioner. This allowed the team to leverage prior telemedicine experience and ensured compliance with accreditation bodies and regulatory compliance organizations.53 Darley and Logan reported collaborating with a business technology company to provide pharmacy students with a course on automation technology. Company representatives demonstrated their automation equipment on-site, and the course focused on the role of technology in medication usage.34 This approach resulted in significant increases in student confidence and understanding of informatics technologies.34
Lastly, some pharmacy schools collaborated to co-develop a course.18,20 Rodis et al18 reported a collaboration between Ohio State University and Massachusetts College of Pharmacy schools to develop an innovative learning experience for pharmacy students centered on identifying, reviewing, and using medical apps. The course successfully improved perceptions of student skills (n = 119), in the following areas: finding (44% vs 95%), evaluating (15% vs 93%), and using medical applications in patient care (26% vs 90%).18
Challenges and facilitators in course development
Several challenges to incorporating digital health into pharmacy education have been recognized, including the ongoing evolution of digital health, educator training, and the expense and time required to develop such courses.20,21,23,25,37,69–72 The broad scope of HI, and the field’s constant innovation, make it challenging to maintain course content.69,71 There is also a lack of skilled professionals with the necessary knowledge to educate others.20,25,70 The cost of DHTs, and the number of devices required, were further challenges, along with the need to frequently update the equipment to meet modern standards.20,23 Additionally, some pharmacy schools reported hesitation in investing resources in new courses without evidence of their effectiveness.21,37,72
On the other hand, numerous research has identified factors that encourage the development of digital health education and training.21,22,31,34,55,63,66 Collaboration with external partners, such as DHT businesses may also be financially advantageous. In 2 studies, DHT providers gave system demos for free, as a part of community service.31,34 One university also received support from an EHR provider—MEDITECH software package—to implement the system across all health professions programs.63
Discussion
Principal findings
This is the first study to focus on incorporating digital literacy education into undergraduate pharmacy programs. The scoping review provides a comprehensive analysis of the existing literature across multiple countries, identifying 57 relevant studies that present strategies for educating, training, and assessing digital literacy in pharmacy students and trainees. The educational approaches presented in these studies focused on various areas, including theoretical understanding of HI, familiarity with digital technologies, patient-centered care through digital technologies, and interprofessional collaboration using digital technologies. A range of delivery and assessment approaches were also identified, with blended approaches being the most utilized. The studies showed a diverse range of approaches to course development. Our analysis suggests that external engagement with practitioners and/or industrial partners can enhance the successful implementation of digital literacy education in undergraduate pharmacy programs. These collaborations provide access to real-world platforms, DHTs, and the opportunity to interact with industry professionals.34 However, there is potential for conflict of interest with third parties which can undermine the integrity and quality of the educational course. Following policies and guidelines to ensure sustainable and ethical collaborations is therefore vital.75–77 Our review also identified foundational skills in pharmacy undergraduate education, including fundamental concepts related to HI, such as terminology, basic knowledge, ethical principles, health data management, and communication skills. Exposure to commonly used pharmacy technologies, such as EHRs, mHealth, and telehealth was considered important. Additionally, advanced skills encompassed domains like interoperability standards, biomedical informatics, information system design and modeling, and emerging technologies (eg, digital medicine, genomics, AI, and robotics).
The findings align with previous research in medical and nursing education, which emphasizes a growing recognition of the digital health role.78–80 Healthcare professionals, including pharmacists, are expected to use DHT in a patient-centered manner. This involves learning how to effectively engage patients in the use of technologies and as a communication tool.81 In a comprehensive study by Zainal et al.78 several essential components of a clinical informatics (CI) curriculum for medical schools were identified, including CI utilization in clinical practice, ethical implications, CI-key concepts, and digital health. They also reported similar modes of delivery, including lectures, problem-based learning (PBL), and e-learning.78 Notably, they emphasized the importance of incorporating ethical principles when utilizing machine learning and AI tools, as well as understanding the potential limitations and biases of technology. This is essential to ensure a balance between technological advancements and compassionate patient care that ensures ethical and humanistic principles are upheld. Harerimana et al79 noted a lack of certainty around how to integrate HI concepts into undergraduate nursing education in Australia. However, utilizing DHTs within nursing education was reported in various studies, as they play a pivotal role in supporting teaching and learning using different formats, including face-to-face, online, and blended learning, during both classroom and simulation sessions.79
Our study assessed the alignment of learning objectives with the HEE capability framework for digital literacy in pharmacy education. While the framework provided useful guidance, we identified certain gaps in the current teaching practices. Specifically, there was limited emphasis on the development of skills related to creating and innovating digital health solutions to improve patient care and outcomes, as well as conducting research using DHT. Additionally, the importance of interdisciplinary collaboration in digital literacy education and the need to create appropriate digital identities in professional and personal contexts were often overlooked. These findings indicate potential areas for future development and improvement in digital literacy education within undergraduate pharmacy programs.
Different approaches to introducing HI into the educational curriculum were explored in our study. While there exists no consensus on the best approach, O’Connor and LaRue proposed a spiral approach of gradually and iteratively introducing new concepts to students.82 Further work is needed to explore the effectiveness of this.82 Hare et al,83 meanwhile suggested a tiered informatics curriculum design for medical education, which included different levels of depth within courses that students could opt into, ranging from shorter crash courses to longer fellowships based on their preferences. Successful implementation, however, depends on careful planning, resource allocation, and collaboration with healthcare institutions.83
The presence of educators with informatics expertise is critical for effective digital literacy education. Our study findings indicate that a lack of skilled informatics professionals has been identified as a significant barrier in the development of digital literacy courses.20,25,70 To address this challenge, informatics experts who are actively engaged in the field should be involved as educators, to enhance digital literacy education for aspiring healthcare professionals. It will therefore be necessary to provide support and resources to these educators, for example, dedicated time for education and establishing clear pathways for their continued informatics learning.
Recommendation
The review provides recommendations for incorporating digital literacy education into undergraduate pharmacy programs. Table 3 presents specific recommendations to guide pharmacy schools in curricula development. These recommendations include emphasizing foundational knowledge, educators keeping up with emerging technology, recognizing the fast-paced nature of DHT, using standardized guidelines for reporting educational interventions to allow better sharing of good practice and harnessing partnerships with stakeholders to co-develop digital literacy education.
Table 3.
Key recommendation | Specific recommendations |
---|---|
|
|
|
|
|
|
|
|
Limitations
Although a comprehensive search of major bibliographic databases was conducted, it is possible that some relevant sources, such as unpublished studies or articles in the gray literature, may have been missed. Additionally, some reviewed studies lacked detailed descriptions of digital health courses design and implementation, including learning objectives, development processes, and assessment approaches, which limit the depth of findings. It is also important to note that interrater reliability between reviewers was not assessed, which might have led to some discrepancies in how the included studies are interpreted and evaluated. Finally, as the majority of the studies were conducted in the United States and primarily in English, the applicability of specific learning approaches in an international context may differ due to differences in healthcare and educational systems.
Conclusions
In summary, this scoping review highlights the importance of integrating digital literacy education into undergraduate pharmacy curricula. It identifies learning objectives, skills, competencies, teaching, and assessment methods to achieve this goal. Given the rapid advancement of DHT, healthcare professionals and educators must remain up to date with new developments to provide optimal patient care and appropriate education. We have also identified several recommendations, including prioritizing foundational knowledge, identifying advanced digital literacy competencies, keeping pace with emerging technology, using standardized guidelines for reporting interventions, and partnering with stakeholders to facilitate the development of digital literacy courses. Despite limitations, this study offers a valuable overview of the current state of digital literacy education in undergraduate pharmacy programs and practical suggestions for enhancing digital literacy education to equip future pharmacists for the changing landscape of DHT.
Supplementary Material
Contributor Information
Mashael Alowais, School of Pharmacy, Newcastle University, Newcastle upon Tyne, NE1 7RU, United Kingdom; Department of Pharmacy Practice, Unaizah College of Pharmacy, Qassim University, Unaizah, 51911, Saudi Arabia.
Georgina Rudd, School of Pharmacy, Newcastle University, Newcastle upon Tyne, NE1 7RU, United Kingdom.
Victoria Besa, School of Pharmacy, Newcastle University, Newcastle upon Tyne, NE1 7RU, United Kingdom.
Hamde Nazar, School of Pharmacy, Newcastle University, Newcastle upon Tyne, NE1 7RU, United Kingdom.
Tejal Shah, School of Computing, Newcastle University, Newcastle upon Tyne, NE4 5TG, United Kingdom.
Clare Tolley, School of Pharmacy, Newcastle University, Newcastle upon Tyne, NE1 7RU, United Kingdom.
Author contributions
M.A., C.T., and H.N. conceived the scoping review. M.A. conducted the literature search and screened the studies with G.R. and V.B. M.A. and G.R. extracted the data. M.A. led the writing of the manuscript, with C.T., H.N., and TS providing feedback on subsequent drafts. All authors gave their approval for the final version to be published.
Supplementary material
Supplementary material is available at Journal of the American Medical Informatics Association online.
Funding
This work was supported by the Saudi Arabian Cultural Bureau in the United Kingdom and Qassim University in Saudi Arabia. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Conflicts of interest
None declared.
References
- 1. World Health Organization. Global Diffusion of eHealth: Making Universal Health Coverage Achievable: Report of the Third Global Survey on eHealth. 2017. Accessed July 27, 2023. https://www.who.int/publications/i/item/9789241511780
- 2. Park Y. Emerging new era of mobile health technologies. Healthc Inform Res. 2016;22(4):253-254. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. World Health Organization. Recommendations on Digital Interventions for Health System Strengthening: WHO Guideline. World Health Organization; 2019:150. [PubMed] [Google Scholar]
- 4. Keasberry J, Scott IA, Sullivan C, et al. Going digital: a narrative overview of the clinical and organisational impacts of eHealth technologies in hospital practice. Aust Health Rev. 2017;41(6):646-664. [DOI] [PubMed] [Google Scholar]
- 5. Chen Y, Banerjee A.. Improving the digital health of the workforce in the COVID-19 context: an opportunity to future-proof medical training. Future Healthc J. 2020;7(3):189-192. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Monaghesh E, Hajizadeh A.. The role of telehealth during COVID-19 outbreak: a systematic review based on current evidence. BMC Public Health. 2020;20(1):1193. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Silva R, de Araújo DCSA, dos Santos Menezes PW, et al. Digital pharmacists: the new wave in pharmacy practice and education. Int J Clin Pharm. 2022;44(3):775-780. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Jamie K. New technologies in British pharmacy practice. In: Wasen K, ed. Emerging Health Technology. Springer Verlag; 2013:53-74. [Google Scholar]
- 9. Vatanka P, Lofton JC.. Re-envisioning the pharmacist's role in the era of digital health—CPhA’s inaugural digital health conference. J Contemp Pharm Pract. 2020;67(2):23-30. [Google Scholar]
- 10. Topol E. The Topol Review. Preparing the Healthcare Workforce to Deliver the Digital Future. An independent report on behalf of the Secretary of State for Health and Social Care; 2019:1-48. [Google Scholar]
- 11. NITA United States Department of Commerce. Exploring the Digital Nation: America’s Emerging Online Experience. 2013. Accessed June 16, 2023. https://www.ntia.doc.gov/files/ntia/publications/exploring_the_digital_nation_-_americas_emerging_online_experience.pdf.
- 12. The Accreditation Council for Pharmacy Education. Accreditation Standards and Key Elements for the Professional Program in Pharmacy Leading to the Doctor of Pharmacy Degree. 2015. Accessed June 16, 2023. https://www.acpe-accredit.org/pdf/Standar#ds2016FINAL.pdf.
- 13. Health Education England. Digital Capabilities for the Pharmacy Workforce. 2021. Accessed March 4, 2022. https://www.hee.nhs.uk/our-work/digital-literacy/profession-service-specific-digital-capabilities.
- 14. Mantel-Teeuwisse AK, Meilianti S, Khatri B, et al. Digital health in pharmacy education: preparedness and responsiveness of pharmacy programmes. Educ Sci. 2021;11(6):296. [Google Scholar]
- 15. Peters M, Godfrey C, McInerney P, et al. Methodology for JBI scoping reviews. JBI Database System Rev Implement Rep. 2015;13(2):1-2. [DOI] [PubMed] [Google Scholar]
- 16. Masic I. Five periods in development of medical informatics. Acta Inform Med. 2014;22(1):44-48. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Biggs J. Enhancing teaching through constructive alignment. High Educ. 1996;32(3):347-364. [Google Scholar]
- 18. Rodis J, Aungst TD, Brown NV, et al. Enhancing pharmacy student learning and perceptions of medical apps. JMIR Mhealth Uhealth. 2016;4(2):e55. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Breeden EA, Clauson KA.. Development and implementation of a multitiered health informatics curriculum in a college of pharmacy. J Am Med Inform Assoc. 2016;23(4):844-847. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. Estes K, Gilliam E, Knapfel S, et al. Discovering eHealth technology: an innovative interprofessional graduate student learning experience. Stud Health Technol Inform. 2016;225:242-246. [PubMed] [Google Scholar]
- 21. Hincapie AL, Cutler TW, Fingado AR.. Incorporating health information technology and pharmacy informatics in a pharmacy professional didactic curriculum -with a team-based learning approach. Am J Pharm Educ. 2016;80(6):1-11. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Leibfried M, Pisano M.. The utilization of a simulated electronic medical record in an introductory pharmacy practice experience. Curr Pharm Teach Learn. 2016;8(4):458-462. [Google Scholar]
- 23. Miranda AC, Serag-Bolos ES, Aungst TD, et al. A mobile health technology workshop to evaluate available technologies and their potential use in pharmacy practice. BMJ Simul Technol Enhanc Learn. 2016;2(1):23-26. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24. Nemec EC, Pawola JA, Shah B.. Online resources for informatics education and training. Am J Health Syst Pharm. 2016;73(15):1131-1134. [DOI] [PubMed] [Google Scholar]
- 25. Fox BI, Flynn A, Clauson KA, et al. An approach for all in pharmacy informatics education. Am J Pharm Educ. 2017;81(2):38. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26. Flynn A, Fox BI, Clauson KA, et al. An approach for some in advanced pharmacy informatics education. Am J Pharm Educ. 2017;81(9):6241. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27. Fox BI, Umphress DA, Hollingsworth JC.. Development and delivery of an interdisciplinary course in mobile health (mHealth). Curr Pharm Teach Learn. 2017;9(4):585-594. [DOI] [PubMed] [Google Scholar]
- 28. Steckler TJ, Brownlee MJ, Urick BY, et al. Pharmacy informatics: a call to action for educators, administrators, and residency directors. Curr Pharm Teach Learn. 2017;9(5):746-749. [DOI] [PubMed] [Google Scholar]
- 29. Rocchi M, Bishop L, Cooper J, et al. The evolution of a national electronic educational resource for informatics [Abstract]. Am J Pharm Educ. 2018;82(5):483. [Google Scholar]
- 30. Perkins S, Kelly K.. Implementation of electronic health records in the first professional year [Abstract]. Am J Pharm Educ. 2018;82(5):484. [Google Scholar]
- 31. Logan L, Darley A, Seagraves G.. Impact of a health information technology skills lab on second year student pharmacists [Abstract]. Am J Pharm Educ. 2018;82(5):514. [Google Scholar]
- 32. Wasynczorguk J, Sheehan AH.. Incorporating a teaching electronic medical record to answer patient-specific drug information questions in the didactic setting [Abstract]. Am J Pharm Educ. 2018;82(5):579. [Google Scholar]
- 33. Coons JC, Kobulinsky L, Farkas D, et al. Virtual electronic health record technology with simulation-based learning in an acute care pharmacotherapy course. Pharmacy. 2018;6(4):123. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34. Darley A, Logan LD.. Pharmacy student exposure to health information technology through college and industry collaborations. Am J Health Syst Pharm. 2019;76(23):1908-1911. [DOI] [PubMed] [Google Scholar]
- 35. Smith JN, Scholtz JM.. Impact of a simulated electronic health record on pharmacy students’ perceptions of preparedness for clinical practice. Curr Pharm Teach Learn. 2018;10(12):1624-1630. [DOI] [PubMed] [Google Scholar]
- 36. VanLangen K, Wellman G.. Trends in electronic health record usage among US colleges of pharmacy. Curr Pharm Teach Learn. 2018;10(5):566-570. [DOI] [PubMed] [Google Scholar]
- 37. Skelley JW, Wulz JL, Thibodeaux AM.. Implementation of an electronic medical record simulation activity aligned with the pharmacist patient care process in an ambulatory care elective course. Pharm Educ. 2018;18(00):91-98. [Google Scholar]
- 38. Gibson CM, Kwon HI, Tatachar A.. Impact of a low-cost simulated electronic medical record on perceptions of APPE readiness. Curr Pharm Teach Learn. 2019;11(7):736-741. [DOI] [PubMed] [Google Scholar]
- 39. Martin LG, Warholak TL, Hincapie AL, et al. Health informatics competencies for pharmacists in training. Am J Pharm Educ. 2019;83(2):6512. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40. Pontefract SK, Wilson K.. Using electronic patient records: defining learning outcomes for undergraduate education. BMC Med Educ. 2019;19(1):30. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41. Carr-Lopez SM, Strohecker L, Mai Y, et al. A module in medication safety involving remote access and review of veterans’ health records. Am J Pharm Educ. 2020;84(12):7664. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42. Ives AL, Tucker SR, Trovato JA.. Using electronic health record technology to teach inpatient medication order verification to pharmacy students. Am J Pharm Educ. 2020;84(8):1071-1076. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43. O’Connor M, Badowski M, Michienzi S, et al. Exploration of telemedicine in the classroom setting. J Am College Clin Pharm. 2020;3(1):338-339. [Google Scholar]
- 44. Vlashyn OO, Adeoye-Olatunde OA, Illingworth Plake KS, et al. Pharmacy students’ perspectives on the initial implementation of a teaching electronic medical record: results from a mixed-methods assessment. BMC Med Educ. 2020;20(1):187. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45. VanLangen KM, Elder KG, Young M, et al. Academic electronic health records as a vehicle to augment the assessment of patient care skills in the didactic pharmacy curriculum. Curr Pharm Teach Learn. 2020;12(9):1056-1061. [DOI] [PubMed] [Google Scholar]
- 46. Cook K, Cochran G, Gali H, et al. Pharmacy students' readiness to use the electronic health record: A tale of two institutions. Curr Pharm Teach Learn. 2021;13(4):327-332. [DOI] [PubMed] [Google Scholar]
- 47. Cook K, Elder KG, Richter SK, et al. Electronic health records in pharmacy skills-based curricula. Am J Pharm Educ. 2021;85(7):8453. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 48. Neumann-Podczaska A, Seostianin M, Madejczyk K, et al. An experimental education project for consultations of older adults during the pandemic and healthcare lockdown. Healthcare. 2021;9(4):425. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49. Perkins S, Kelly K, Bowers R.. Implementation and justification of a curriculum-wide electronic health record platform. Curr Pharm Teach Learn. 2021;13(9):1194-1199. [DOI] [PubMed] [Google Scholar]
- 50. Reynolds P, Trinkley K, Gleason S.. Implementation of a learning electronic health record in an online, distance-based capstone pharmacotherapy course. J Am College Clin Pharm. 2021;4(12):1664. [Google Scholar]
- 51. Wasynczuk J, Sheehan AH.. Implementation of a teaching electronic medical record within didactic instruction using a drug information question assignment. Pharmacy. 2021;9(1):35. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 52. Ngo PQ, Bajaj SK, Hincapie AL, et al. Characterization of doctor of pharmacy/health informatics dual degrees in the United States. Curr Pharm Teach Learn. 2022;14(4):415-424. [DOI] [PubMed] [Google Scholar]
- 53. Seifert CF, Veronin MA, Diane Kretschmer T, et al. The training of a telepharmacist: Addressing the needs of rural West Texas. Am J Pharm Educ. 2004;68(3):60-69. [Google Scholar]
- 54. Brown MC. Internet-based medical chart for documentation and evaluation of simulated patient care activities. Am J Pharm Educ. 2005;69(2):30-212. [Google Scholar]
- 55. Brown MC, Kotlyar M, Conway JM, et al. Integration of an Internet-based medical chart into a pharmacotherapy lecture series. Am J Pharm Educ. 2007;71(3):53. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 56. Zagar M. Preparing PharmD students to participate in Medicare Part D education and enrollment. Am J Pharm Educ. 2007;71(4):77. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 57. Cutler TW, Stebbins MR, Lai E, et al. Problem-based learning using the online Medicare Part D Plan Finder tool. Am J Pharm Educ. 2008;72(3):47. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 58. Fox BI, Karcher RB, Flynn A, et al. Pharmacy informatics syllabi in doctor of pharmacy programs in the US. Am J Pharm Educ. 2008;72(4):89. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 59. Fuji KT, Galt KA.. Case study of a health informatics course for pharmacy students [Abstract]. Am J Pharm Educ. 2010;74(5):121.21088726 [Google Scholar]
- 60. Frenzel JE. Using electronic medical records to teach patient-centered care. Am J Pharm Educ. 2010;74(4):71. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 61. Gardner S, Stowe C.. Implementation of an interprofessional seminar series on an academic health center campus [Abstract]. Am J Pharm Educ. 2011;75(5):91. [Google Scholar]
- 62. Fox BI, Flynn AJ, Fortier CR, et al. Knowledge, skills, and resources for pharmacy informatics education. Am J Pharm Educ. 2011;75(5):93. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 63. Kirwin JL, DiVall MV, Guerra C, et al. A simulated hospital pharmacy module using an electronic medical record in a pharmaceutical care skills laboratory course. Am J Pharm Educ. 2013;77(3):62. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 64. Miranda AC, Serag-Bolos ES, Gelot S, et al. Integrating electronic medical records (EMR) into a college of pharmacy curriculum [Abstract]. Am J Pharm Educ. 2014;78(5):67.24761028 [Google Scholar]
- 65. Leibfried M, Pisano M.. Evaluation of a simulated electronic medical record (SimEMR) into a introductory pharmacy practice experience [Abstract]. Am J Pharm Educ. 2014;78(5):20.24558288 [Google Scholar]
- 66. Cutler TW, Fingado AR, Lim S, et al. Statewide impact of an online-based pharmacy informatics curriculum [Abstract]. Am J Pharm Educ. 2014;78(5):77.24850939 [Google Scholar]
- 67. Manning D, Zarcone J, Welch A.. The development of a skill-based course in pharmacy informatics [Abstract]. Am J Pharm Educ. 2014;78(5):82.24850944 [Google Scholar]
- 68. Aungst TD. Integrating mHealth and mobile technology education into the pharmacy curriculum. Am J Pharm Educ. 2014;78(1):19. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 69. Fuji KT, Galt KA.. An online health informatics elective course for doctor of pharmacy students. Am J Pharm Educ. 2015;79(3):41. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 70. Pantazi SV, Pantazi F, Daly K.. Clinical informatics in undergraduate teaching of health informatics. Stud Health Technol Inform. 2011;164:58-63. [PubMed] [Google Scholar]
- 71. Mihalas GI. Health informatics education for pharmacy students in Romania. Stud Health Technol Inform. 1997;43(Pt B):702-706. [PubMed] [Google Scholar]
- 72. Ab Rahman AF, Ibrahim MIM, Bahari MB, et al. Design and evaluation of the pharmacoinformatics course at a pharmacy school in Malaysia. Drug Information J. 2002;36(4):783-789. [Google Scholar]
- 73. Toh TW, Chui WK, Yap KYL.. Development of a virtual patient record mobile app for pharmacy practice education. Arch Pharma Pract. 2014;5(2):66-71. [Google Scholar]
- 74. Leidl DM, Ritchie L, Moslemi N.. Blended learning in undergraduate nursing education—A scoping review. Nurse Educ Today. 2020;86:104318. [DOI] [PubMed] [Google Scholar]
- 75. Rybnicek R, Königsgruber R.. What makes industry–university collaboration succeed? A systematic review of the literature. J Bus Econ. 2019;89(2):221-250. [Google Scholar]
- 76. Awasthy R, Flint S, Sankarnarayana R, et al. A framework to improve university–industry collaboration. J Ind Univ Collab. 2020;2(1):49-62. [Google Scholar]
- 77. Allen T, Donde N, Hofstädter-Thalmann E, et al. Framework for industry engagement and quality principles for industry-provided medical education in Europe. J Eur CME. 2017;6(1):1348876. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 78. Zainal H, Tan JK, Xiaohui X, et al. Clinical informatics training in medical school education curricula: a scoping review. J Am Med Inform Assoc. 2022;30(3):604-616. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 79. Harerimana A, Wicking K, Biedermann N, et al. Nursing informatics in undergraduate nursing education in Australia before COVID-19: A scoping review. Collegian. 2022;29(4):527-539. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 80. Tudor Car L, Kyaw BM, Nannan Panday RS, et al. Digital health training programs for medical students: scoping review. JMIR Med Educ. 2021;7(3):e28275. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 81. Clark M, Clark T, Bhatti A, et al. The rise of digital health and potential implications for pharmacy practice. J Contemp Pharm Pract. 2017;64(1):32-40. [Google Scholar]
- 82. O’Connor S, LaRue E.. Integrating informatics into undergraduate nursing education: a case study using a spiral learning approach. Nurse Educ Pract. 2021;50:102934. [DOI] [PubMed] [Google Scholar]
- 83. Hare AJ, Soegaard Ballester JM, Gabriel PE, et al. Training digital natives to transform healthcare: a 5-tiered approach for integrating clinical informatics into undergraduate medical education. J Am Med Inform Assoc. 2022;30(1):139-143. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 84. Hoffmann TC, Glasziou PP, Boutron I, et al. Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. BMJ. 2014;348:g1687. [DOI] [PubMed] [Google Scholar]
- 85. Phillips AC, Lewis LK, McEvoy MP, et al. Development and validation of the guideline for reporting evidence-based practice educational interventions and teaching (GREET). BMC Med Educ. 2016;16(1):237. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.