Real-time use of the iPad by third-year medical students for clinical decision support and learning: a mixed methods study

Abstract

Purpose

Despite widespread use of mobile technology in medical education, medical students’ use of mobile technology for clinical decision support and learning is not well understood. Three key questions were explored in this extensive mixed methods study: 1) how medical students used mobile technology in the care of patients, 2) the mobile applications (apps) used and 3) how expertise and time spent changed overtime.

Methods

This year-long (July 2012–June 2013) mixed methods study explored the use of the iPad, using four data collection instruments: 1) beginning and end-of-year questionnaires, 2) iPad usage logs, 3) weekly rounding observations, and 4) weekly medical student interviews. Descriptive statistics were generated for the questionnaires and apps reported in the usage logs. The iPad usage logs, observation logs, and weekly interviews were analyzed via inductive thematic analysis.

Results

Students predominantly used mobile technology to obtain real-time patient data via the electronic health record (EHR), to access medical knowledge resources for learning, and to inform patient care. The top four apps used were Epocrates^®, PDF Expert^®, VisualDx^®, and Micromedex^®. The majority of students indicated that their use (71%) and expertise (75%) using mobile technology grew overtime.

Conclusions

This mixed methods study provides substantial evidence that medical students used mobile technology for clinical decision support and learning. Integrating its use into the medical student's daily workflow was essential for achieving these outcomes. Developing expertise in using mobile technology and various apps was critical for effective and efficient support of real-time clinical decisions.

There have been several survey-based studies of medical students’ use of iPads (1–4). More extensive research is needed to fully understand when and for what purposes mobile technologies are used to assist in evidence-based, real-time decision support in the care of patients. During a year-long mixed methods study, we sought to deepen our understanding of the use of mobile technologies with a focus on third-year medical students on an Internal Medicine clerkship.

Introduction

Over the past 10 years, small handheld Personal Digital Assistants (PDAs) were replaced by the popular smartphone but now by devices with a much larger landscape – tablet computers, such as the Apple iPad^® and Samsung Galaxy Tablet^®. Mobile technology has rapidly become the norm in the preclinical years of medical education because it affords flexible, wireless, and mobile access to endless amounts of medical content and knowledge (5). Since 2009, a growing number of medical schools have implemented mobile device medical curriculums (6). Many of these medical schools purchase devices for students and pre-load them with electronic textbooks and the medical school curriculum. Other medical schools simply require or recommend mobile devices to students for purchase (7).

The ways in which mobile technologies are being used in clinical practice is also expanding. At least 75% of US physicians are currently using iOS mobile devices and technology (8). Healthcare organizations are now integrating their electronic health records (EHRs) with mobile devices, such as the iPad. The use of iPads has been embraced by Chief Medical Information Officers (CMIO) in many hospitals across the country and has created a more efficient workflow for practitioners (9). This expansion of access has afforded ready availability of the EHR at the bedside for patient education, electronic prescription writing, laboratory, and x-ray order as well as access to evidence-based practice tools for patient care (10). With thousands of medical apps available at the tap of a finger on the screen, the iPad has enabled even more opportunities for evidence-based decision support in real-time at the point of care.

Research related to the use of mobile technology for patient care is not new (11). While initial reports of iPad use by medical professionals are promising (12), there are no in-depth research studies (i.e., beyond questionnaires) that explore the use of the iPad in clerkships by third-year medical students. The purpose of this year-long mixed methods study was to further understand medical students’ use of the iPad during their Internal Medicine clerkship. Specifically, we sought to answer the following questions:

1. In what ways did the students use mobile technology for learning and clinical decision support?

2. What apps did the students use in the care of patients?

3. Did the amount of time spent and the students’ expertise in using mobile technology grow overtime?

Methods

Participants and iPads

Following approval by the University of Georgia Institutional Review Board, we invited 37 third-year medical students from the partnership medical school campus (GRU/UGA Medical Partnership) who rotated to a local community hospital (St. Mary's Hospital) to participate in the study. All students were issued a Wi-Fi only, third-generation iPad with 64 Gb of storage. Management of the iPads was enabled by a mobile device management (MDM) program that required passcodes, provided lost device tracking and remote device erasure capabilities. All students were required to sign an iPad Agreement attesting to the terms and conditions when using the iPad to be HIPAA compliant. Students were free to install additional applications on their iPads purchased with their own funds.

Each device was pre-loaded with a variety of applications (e.g., PDF Expert^®, VisualDx^®, Penultimate^®), as well as bookmark links to PubMed^®, the medical school library and MedlinePlus^®. All students were required to complete a minimum of three hours of iPad training. Training by members of the research team involved extensive review of the technology as well as case-based scenarios to practice use of the apps. Broad topics covered during the training included: 1) basic use of the iPad, 2) utilization of productivity apps, and 3) how to access appropriate medical knowledge resources and apps for clinical decision support.

Research design, procedures, and analysis

This year-long mixed methods study involved quantitative and qualitative data collection (13) from 37 third-year medical students. Four data collection instruments were used: 1) beginning and end-of-year questionnaires, 2) iPad usage logs, 3) weekly rounding observations, and 4) weekly semi-structured medical student interviews. The team of five researchers collected data over a 12-month period (July 2012–June 2013).

We administered a baseline questionnaire prior to beginning the Internal Medicine clerkship which included demographic questions and eight additional questions assessing the students’ past and present use of mobile technology and Apple^® computers. A second questionnaire was administered at the completion of the 48 weeks of clerkships and included demographics questions and additional questions that related to their experience with the iPad over the past year.

Students completed weekly iPad usage logs during their 8-week Internal Medicine clerkship. Students logged weekly the types and amount of time used on medical resources and apps on the iPad in the care of patients. The time used for the apps ranged from 1 min to a few hours depending on the application. Logs were submitted weekly to the research team.

We conducted weekly, 1-hour observations at the hospital as students rounded with their preceptors. Researchers were able to observe directly how the iPad was being utilized in real-time and recorded notes on a structured observation log. Weekly semi-structured, one-on-one interviews (face-to-face, phone or Facetime^®) were conducted with each medical student during the Internal Medicine clerkship. The end-of-the week interviews with students ranged from 10 to 30 min. Interviews were recorded and transcribed confidentially.

Descriptive statistics were generated for the questionnaires as well as for the apps and resources reported in the iPad usage and observation logs. Inductive analysis (14) was used by a member of the research team to code the open-ended responses on the iPad usage and observation logs, as well as the weekly medical student interviews. Following a robust data analysis protocol using Microsoft Word^® (15) by a member of the research team, patterns and themes were identified across participants and used to inform the results.

Results

Students’ use of mobile technology in patient care

All of the students indicated they used the iPad in a variety of ways to assist in the care of patients. These activities are intimately linked; therefore, the results are reported together in two major areas: 1) clinical decision support and 2) student learning and productivity.

Clinical decision support

The students reported using the iPad at all stages of patient care: before, during, and after patient encounters. Two primary uses were indicated in the data: obtaining real-time patient data via the EHR and finding additional information for clinical decision support. Most students, as reported in all four sources of data, indicated daily use of the EHR to obtain real-time patient data. This use is well summarized in these quotes from weekly interviews when talking about the use of the iPad on rounds:

I feel like when we're doing rounds, to have access to the medical record if there's a lab value or something that I forgot to check or needed to pull up, … I can use the iPad, what's useful for that is I can use it to look up stuff which is really helpful on rounds when I have to go present my patients … I've used it a lot for accessing the medical records.

These quotes as well as the other data indicate the value of the iPad for enabling real-time access to the EHR to assist with clinical decision support.

Identifying medical knowledge resources for clinical decision support was the second most frequent way that students reported using the iPad. Various medical knowledge resources were used including library resources and a multitude of iPad apps. Library resources were the fifth highest used resource as reported by 65% of the students on the end-of-year questionnaire. As a student detailed in an interview:

I mainly used it to get access to the journals and I didn't need to get it through the school [physically]. And so what was very helpful from the library was that by logging on, I could get the pdf [of the article].

More details about use of apps on the iPad are reported later in the article; the data from the interviews indicated that easy and ready access to information was an important use of the iPad by the students in their Internal Medicine clerkship.

Student learning and productivity

While patient care accounted for the vast majority of the use of the iPad, students also indicated using the iPad for personal learning and productivity throughout the day. The iPad usage and observation logs, and end-of-year questionnaire data indicated that the highest productivity use of the iPad were email, note taking and word processing (e.g., QuickOffice ^®, Notepad ^®). The high value of easy access to productivity tools was also explained during weekly interviews, for example:

I had been using it primarily as a study tool with my PDFs on it … I was still doing research through First Consult ^®, DynaMed ^® and Epocrates ^® but now … I'm taking patient histories on my wireless keyboard on the iPad and I'm presenting the patient histories off of the iPad to the preceptor, as well as just like doing research on the go rather than sitting at home using it.

The students also provided evidence about using the iPad for learning. The most commonly reported resources included question banks (e.g., USMLE, Kaplan), medical knowledge resources, and documents related to the medical school curriculum. As students explained during weekly interviews and in the end-of-year questionnaires:

I use my iPad as an eBook reader for my textbooks.

The iPad was VERY useful for doing practice questions and reading while studying.

I use it for studying, I read my textbooks on there and I do review questions, because I can carry it around and have an infinite number of review questions, and my textbooks, anywhere in the hospital.

As the data indicated, the students reported multiple uses of their iPads for personal learning and productivity.

Resources and apps used

The third-year medical students used a multitude of apps for clinical decision support in the care of patients. For example, the top apps reported in the end-of-year questionnaires are summarized in Fig. 1.

Fig. 1.

Top apps used on the iPad end-of-year questionnaire.

The top three apps reported as widely used by the students on the end-of-year questionnaire included Epocrates ^®, PDFExpert ^®, and VisualDx ^®. The data is corroborated by the usage and observation logs. Top apps recorded in the iPad usage logs included Micromedex ^®, DynaMed ^®, and Epocrates ^®. Twenty-three of the 37 students also reported loading additional apps, with the top three being: 1) First Consult ^®, 2) DrawMD ^®, and 3) USMLE ^® World Q Bank. Student-loaded apps reported by more than one student are listed in Table 1.

Table 1.

Additional apps loaded by more than one medical student onto the iPad^®

	AHRQ ePSS		Pocket Card series
	CHM Viewer		Qx Calculate
	Diagnosaurus		Shoulder Decide
	DrawMD		Skyscape
	Epocrates		Spine Decide
	Foot Decide		USMLE World
	Knee Decide		VisualDx
	Medscape		Visible Body Anatomy Atlas

The overall usefulness of the apps and iPad for clinical decision support is best conveyed in open-ended responses to the end-of-year questionnaire as well as in the weekly interviews. As described by these quotes from students:

Initially I was using Micromedex ^® a lot to review mechanisms or look up side effects of drugs … I used VisualDx a lot when looking at rashes; I used it a couple of times with patients to see if the images I had looked similar to what their rash initially looked like …. As time went by I learned that Epocrates was a great tool for looking up recommended treatments and quick facts about diseases; even helped my preceptors look up recommended dose of medications or alternative treatments for certain situations.

The iPad made it easy to look up a patient's – or physician's – question without leaving the room, or within minutes of leaving. The accessibility of information meant we were all more informed more quickly, and that our knowledge of labs, cultures, and studies could always be up to date at the time of rounding …

As illustrated in these quotes, not only were students using the iPad to assist their own learning, they used them to assist with patient education and to provide real-time access to information for their preceptors.

Integrating the iPad into the daily workflow

In the end-of-year questionnaire, the majority of the third-year medical students (71%; n=20) reported that the amount of time they spent using the iPad for clinical decision support grew overtime (see Fig. 2). Of those who indicated strongly agree or agree, the majority (58%; n=11) reported using their iPad many times a day. The 18% of students who disagreed provided insights into why their use did not grow, ranging from the size and weight of the iPad to the use of other electronic devices (e.g., desktop computers, iPhone^®). Students also reported that the fast pace of rounds or lack of access to a stable and consistent wireless network impacted their use.

Fig. 2.

Amount of time spent and perceived expertise using the iPad^®.

The majority of the third-year medical students (75%; n=21) also reported that their expertise in using the iPad for clinical decision support grew overtime (see Fig. 2). All students (n=28) indicated that their level of expertise in using the iPad at the end-of-year was expert or intermediary.

Those who indicated strongly agree or agree provided insights as to why their expertise grew over time. One major theme was that the students’ comfort level with the use and navigation increased over time, contributing to the growth in expertise using the iPad. As stated by one student: ‘I did become much more comfortable using a number of medical apps; I also felt more comfortable using my iPad with my preceptor around’. Secondly, students indicated that they learned how to use specific apps to meet specific needs. As described by a student:

I learned to use the apps that were appropriate for each case more effectively. Initially I would be just searching for information on Google ^® or different apps. As time went by I knew which apps would have the information that I was looking for.

Expertise in using the iPad allowed students to use the iPad in various clinical settings beyond Internal Medicine. As one student reported: ‘By the end of the year I was able to use the iPad in any clinical setting. Initially started as a personal reference tool, then a presentation tool with peers and preceptors, then a patient education tool’. The students not only became more comfortable with use, some also started thinking about the use of the iPad as just what you do: ‘Everyday use of the iPad is a habit now, and I do not have to think about where to find information and how to use it’.

The 11% (n=3) who disagreed with their expertise growing over time stated they did not choose to use the iPad (e.g., worried about losing it, like paper and books, don't have time to learn how to use it). While it is important to recognize that not all students grew in use and/or expertise with the iPad, it is noteworthy that the majority of the students did grow in both areas. This is promising for continued use of mobile technologies for real-time clinical decision support.

Discussion and conclusions

This year-long, mixed methods study provides substantial evidence that medical students used mobile technology to support their clinical decision support and learning in the care of patients. Although numerous studies have been conducted using surveys as their primary methodology, this study significantly extends the data collected by studying the use of the iPad using weekly usage and observation logs, interviews and questionnaires. This study supports the consistent conclusions of multiple studies 1–4, (16), that tablet computers are being used to enhance patient care and learning in clinical contexts for students and residents. This study finds that students’ primary uses were to obtain real-time patient data via the EHR during rounds and accessing other medical knowledge resources (e.g., apps, library databases, e-Textbooks) to support clinical decisions and personal learning. In addition, students used mobile technology to access productivity apps for note taking during and after rounds, e-mail, and studying. These uses were generally consistent with the findings of a national survey (16) of third-year US medical students that found the highest uses for medical reference apps (47%), USMLE preparation (49%), and e-Books (43%), but relatively lower use for EHR access (23%). In contrast yet consistent with Ellaway's (17) findings that student choices are shaped by the learning context, evidence from this study indicates that the EHR use was much higher because it was conducted in a hospital setting and preceptors also were given iPads. One supposition is that the tablet computers’ ability to display large, high-quality images enables a clearer reading of the EHR, radiographic images, as well as articles and e-textbooks than do other mobile devices (2, 18) providing the necessary affordances to use mobile technology for real-time clinical decision support.

The most used apps were ones for clinical decision support, learning and productivity. This study found that the primary applications that students used for clinical decision support were Epocrates ^®, PDF Expert ^®, Micromedex ^®, DynaMed ^®, and VisualDx ^® in addition to more traditional library resources. The students indicated that the use of the apps enabled easy access to information for clinical decision support and their learning by connecting knowledge to real-time needs. As found in the Boruff and Storie study (2), Epocrates was a favorite resource because it provided quick facts on diseases, drugs and treatments.

Integrating mobile technology into the medical student's daily workflow was essential for learning and clinical decision support. Consistent with Boruff and Storie's (2) finding that 70% of third- and fourth-year medical students used mobile devices at least once a day, the majority of the students in this study reported using their iPad many times a day as it became a seamless part of their work. The primary reasons indicated for the students’ increased usage were that practice using the iPad shortened access time, increased their comfort level using it in patient care, and improved their knowledge of useful resources and apps. The importance of practice using the iPad is supported by Lombardo and Honisett's (19) results from a pediatric clerkship that found 70% of students agreed that the iPad was useful in achieving the learning objectives of the clerkship and 84% agreed the technology skills acquired by using the iPad would be useful in future medical careers. In our study, the students who did not report an increase in use and expertise indicated alternative ways to access information.

This study has several limitations: 1) the 37 students were the first class to rotate in one community hospital during a third-year Internal Medicine clerkship, 2) the wireless network in the community hospital that served all users (including patients and visitors) was unstable and lacked the robustness to support consistent mobile technology use, and 3) the iPad usage logs were self-report and could have been more accurate with automatic tracking of use.

The current study indicates that developing expertise in using mobile technology and various apps was critical for effective and efficient support of real-time clinical decisions. Although this study used iOS devices, certainly other mobile technology devices (e.g., Windows, Android) may have produced similar results. However, due to the limited medical-related apps on other platforms, healthcare providers are choosing iOS devices (8).

Several recommendations result from the study and include: 1) encouraging medical students to use mobile technology to access medical knowledge resources including the EHR; 2) providing data service capability and mid-level storage capacity on each device; 3) integrating quarterly app training to increase effectiveness in clinical decision support; and 4) providing tablets that fit into a white coat pocket.

Future research could include the impact of mobile technology on 1) patient outcomes and length of stay, 2) establishing patient/doctor rapport, 3) student performance on shelf tests and USMLEs, and 4) the preceptors’ experience with students use of mobile technology.

Ethical approval

Ethical approval was granted by the Institutional Review Board at The University of Georgia, April 2012, 2012-108951.

Previous presentations

Initial results of this study were presented at the following events:

1. eLearn, October 2012, Montreal, Quebec, Canada

2. Institute for Evidence-based Health Professions, January 2013, Athens GA

3. College of Education, UGA Faculty Research Conference, February 2013, Athens, GA

4. St Mary's Continuing Medical Education, March 2013, Athens, GA

5. American Educational Research Association, April 2014, Philadelphia, PA

Disclosure

No funding was received for this project.

Conflict of interest and funding

The authors have not received any funding or benefits from industry or elsewhere to conduct this study.