Medical Student Appraisal

Abstract

Background

The rapidly growing industry for mobile medical applications provides numerous smartphone resources designed for healthcare professionals. However, not all applications are equally useful in addressing the questions of early medical trainees.

Methods

Three popular, free, mobile healthcare applications were evaluated along with a Google^TM web search on both Apple^TM and Android^TM devices. Six medical students at a large academic hospital evaluated each application for a one-week period while on various clinical rotations.

Results

Google^TM was the most frequently used search method and presented multimedia resources but was inefficient for obtaining clinical management information. Epocrates^TM Pill ID feature was praised for its clinical utility. Medscape^TM had the highest satisfaction of search and excelled through interactive educational features. Micromedex^TM offered both FDA and off-label dosing for drugs.

Discussion

Google^TM was the preferred search method for questions related to basic disease processes and multimedia resources, but was inadequate for clinical management. Caution should also be exercised when using Google^TM in front of patients. Medscape^TM was the most appealing application due to a broad scope of content and educational features relevant to medical trainees. Students should also be cognizant of how mobile technology may be perceived by their evaluators to avoid false impressions.

Background

The past decade has played witness to an enormous increase in mobile devices, placing near limitless resources into the palm of a user’s hand. The simultaneous transition from paper to electronic medical records poses a novel opportunity to markedly enhance efficiency and quality of patient care. Enthusiastic software developers of every genre have supported these new devices, wholeheartedly embracing Apple’s^TM classic “there’s an app for that” mentality. In fact, there are more than 500,000 applications (“apps”) on both the iTunes^TM [1] and Google Play^TM [2] stores, and both platforms report more than 25 billion apps downloaded as of September 2012 [3, 4]. However, not all resources deliver the same quality of information, as has been demonstrated in the context of clinical decision support observed between evidence-based medicine recommendations and information found in online markets [5]. Furthermore, many of these applications are designed for a target audience of practicing physicians and are appropriately tailored towards clinical management with less emphasis on basic physiology or pathophysiology. As early trainees, the scope of our educational needs is broader, often necessitating some basic science explanation in conjunction with clinical management as we become more independent in patient care. Additionally, many applications are priced well above a trainee’s budget. The purpose of this article is to evaluate four popular search modalities used by medical students to answer daily clinical questions, including a Google^TM web search as well as three free mobile healthcare applications.

Methods

The evaluation cohort consisted of six third year medical students rotating in various specialties at a large academic medical center. Three participants used Apple iOS and three used the Android operating system. Evaluation consisted of one week of exclusive use of each modality in the clinical setting. All applications were evaluated by testers in the same order, beginning with a Google^TM web search and followed by Epocrates Essentials^TM [6], Medscape^TM [7], and Micromedex^TM [8] in subsequent weeks. These applications were selected via a poll of the authors regarding which mobile applications were most commonly recommended by older students, residents, or senior physicians. A Google^TM web search was included given its familiarity to most users and commonplace, albeit perhaps unprofessional, use in the clinic.

Data was collected on search queries, satisfaction of search, and frequency of use. “Satisfaction of search” was characterized by the user’s acquisition of the information they were seeking from a credible source. Access frequency was reported in number of times each person used the resource per day. Participants were asked to freely comment on each modality’s overall strengths and weaknesses and score each perceived user interface. Scores were reported on a scale of 1 to 5, with a value of 1 representing lowest quality and 5 representing the highest mark.

Results

Usage Data

Satisfaction of search revealed Medscape^TM received the highest score (4.92) followed by Micromedex^TM (4.58), Epocrates^TM (4.42) and lastly Google^TM (4.08). User interface responses for Medscape^TM and Epocrates^TM were equal (4.16), followed by Micromedex^TM (4.0) and Google^TM (3.91). Average daily access frequency demonstrated Google^TM (5.2) was most frequently used followed by Medscape^TM (3.5), Epocrates^TM (3.0), and Micromedex^TM (2.7). These scores are tabulated in ►Table 1. Additionally, a summary of the strengths and weaknesses of each application is provided in ►Table 2.

Table 1.

Comparison of App Qualities and Usage. Scores for Satisfaction of Search and User Interface are based on a 1 to 5 scale, with 1 representing lowest quality and 5 representing highest quality. Access frequency values reflect averages from all users.

	Satisfaction of Search	User Interface	Access Frequency (times/day)
GoogleTM	4.08	3.91	5.2
EpocratesTM	4.42	4.16	3.0
MedscapeTM	4.92	4.16	3.5
MicromedexTM	4.58	4.00	2.7

Table 2.

Strengths and Weaknesses of Evaluated Modalities

	Strengths	Limitations
GoogleTM	•Low required specificity of search terms •Multimedia queries •Rapid and efficient for basic information	•Questionable reliability of sources •Inefficient for clinical management •Requires internet connectivity
EpocratesTM	•Pill identification and pricing features •Drug interactions •Reliable, trusted source	•Poor specificity for search terms •Lack of search history •Bulky user interface
MedscapeTM	•New filters for medical students •Interactive features – procedural guides, risk stratification, patient education •Partial off-line access	•Search query requires exact terms as listed in Medscape, does not search article text •No search history
MicromedexTM	•Drug mechanisms well explained •Identifies both FDA and off-label uses •Reliable, trusted source	•Single-purpose application •Bland user interface •Lack of built-in drug calculator

Google^TM

Strengths

Google’s^TM search engine required low specificity in search terms. This allowed students greater flexibility in generating relevant results despite spelling errors or acronyms. Google’s^TM greatest strength was its ability to return an endless array of multimedia resources, including images and video. In fact, one member reported that his attending physician even performed a Google^TM search on his phone to quickly find an image to illustrate a concept on morning rounds. Admittedly, this is perhaps an unfair advantage over mobile applications that are subjected to file size constraints. In regard to drug searching, one tester describes a common use reported by all of the medical students:

“You can type a drug name and the NIH page will come up as the first result and I think it is by far the fastest way to figure out the generic name for a brand name or vice versa.”

Limitations

A multimedia query also represents Google’s^TM greatest weakness, as it necessitates an Internet connection that may not be available in every clinical environment. An additional drawback was the variation in reliability of sources. Specifically, clinical management guidelines and drug regimens were often tedious to find from a reputable source using this method. For instance, adding “treatment” as a search parameter diluted results with suggestions from public forums rather than evidence-based management.

Epocrates^TM

Strengths

Epocrates^TM was favored for its quality reputation and inclusion of specific information that would be difficult to find elsewhere such as pricing, alternative drug regimens, and pill pictures. Specifically, one evaluator described the usefulness of the Pill ID feature:

“One of my patients knew why he took his medications but had no clue to their names, much less doses. By using Epocrates^TM Pill ID feature I was able to have him describe them to me and correctly identify about 70% of his home medication regimen.”

The app provides a very user-friendly drug interaction feature and drug calculator. It also offers several educational features appealing to medical students, including “Doc Alert” picture quizzes and concise tables on a multitude of topics that can be used for rapid review.

Limitations

One consistent criticism was difficulty in performing searches due to required exactness of search terms and lack of search history. One student also colorfully critiqued the user interface as “bulky,” going on to explain that he was often required to expand multiple table views to finally arrive at the desired information. Drug mechanisms of action were also not well explained, which is a particular weakness for medical students trying to learn an overwhelming number of pharmaceuticals. One additional consideration in evaluating Epocrates^TM is its cost for certain users. Large academic centers likely subscribe to institutional access, but not all students may have this benefit. Without an institutional license, Epocrates^TM can cost $99-$199 per year for a subscription [6].

Medscape^TM

Strengths

Medscape^TM was consistently praised for its comprehensive scope and intuitive, well-organized user interface. This included the option to download popular content to the mobile device, enabling access to content in the absence of an Internet connection. News updates were also provided with the ability to filter by topic or specialty, including a specific category for medical students. Users were even able to save reference articles for rapid access later or share with a colleague. The app offered concise procedural guides across a variety of specialties, and also included multimedia and interactive features salient to an article. For instance, risk stratification tables were often embedded as popups, which one student praised, saying, “There are tons of scoring criteria we aren’t familiar with, but the residents love to quiz us on.” Perhaps where Medscape^TM was most unique was that for each disease article there existed a Patient Education tab. One student explained its utility as follows:

“The section on ‘patient education’ is high yield, especially for med students as we are often the ones who have additional time to sit and talk with the patients. We can search for any condition and look at the patient education for a quick checklist of what is important to review with them prior to discharge.”

Limitations

Similar to other applications, the drawbacks to Medscape^TM were again a high requirement for specificity of search terms, particularly the inability to use acronyms, and a lack of search history. Furthermore, the search function was only applicable to titles of articles, not the full text, and was therefore limited in its ability to assist a medical student in formulating a differential diagnosis or performing symptom-based queries. Browsing for procedural guides was also less intuitive than would be desired, as one student noticed that, “Finding epidurals for labor was under anesthetic techniques, but I thought it could have easily also been under OBGYN, which it was not.”

Micromedex^TM

Strengths

While Micromedex^TM is more of a single-purpose app than the other two healthcare apps, for drug information it was considered one of the premium sources used in the clinic. It was also of particular use because it extensively explained drug mechanisms of action, which is an extremely desirable attribute for students. Additionally, both FDA-approved and off-label uses for drugs were clearly identified, with respective dosing for each indication. The search functionality in Micromedex^TM was also praised as a strength:

“I like that you can search by class and function too. If you type in ‘statin’ you get all the statins, or type in ‘antidote’ and you’ll find antidotes to common toxins.”

Limitations

The major downside to Micromedex^TM was that did not have a built-in calculator. As a single-purpose drug application, this represented a major shortcoming. Micromedex^TM was also critiqued for having a “cluttered” and “intimidating” user interface as well as a relative paucity of clinical information.

Discussion Of Applications

Application Appraisal

Mobile applications can be a tremendous resource to medical trainees. Reference materials, treatment algorithms, differential diagnoses, and medical calculators are all readily accessible [9]. Ideally, this rapid access makes patient care more efficient. It is also likely more pleasant for the medical student, as a phone or tablet is much more convenient to carry around than one or several textbooks. In fact, in 2011, 72% of physicians reported using smartphones [10] and 85% of medical providers in the Accreditation Council for Graduate Medical Educaton (ACGME) reported using their smart-phone to access medical information [11]. This number is likely higher for young medical trainees. Furthermore, many apps can store clinical data directly on the device, allowing access even in absence of an Internet connection.

Our assessment revealed that in general a Google^TM search was the most efficient in terms of time required to find desired information. However, as has been previously reported, this method often generated sources that were considered to be less reliable [12]. While we as medical students should be able to identify reliable sources to answer our queries, in our own experience patients are often mislead when conducting similar searches. Therefore, despite our preference for Google^TM search, we felt that using Google^TM in front of the patient would indirectly reinforce the idea that they should Google^TM their problems rather than seeking medical advice. We agree with a previous report that while the internet is a profound source for medical information that can empower patients to have discussions with their doctors and make informed health decisions, it is important for patients to know how to filter information they find on the web [13]. Ideally, healthcare professionals would facilitate this process by recommending websites to patients in what Gerber and Eiser have termed the “Internet prescription” [14].

Epocrates^TM and Micromedex^TM were both considered excellent resources for drug information, but Epocrates^TM was preferred over Micromedex^TM for its user-friendly design and additional educational content. The most popular mobile application, however, was Medscape^TM due to its comprehensive scope and user-friendly features such as procedure guides with videos and images, references for articles, and the ability to save and share documents. Medscape^TM combined a broad scope of clinical content with an appealing user interface and multiple interactive tools to aid a young clinician, making it the preferred app for daily clinical use.

In short, Google is an excellent mechanism to rapidly obtain explanations for basic disease processes, provided the user knows what sources to trust. It also is unparalleled in its ability to return multimedia resources, but falls short in answering questions regarding complex clinical management. Medscape^TM excels by combining a broad scope of pathophysiology and treatment approaches and outperforms other applications by embracing innovative educational features particularly useful for medical students, thereby earning our endorsement.

Clinical Awareness as Medical Students

Mobile applications have become an integral part of our daily lives and offer promising benefits to trainees in the health care setting. The culture of medicine, however, is not universally accepting of this new technology, especially among the older generation [15]. The problem arises from the fact that the same device that is such a powerful clinical tool is also connected to social media, email, and other non-medical outlets, posing the potential for inappropriate use in the workplace [16]. Attending physicians and supervisors may assume that a student on his phone is using it inappropriately rather than to advance his own education or aid in patient care.

In our experience, a gap exists between a generation of physicians who rely on medical textbooks for clinical information and those who have adopted electronic resources. Some students reported that utilizing their cell phones to look for clinical information were evaluated by their attending physicians as being “distracted.” However, when students looked up clinical information in textbooks or physical computers they were often commended for their initiative, as these mediums are both more transparent to onlookers as well as accepted in the workplace.

The acceptance of this new technology in the clinical setting will undoubtedly become commonplace in time; however, it currently presents a challenge to trainees who do not wish to be unfairly judged as disinterested in patient care. A possible solution to this dilemma is open dialogue with attending physicians about the use of mobile phones and tablets for the purpose of accessing clinical information. As medical students are often asked to give brief presentations to their clinical teams, a more educational approach could involve a brief synopsis of the student’s favorite clinical applications or resources. Many attending physicians may be interested in this new functionality, particularly as more residency and even medical school programs are adopting tablets such as the iPad^TM [17-20], or otherwise investing in mobile access to patient medical records.

Conclusion

Smartphones and mobile applications are an important part of our interconnected lifestyle. The rapidly growing industry for mobile medical applications offers new opportunities and challenges to medical trainees, who now have a wealth of information constantly available. However, there is little unbiased data regarding the accuracy of the information provided in mobile applications, which often carry a price tag that precludes utilization by medical trainees. In our evaluation, a Google^TM web search was the most preferred method for clarifying common questions posed by medical students, though its use was limited in clinical management. Medscape^TM was the most preferred free mobile application evaluated due to its interactive and educational features. As mobile applications continue to evolve, we hope the culture of medicine embraces the utilization of these resources in both the education and delivery of quality patient care by physicians of all stages of training.

Clinical Relevance

Mobile applications for healthcare are abundant despite a paucity of unbiased data evaluating their efficacy in promoting education or patient care. Google^TM web search is a familiar and commonly used means of acquiring clinical information. Increased awareness and evaluation of the capabilities of free mobile resources may help physicians make informed decisions before investing in expensive devices or applications.

Conflicts Of Interest

The authors declare that they have no conflicts of interest in this research.

Protection Of Human And Animal Subjects

This project was deemed “Exempt” by the Johns Hopkins Institutional Review Board. Human subjects outside of the authors were not included in this research.