Direct Comparison of a Tablet Computer and a Personal Digital Assistant for Point-of-Care Documentation in Eye Care

Abstract

New mobile computing devices including personal digital assistants (PDAs) and tablet computers have emerged to facilitate data collection at the point of care. Unfortunately, little research has been reported regarding which device is optimal for a given care setting. In this study we created and compared functionally identical applications on a Palm operating system-based PDA and a Windows-based tablet computer for point-of-care documentation of clinical observations by eye care professionals when caring for patients with diabetes. Eye-care professionals compared the devices through focus group sessions and through validated usability surveys. We found that the application on the tablet computer was preferred over the PDA for documenting the complex data related to eye care. Our findings suggest that the selection of a mobile computing platform depends on the amount and complexity of the data to be entered; the tablet computer functions better for high volume, complex data entry, and the PDA, for low volume, simple data entry.

INTRODUCTION

The need to facilitate collection of clinical data by clinicians at the point of care will continue to increase if the projected proliferation of electronic health records (EHRs) comes to fruition. Fortunately, new portable devices have emerged in the form of tablet computers and personal digital assistants (PDAs) that enable point-of-care computing.¹ A recent report based on a survey from over 100 in-depth interviews with technology-savvy clinicians in the U.S. suggested that neither the PDA nor the tablet were ideally suited for clinical care.² PDAs were preferred for their lighter weight, portability, and longer battery life, while tablet computers were favored for their larger screen size, capacity to run more complex functions, full-size keyboard and advanced handwriting recognition capabilities. Consequently, since there are known advantages and disadvantages to either device, significant questions still remain regarding which device is most appropriate for a given application. While there is a growing body of scientific literature on the use of PDAs in health care,³ most reports are descriptive^3,4 or address limitations of the new technology.^5,6 A few studies have compared point-of-care documentation on a PDA with paper and found that the electronic format can increase the detail and quality of the entered data.^7,8 One study compared a PDA with a traditional computer terminal for entry of triage data and determined that the traditional computer was preferred.⁹ We identified no studies that performed a direct head-to-head comparison of a PDA and a tablet computer running identical applications. Therefore, to address the issue of which mobile device is optimal for recording clinical observations at the point of care, we created fully-functional prototypes for documentation of eye care on a Palm operating system-based PDA and a Windows-based tablet computer. This paper describes this comparison of devices through focus group sessions and usability surveys involving practicing ophthalmologists and optometrists from both academic and community settings.

METHODS

System requirements

We defined the documentation requirements for data collection for diabetic eye disease by abstracting recommendations from the American Academy of Ophthalmology’s guideline on diabetic retinopathy management,¹⁰ by auditing the encounter notes from patient charts at the Duke Eye Center, and by interviewing optometrists, ophthalmologists, and clinic technicians. Through this process, we identified over 600 data fields that are necessary for complete documentation of an encounter for diabetic eye care. Of these 600 data fields, 142 required free-text data entry for comments, 8 required drawings for illustrating eye findings observed during the patient examination, and the remainder required selection of specific options (implemented as pull-down menus, checkbox selections, or radio buttons). A sample of one of the screens from the tablet prototype illustrating a combination of these data fields is shown in Figure 1. In addition to documenting encounter observations, the system also generates suggestions for staging the extent of diabetic retinopathy based on findings entered by the clinician.

Figure 1.

Sample Tablet Screen Showing Diverse Types of Data Collected by the System.

Initial Device Selection

In selecting mobile equipment for this project, PDAs and tablet computers were the only hardware platforms considered since they represent the only mobile devices that could be easily used at the point of care. Laptop computers were not considered because they were too awkward to move between exam rooms and were not amenable to the annotation of diagrams.

We evaluated available PDAs and tablet computers in order to select the best candidate devices for prototype development. The chief determining factors were weight, screen size, screen resolution, and battery life. We also considered device response speed, price, operating system and manufacturer reputation. From this evaluation, we selected the Palm Tungsten T3, the Sony PEG-UX50, and the HP TC1100 as the platforms for prototype development. Table 1 outlines the salient features of these systems. We used Satellite Forms (www.satelliteforms.net) to develop the PDA version of the application and Visual Basic.NET (msdn. microsoft.com/vstudio/) to develop the tablet version.

Table 1.

Devices Selected for Prototype Development.

Feature	Computer
	Palm Tungsten T3	Sony PEG-UX50	HP TC1100 Tablet
Weight	5.5 oz.	6.2 oz.	3 lbs.
Screen Size	320 X 480	320 X 480	1024 X 768
Battery Life	2 Hours	7 Hours	4 Hours
O/S	Palm OS	Palm OS	Windows XP Tablet Edition
List Price	$399.95	$649.99	$2200.00
Other	Voice	Keyboard	Wireless
Features	Memo	Camera Voice Memo	Networking
Manufacturer	Palm	Sony	Hewlett Packard

Focus group sessions

In order to assess which device was most suitable for documenting an encounter for diabetic eye disease at the point of care, we conducted two focus groups, one for optometrists and one for ophthalmologists. Study investigators invited participants from both academic and private practice settings.

Each focus group session began with a description of our study, the agenda, and goals for the session. Participants then completed a validated survey to assess their level of computer experience.¹¹ Next, we provided a brief overview of the PDAs and the data collection application designed for the PDAs. This introduction was followed by a “think aloud” session in which each participant was provided with a PDA running the data collection application and paired with an observer who recorded the user’s comments and feedback in the course of entering data from patients they had seen in the past with diabetic eye disease. After a minimum of 30 minutes of hands-on experience with the PDA, users were given a usability survey based on validated assessment instruments.^12–15 We then provided an overview presentation on the tablet computer and repeated hands-on “think aloud” sessions and usability surveys for the tablet computer and its data collection application. Next, we gave participants a survey to record their device preferences and their reasons behind their preferences. The final activity of the session was a facilitated focus group discussion about the data entry devices and their respective applications. The comments from the focus groups were recorded by two scribes and later grouped into specific themes.

Usability Surveys

We created surveys to assess the usability of each device by selecting context-relevant questions adapted from four validated survey instruments that had different areas of emphasis, including user interaction satisfaction,¹² end user computing satisfaction,¹³ perceived usefulness,¹⁴ and doctors’ attitudes.¹⁵ Each response was based on a 5-point Likert scale. We combined participant responses into means and standard deviations for each question. Statistical significance was assessed using a two-sided, paired t-test at the 0.05 level of significance.¹⁶

The Duke University School of Medicine Institutional Review Board approved this study.

RESULTS

Application Characteristics

We created two functionally identical applications, one for Palm operating system-based PDAs and one for Windows-based tablet computers, to collect clinical observations from clinicians at the point of care for patients with diabetic eye disease. Both applications were developed to capture the same 600 required data fields and to generate recommendations for staging the extent of retinal disease in a specific patient based on the observational data entered by the clinician. On average, one data entry screen in the tablet-based application would require five screens in the PDA application to capture an equivalent amount of data. Consequently, more complex navigation was required when using the PDA. Examples of equivalent data entry screens on the two platforms are provided in Figures 3 and 4. The PDA application had 58 data collection screens and 8 screens for recommending staging and the tablet application had 15 data collection screens and 3 screens for recommending staging.

Figure 3.

Typical Data Entry Screen on Tablet Computer.

Figure 4.

PDA Data Entry Screens for Documentation Equivalent to the Tablet Screen in Figure 3.

Focus Group Findings

Four ophthalmologists and five optometrists participated in the focus groups. With regard to previous computer experience: five of the participants (56%) used a standard PC-type computer more than 15 hours per week; five (56%) also used a PDA one or more hours per week; six (67%) were self-taught computer users; and four (44%) considered themselves sophisticated computer users. Thus, our participants included both sophisticated and less sophisticated computer users.

Overall reaction to devices

Users’ overall reactions to the systems were measured using items drawn from the Questionnaire on User Interface Satisfaction (QUIS)¹² that were appropriate for our context (Table 2). The users’ responses to these questions revealed that the tablet computer was preferred over the PDA overall. Users found the tablet computers to be easier, more satisfying, and more stimulating to use than the PDAs. They also found tablet computers to be faster than the PDAs.

Table 2.

Users’ Overall Reaction to the Devices.

Question	Mean (Std Dev)
	PDA	Tablet
Overall reaction to the device: 1 = Terrible 5 = Wonderful	3.00 (0.76)	3.67 (0.71)
* Overall reaction to the device: 1 = Difficult 5=Easy	2.56 (1.01)	3.78 (0.44)
Overall reaction to the device: 1 = Frustrating 5 = Satisfying	3.00 (1.00)	3.56 (0.73)
Overall reaction to the device: 1 = Dull 5 = Stimulating	3.00 (1.00)	3.89 (0.60)
Overall reaction to the device: 1 = Slow 5 = Fast	2.22 (1.20)	3.38 (0.74)

^*statistically significant difference at p<0.05

User interface preference

Users’ views on the user interface were also measured using questions adapted from the QUIS¹² (Table 3). Users’ responses to these questions again revealed the tablet computers as being preferable to the PDAs. Compared to the PDA application, the tablet computer application was found to be easier to understand and navigate.

Table 3.

User Impressions of the User Interface.

Question	Mean (Std Dev)
	PDA	Tablet
Characters on the PDA/Tablet are: 1 = Hard to read 5 = Easy to read	4.11 (0.78)	4.44 (0.53)
* Organization of the information on the PDA/Tablet is: 1 = Confusing 5 = Very Clear	3.67 (0.86)	4.22 (0.44)
Sequence of the screens on the PDA/Tablet is: 1 = Confusing 5 = Very Clear	3.56 (1.13)	3.89 (0.60)
Learning to navigate the PDA/Tablet seems: 1 = Difficult 5 = Easy	3.67 (0.71)	3.89 (0.78)
Tasks can be performed in a straightforward manner: 1 = Never 5 = Always	3.78 (0.44)	3.78 (0.97)
* My location within the PDA/Tablet at any given moment was: 1 = Never apparent 5 = Always apparent	3.56 (0.53)	4.44 (0.72)

^*statistically significant difference at p<0.05

Satisfaction with information management

Users’ satisfaction with the information provided by the system was evaluated using the end user computing satisfaction scales¹³ (Table 4). Users’ responses to these questions revealed that both the PDA and tablet computer systems provided information in a satisfactory manner; however, users showed a slight preference for the tablet computers for meeting their information needs.

Table 4.

User Satisfaction with Information Provided.

Question (1 = Almost Never, 5 = Almost Always)	Mean (Std Dev)
	PDA	Tablet
Does the PDA/Tablet provide the precise information you need?	3.44 (0.88)	3.56 (0.53)
Does the information content meet your needs?	3.44 (0.73)	3.78 (0.67)
Does the PDA/Tablet provide sufficient information?	3.67 (0.87)	4.00 (0.50)
Does the PDA/Tablet seem to have accurate information?	3.78 (0.83)	4.11 (0.60)
Are you satisfied with accuracy of the PDA/Tablet?	3.78 (0.83)	4.00 (0.71)
Do you think the output presented in a useful format?	3.25 (1.04)	3.67 (0.50)
Is the information clear?	3.62 (0.92)	4.00 (0.53)

Impact on work patterns

We used the perceived usefulness scale¹⁴ and the doctors’ attitude scale¹⁵ to assess the systems’ impacts on the participants’ work patterns (Table 5). The participants’ responses to these questions revealed a general ambivalence regarding the use of either device in their daily work. In particular, users were concerned that use of either device would be time-consuming and increase their workload, although they felt these negative aspects would be more of an issue if they used the PDA system rather than the tablet computer system. Of note, these data entry applications prompted for the collection of many more data elements than are typically documented in an eye care visit for a patient with diabetes because our data entry templates were based on professional care guidelines that recommend collecting more data in order to meet a documented standard. This increased documentation requirement may account in part for users’ concerns regarding the impact of either device on their work patterns.

Table 5.

Users’ Views on How Use of the System Would Impact Their Work Patterns.

Question Anchors (1=Strongly Agree, 5=Strongly Disagree)	Mean (Std Dev)
	PDA	Tablet
Using the PDA/Tablet is time- consuming	1.89 (0.93)	2.39 (0.93)
Using the PDA/Tablet will increase my workload	2.11 (1.36)	2.33 (0.87)
By using the PDA/Tablet I will order more tests than I would have otherwise	3.78 (0.97)	3.44 (0.88)
* Using the PDA/Tablet will make a lasting difference on my style of practice	3.50 (0.93)	2.88 (0.78)
* Using the PDA/Tablet will shape my approach to managing patients	3.38 (0.52)	2.67 (0.71)
Using the PDA/Tablet will increase my knowledge of patient management	2.89 (0.93)	2.89 (0.78)
Using the PDA/Tablet will increase my confidence in my management of patients	2.89 (0.93)	2.75 (1.16)
The PDA/Tablet will be helpful in patient care	2.78 (0.66)	2.44 (0.73)
Patients will receive better care if I use the PDA/Tablet	3.56 (1.24)	2.83 (1.12)
Using the PDA/Tablet in my job will enable me to determine a patient’s retinal stage more quickly	3.44 (0.73)	3.00 (0.92)
Using the PDA/Tablet would improve my job performance	3.33 (1.32)	3.39 (0.49)
Using the PDA/Tablet in my job would increase my productivity	4.22 (0.83)	3.78 (0.83)
Using the PDA/Tablet would enhance my effectiveness on the job	3.33 (1.12)	3.11 (0.93)
Using the PDA/Tablet would make it easier to do my job	4.00 (0.87)	3.44 (0.53)
I would find the PDA/Tablet useful in my job	3.11 (1.05)	2.78 (0.83)

^*statistically significant difference at p<0.05

Overall device preference

The final survey questions dealt with overall device preference. Given the choice between the tablet computer and the two PDAs, the tablet computer was favored by 6 of the participants (67%). Among the participants preferring the PDAs, the participants preferred the Palm Tungsten T3 (22%) over the Sony PEG-UX50 (11%). The Sony included a camera and a miniature keyboard which generated interest, but these features were determined to add size and weight without offering significant utility.

DISCUSSION

In this study, we developed functionally equivalent applications for Palm operating system-based PDAs and Windows-based tablet computers to capture clinical observations pertaining to eye care directly from clinicians at the point of care. We compared the utility and usability of these applications on each device through focus groups and surveys involving practicing eye care professionals. For documenting the complex, image-rich data for eye care, we found that the tablet computers were preferred over the PDA for various reasons, including ease of navigation, the ability to record hand drawn images, and the ability to display information more clearly. These features are due mainly to the significantly larger screen size of the tablet computer. While PDAs are smaller and less expensive, their small screen size made them unsuitable for the collection of the large amounts of data recommended for diabetes eye care. We observed that the tablet computer is better suited for larger volumes of data and data of greater complexity (including extensive free text and images) whereas the PDA is better suited for smaller amounts of more simple data.

Strengths and limitations

Strengths of this study include the head-to-head comparison of functionally identical applications on a tablet computer and a PDA and the use of validated instruments to evaluate user preferences. The limitations of this study include the small number of participants and the focus on eye care professionals. In most instances, the small sample size only allowed us to detect trends in user preference as opposed to statistically significant differences at p<0.05. Because of the exploratory nature of this study we did not adjust our level of significance to account for multiple tests. With regard to the formative input, the impact of the small sample size is lessened by the observation that in usability studies, the return of additional data diminishes significantly after 5 subjects.¹⁷ We also acknowledge that the order in which the data entry devices were introduced could have potentially biased the users in favor of the device they used last (tablet). We sought to lessen the impact of exposure order by surveying users immediately after they finished using each devise. Our focus on eye care compelled us to address the collection of complex, detailed data that required a significant number of diagrams. Eye care may represent one of the more extreme challenges for point-of-care data entry by clinicians.

Implications

In this study we have illustrated a methodology for determining which type of mobile computing platform is most appropriate for a specific application. We created functionally equivalent application prototypes for both the PDA and the tablet computer and then assessed the acceptability of each device with practicing clinicians through focus groups, think-aloud sessions and validated usability surveys. While our focus was on the domain of eye care, these findings can be generalized to point-of-care data collection activities for other disciplines. In general, our findings show that the tablet computer is preferable because of its easier data entry, while the PDA offers some advantages due to its smaller size. The main limitations of the tablet computer are its larger size and higher cost, while the main limitation of the PDA is its small screen size.