Comparing Text-based and Graphic User Interfaces for Novice and Expert Users

Abstract

Graphic User Interface (GUI) is commonly considered to be superior to Text-based User Interface (TUI). This study compares GUI and TUI in an electronic dental record system. Several usability analysis techniques compared the relative effectiveness of a GUI and a TUI. Expert users and novice users were evaluated in time required and steps needed to complete the task. A within-subject design was used to evaluate if the experience with either interface will affect task performance. The results show that the GUI interface was not better than the TUI for expert users. GUI interface was better for novice users. For novice users there was a learning transfer effect from TUI to GUI. This means a user interface is user-friendly or not depending on the mapping between the user interface and tasks. GUI by itself may or may not be better than TUI.

1. Introduction

Electronic Dental Record (EDR) has been used in the United States in various dental health settings. Dentistry is one of the healthcare professions that adopted computer systems fairly early in administration.¹ Although EDR has been widely adopted, few empirical or theoretical studies have examined the impact of user interface types on clinical work performance.

1.1. Design of User Interface

Users interact with a software application through its user interface. The main goal for interface design is to make it usable in terms of ease of use, ease of learning, and resistance to error. The transition from “technology-centered” to “user-centered” design is partially responsible for the wider use of information technology during the last 15 years² by users who are not trained in information technology and computer science. By definition, user-centered design focuses on users (including their characteristics, tasks and environment) whose work is to be supported by an application (i.e., functional requirements developed from a user’s perspective and referred to as user requirements). In support of the activities for identifying users and determining their support requirements many methods were developed including field observations, task analysis, scenarios, use cases, expert review, and formal usability testing with users.³ Although there are many methods for user interface design, few contain specific descriptions of how a designer transforms the information gathered about users and their work into an effective user interface.

1.2. Graphic User Interface

Direct manipulation is known to be a good property for interfaces, and many interfaces that have this property have been well received by users.⁴ The first major landmark of direct manipulation program is Sutherland’s Sketchpad, a graphic design program developed in 1963.⁵ Gradually GUI was accepted by designers and developers and it is now considered as a standard design feature for software applications. In general, GUI refers to techniques using graphics, keyboard, and mouse to provide an interface to an application. GUI is based on the idea of direct manipulation. One of the goals in GUI development is to match the GUI display to the user’s semantics or models. However, not every GUI is developed this way.

1.3. Comparing GUI & TUI

The trend in user interface design has shifted from TUI to GUI. This has been supported by many studies that show the advantages of GUI. Rauterberg evaluated menu selection using a TUI and a GUI for both novices and experts. Users were faster with the GUI: experts needed 51% less time to complete tasks.⁶ Davis and Bostrom examined the differences between a command-based (text) interface and a Macintosh GUI during the performance of file directory and structure utility tasks. They found a significant improvement in the ability of novice users to learn and perform with the GUI.⁷ Barker & Sloane, Inc. compared the word-processing and spreadsheet task activities using a GUI and TUI. The users completed 35 % more tasks, were 17% more accurate, were less frustrated and less fatigued with GUI than with TUI.⁸ Staggers compared a legacy TUI and a GUI prototype to discover the differences in nurse’s response time, errors, and satisfaction. The result shows GUI superior to TUI.⁹

However, a GUI does not automatically mean direct manipulation or ease of use. Whiteside compared outcomes for users of three different systems—command, menu, and iconic (GUI) systems. They found no significant differences in performance between the three formats for experts and a performance degradation for novices using the GUI interface.¹⁰ D'Ydewalle also discovered that experienced users of word-processing programs performed worse with a GUI than with a TUI.¹¹ Carroll and Mazur found that experienced command-based users learning Apple applications had problems selecting icons, opening and closing system files, and creating documents.¹²

These inconsistent results show that, despite the popularity of GUI and the perceived superiority of GUI over TUI, caution needs to be exercised with regard to the evaluation of GUI and TUI. Many factors can affect the usability of an interface, whether it is graphic or text-based. These factors include user types, expertise, tasks, etc.

2. Methods

2.1. System Description

Two isomorphic (functionally equivalent but representationally different) user interfaces, GUI (Figure 1) and TUI (Figure 2) for the Clinical Information System in a major dental setting in Houston were evaluated in this study.

Figure 1.

Graphic User Interface of the EDR

Figure 2.

Text-based User Interface of the EDR

2.2. Experiment Design

These two isomorphic interfaces were compared when performing a simple task by expert and novice users by empirical data collection and detailed task analysis. The task was “to find out which student takes care of a specific patient”. One observer observed 8 expert users: 4 used TUI and 4 used GUI to perform the task. Ten novice users were also observed. Each novice performed the task using both the text and the graphic interfaces. The order of interface types was randomized for the 10 novice users. An expert user was defined as one who had used this EDR system daily for more than 5 years. The novice user was defined as one who had never used this system. The built-in clock in this EDR system timed both expert and novice users. Instructions were given before the subjects started the task. During observation, the observer did not give any instructions or hints to users. All the behaviors were recorded in field notes. Also the subjects were encouraged to use a think-aloud method to describe the internal representation process including the users questioning themselves, frustrations, trying to find information to help them finish the task.

2.3. Task analysis

Task analysis is the process of identifying the information that has to be processed and the steps that have to be carried out in order to achieve the goal of a task using a specific device or method. The following three methods were employed in this study.

GOMS Analysis

Cognitive task analysis considers both physical and mental actions. Mental actions include perception, manipulation of mental representations, and generation of motor activities. GOMS analysis (Goals, Operations, Methods and Selection rules) is one of the best known models of cognitive task analysis.¹³ GOMS consists of descriptions of the methods needed to accomplish specific goals. In this project, GOMS analysis was used to analyze the physical and mental actions step by step.

Hierarchical task analysis

Hierarchical task analysis (HTA) is one of the most well-known forms of task analysis.⁴ It constructs a graphic representation of a decomposition of a high level task into its constituent subtasks, operations, or actions. It involves an iterative process of identifying tasks, categorizing them, breaking them down into subtasks, and checking the accuracy of the decomposition. Information about tasks is collected from a variety of sources including conversations with users, observation of user activities, job descriptions, and operating manuals.

Distributed representation analysis

Different representations of a common abstract structure can generate dramatically different representational efficiencies, task complexities, and behavioral outcomes.¹⁴ Internal representation is defined as knowledge and structures in the mind such as mental symbols, concepts, propositions, productions, schemas, and neural networks. External representation is defined as knowledge and structures in the environment, such as physical symbols, objects, dimensions, external rules, constraints, or relations embedded in physical configurations. External representation provides information that can be directly perceived and used without being interpreted and formulated explicitly. A task with more internal representation requires more cognitive load for the user.

3. Results

3.1. Statistical result

Table 1 shows the descriptive result of the average time and external steps taken for task performance by expert and novice users. For the expert user, GUI appears to require more time and more steps, but the difference between text and graphic was not significant (time: p=0.365, t-test; step: p=0.618, t-test), probably due to the small sample size (4 per cell).

Table 1.

Comparison of average time (seconds) and steps(number of steps) utilized by experts, novices and both groups combined using GUI or TUI

	Expert	Novice	Combined
GUI time (sec)	52.5	137.5	113.2
TUI time (sec)	36.5	311.3	232.8
GUI step	12.8	19.2	17.4
TUI step	10.8	30.9	25.1

Table 2 shows the descriptive result of the average time and steps used for task performance by novice users only. (Reminder: novice users performed the task with both TUI and GUI). For novice users, interface types made a significant difference in task performance time (p=0.024, General Linear Model (GLM) repeated measurement) and steps (p=0.001,GLM repeated measurement). This result shows that for the beginner using GUI, significantly less time is needed to perform a task than when using a TUI. This is also true for the steps needed for this task. This result did demonstrate GUI is an easier interface for a novice user.

Table 2.

Comparison of average time (seconds) and steps(number of steps) utilized by novices using GUI or TUI first in performing the task

	TUI time (sec)	GUI time (sec)	TUI step	GUI step
TUI first	438.8	81.2	38.6	12.6
GUI first	183.8	193.8	23.2	25.8
Overall	311.3	137.5	30.9	19.2

Another interesting finding is that there is an interaction between interface type and the order of task (time: p=0.019; steps: p=0.001) (see Figures 3 and 4).

Figure 3.

Time used to perform task by using GUI or TUI with different sequence

Figure 4.

Steps used to perform task by using GUI or TUI with different sequence

That is, if a novice used a TUI first, their performance of the same task using GUI was much better. However, if the novice used the GUI first, there was no improvement when they then performed the same task using TUI. This is an asymmetrical learning effect. It also shows that a novice user may use more time when they use the text interface first, but it will help them learn how to use graphic interface better.

3.2. Detailed task analysis result

Table 3 and Table 4 show the case study results of combining GOMS analysis, cognitive distributed representation analysis, and time estimated on each task in either TUI or GUI.

Table 3.

GOMS analysis, distributed representation analysis and time record using GUI to finish a task

Method for accomplishing goal of signing in the system.
Step #	Step description	Cognitive distribution	Time (seconds)
Step 1	Think of user name	Internal	18 sec
Step 2	Key in user name	External
Step 3	Think of password	Internal
Step 4	Key in password	External
Step 5	Hit “Check Mark” key	External
Step 6	Hit Enter key (redundant)	External
Method for accomplishing goal of finding patient
Step 1	Recognize the blank location	Internal	20 sec
Step 2	Locate cursor at last name blank	External
Step 3	Look at patient’s last name	External
Step 4	Type patient’s last name	External
Step 5	Look at patient’s first name	External
Step 6	Type patient’s first name	External
Step 7	Hit “Ok” key	External
Step 8	Recognize patient from the list	External
Step 9	Double click at patient’s name	External
Method for accomplishing goal of finding specific student
Step 1	Think which button can list student’s name	Internal	17 sec
Step 2	Click treatment plan	External
Step 3	Look at the screen and find student’s name	External
Step 4	Hit back button to go back to menu	External
Total: 19 steps, 4 internal15 external representations, 55 sec

Table 4.

GOMS analysis, distributed representation analysis, and time record using TUI to finish a task

Method for accomplishing goal of signing in the system.
Step number	Step description	Cognitive distribution	Time (seconds)
Step 1	Think of user name	Internal	20 sec
Step 2	Key in user name	External
Step 3	Think of password	Internal
Step 4	Key in password	External
Step 5	Hit Enter key	External
Step 6	Hit Enter key (redundant)	External
Method for accomplishing goal of finding patient of the blank
Step 1	Recognize the location	Internal	10 sec
Step 2	Hit Shift and Tab key	External
Step 3	Locate cursor at last name	External
Step 4	Look at patient’s last name	External
Step 5	Type patient’s last name	External
Step 6	Hit Tab key to move cursor to first name	External
Step 7	Look at patient’s first name	External
Step 8	Type patient’s first name	External
Step 9	Hit Enter key	External
Step 10	Recognize patient in the list	External
Step 11	Hit Tab key to locate cursor in front of patient’s name	External
Method for accomplishing goal of finding specific student
Step 1	Think what key can list student’s name	Internal	17 sec
Step 2	Type 70 in the blank	External
Step 3	Look at the screen and find student’s name	External
Step 4	Hit F3 key to go back to Menu External
Total: 21 steps, 4 internal 17 external representation, 47 seconds

The steps are the results of expert performance in each interface. When comparing the number of steps used to accomplish the goal, GUI needed fewer steps than TUI (19: 21 respectively). In cognitive loading, four steps in each interface resulted in internal representation. Therefore, the GUI did not really reduce the cognitive load. Comparing the time spent in accomplishing the goal, users spent more time with the GUI than with the TUI (55 seconds: 47 seconds respectively)

Figure 5 and Figure 6 show the case study results of hierarchical task analysis for both GUI and TUI. The expert workflow is in black; the novice’s workflow is in red. The goal is on the top, tasks are on the second level, and all other listings are the subtasks. The wave shape denotes the question the user was asked. The diamond shape indicates the decision. In Figure 5, the novice used 10 more actions than the expert to find the correct route. For the task, “sign in the program”, both expert and novice did it without problem. But after the user signed in the system, a redundant screen showed up, in which the user had only one choice: to click “enter”. Another problem of the interface is that the explanation on the screen was not very clear. For example, one function needs to click “F23” key, but there was no “F23” key on the keyboard. The main problem in the TUI was in the task “find the patient”. Once the user found the right patient they could easily find the information about the student. In Figure 6, the novice used ten more actions than the expert to find the correct route. For signing in the program, both expert and novice users did it without any problems.

Figure 5.

Hierarchical task analysis using TUI to complete the identical task by expert (black) and novice (red)

Figure 6.

Hierarchical task analysis using GUI to complete the identical task by expert (black) and novice (red)

The following results are found in this study:

• GUI requires significantly less time and fewer steps than TUI when used by a novice.

• For a novice, use of TUI or GUI as the first experience makes a difference for subsequent use of interfaces

• GUI does not reduce cognitive load for either the expert or the novice.

4. Conclusions

Evaluations of these two isomorphic interfaces (GUI and TUI) for an EDR system show that GUI was not necessarily better than TUI for an expert; rather they were better for novice users. For novice users, the first experienced interface made a significant difference in the subsequent use of the interface. When TUI was used first, it had a transferable learning effect to the GUI interface. TUI should be a valuable tool and readily available for training novice users for frequently changing interfaces among software. The task analyses we carried out were, in general, consistent with the empirical findings. One lesson learned from this study is that interface itself, whether GUI or TUI does not correlate with good or bad user performance.