Skip to main content
NCBI home page
AMIA Annual Symposium Proceedings logoLink to AMIA Annual Symposium Proceedings
. 2011 Oct 22;2011:1233–1242.

Usability Evaluation of a Personal Health Record

Noa Segall 1, Jeffrey G Saville 2, Pete L’Engle 2, Boyd Carlson 2, Melanie C Wright 1, Kevin Schulman 3, James E Tcheng 1,2
PMCID: PMC3243224  PMID: 22195184

Abstract

The electronic personal health record (PHR) has been championed as a mediator of patient-centered care, yet its usability and utility to patients, key predictors of success, have received little attention. Human-centered design (HCD) offers validated methods for studying systems effects on users and their cognitive tasks. In HCD, user-centered activities allow potential users to shape the design of the end product and enhance its usability. We sought to evaluate the usability and functionality of HealthView, the PHR of the Duke University Health System, using HCD methods. Study participants were asked to think aloud as they carried out tasks in HealthView. They then completed surveys and interviews eliciting their reactions to the web portal. Findings were analyzed to generate redesign recommendations, which will be incorporated in a future release of HealthView.

Introduction

The electronic personal health record (PHR) has been championed as a mediator of patient-centered care, yet its usability and utility to patients, key predictors of success, have received little attention. PHRs and other web-based functions for health self-management have been shown by some investigators to improve patient outcomes1–4, quality of life, hospital readmission rates, and mortality5, though other studies have found no effect of PHR use on process measures6,7 or outcomes8. Outcome-based PHR evaluations, however, provide little information about factors that contributed to the success or failure of PHRs9. Consequently, it is difficult to apply lessons learned in their development to the design of new PHRs. Human-centered design offers validated methods for studying systems effects on users and their cognitive tasks. These tools can be applied during system development to guide design decisions. Here we describe the application of such methods to the evaluation of a PHR.

Background

The PHR has been defined as “an Internet-based set of tools that allows people to access and coordinate their lifelong health information and make appropriate parts of it available to those who need it10.” PHRs enable care coordination across clinicians, provide patients access to their medical information, and empower them to manage their health and to be active participants in care decisions4,11. PHRs may be particularly useful for patients with chronic illnesses who expect to benefit from using PHRs to monitor and learn about their health problems. PHRs can also reduce costs related to their chronic conditions4,12,13. Chronically ill patients are likely to be early adopters of PHRs since they are active users of computers and the internet to perform health-related tasks14.

Various studies have directly linked PHR use to improved health outcomes1–4. There are several potential benefits of PHRs that likely contributed to these improvements. PHRs enable efficient access to credible health information and knowledge, which patients can leverage to promote their well being and to manage diseases. Most patients’ health data are distributed among the paper and electronic health records of various healthcare systems; a PHR can serve as a single repository for all of a patient’s health information12. Among other advantages, this can lead to more accurate medication reconciliation, evaluation of drug interactions and allergies, and tight control of medication regimens8,15. Clinicians can use patient-entered information to complete their medical records, avoid data re-entry, and recruit patient engagement to improve adherence to treatment and medication protocols12,16. Electronic communication tools can make more efficient use of both the clinician’s and the patient’s time and may substitute some office visits3,15. As a result of these benefits, PHRs may also reduce costs for healthcare payers and purchasers12.

Although there is national interest in PHRs12,17, there are several barriers to their widespread adoption. First and foremost is the issue of acceptance: both clinicians and patients can be skeptical of the usefulness of PHRs. The question of whether patients will make methodical, lifelong use of their PHRs as of now remains unanswered. PHRs introduce a new task, the management of health information, which most people have not previously performed and the value of which may not be immediately apparent14,18. Many patients express an interest in having online access to their medical records, though few keep electronic PHRs in reality10,18. Among those who do maintain a PHR, reactions seem to be mixed: some users reject them while others endorse them enthusiastically6. High-priority concerns of potential users include access rights in emergency situations, privacy and security of health information, and poor interface usability10,14,15. The significance of usability as a factor contributing to the success (or failure) of PHRs is acknowledged by both experts and current users10,19–21. A usable application is easy to learn (even by those who lack in technical expertise), simple to navigate, and intuitive to use. Failure to consider system usability in the design process can breed human errors and user dissatisfaction, and dissatisfied patients will simply abandon a poorly designed PHR22,23.

Human- or user-centered design (HCD) is a product and system design philosophy that aspires to enhance human abilities, overcome human limitations, and foster user acceptance24. It achieves these objectives by designing products around users’ characteristics, tasks, and workflow, as opposed to being driven by the available technology25,26. In HCD, user-centered activities allow potential users to shape the design of the end product and enhance its usability27,28. The application of HCD principles can beget systems that are easy to learn, increase user productivity, acceptance, and satisfaction, decrease errors, and decrease training time; while failing to consider the user in the design process often requires the redesign of a system, a time-consuming and costly process27,29.

Among others, HCD methods include usability testing, in which participants are asked to complete a task, or a set of tasks, using the product being evaluated. Depending on design goals, various metrics can be analyzed, including efficiency (e.g., time to complete a task, time spent recovering from errors, frequency of help documentation access), effectiveness (e.g., number of errors, number of subtasks completed, percent of participants who successfully completed a task), satisfaction (e.g., perceived satisfaction with and acceptance of the system), workload (e.g., perceived mental workload, objectively measured workload), and learnability (e.g., time to complete a task for new users, perceived ease of use and learning)20,30. A think aloud protocol can be incorporated into usability testing to help discover the source of system performance failures. In think aloud experiments, participants are asked to verbalize their thoughts as they complete a task, thus providing detailed insight into usability problems and their underlying causes23. User testing is considered a summative evaluation method30, that is, it is usually carried out during the later stages of design, using higher-fidelity prototypes than those tested in the conceptual design phase.

Applying HCD principles to PHR design is of particular importance, since most patients have not managed their health information online before and little is known about the characteristics of potential users, which functions they would use the most, and what changes in health-related behaviors will rise from adoption of this emerging technology12,14. Unlike other health information technology, health and technology literacy are central considerations in the design of PHRs and it is important to examine their effect on PHR use31,32. Health illiteracy can present users with difficulty in comprehending and applying information they read in their PHR31,33,34. Technology illiteracy and computer anxiety can be limiting factors in the adoption of electronic PHRs by older adults and patients with cognitive impairments33. In addition, patients of low socioeconomic status may not own a computer or an internet account, limiting their access to online PHRs35. By applying HCD principles to PHR design and evaluation with special attention to these considerations, functionality and usability can be enhanced, thereby increasing the likelihood of promoting beneficial health behaviors and health outcomes among patients20. However, few studies have analyzed PHRs from the user’s perspective.

The goal of this work was to evaluate the usability and functionality of HealthView, the PHR of the Duke University Health System. HealthView is a web-based portal that supports both administrative and clinical applications such as appointment scheduling, bill payment, advance registration prior to clinic visits, and review of medication lists, selected lab results (Figure 1), vital signs, and more. Study participants were asked to think aloud as they carried out tasks in HealthView. Afterwards, they completed surveys and interviews eliciting their reactions to the web portal. Findings were analyzed to generate redesign recommendations, which will be incorporated in a future release of HealthView.

Figure 1.

Figure 1.

Open in a new tab

Trended lab test results in HealthView.

Methods

Twenty participants volunteered for this study (prior research has shown that even 8–10 participants can lead to identification of up to 80% of usability problems36). Since our goal was to evaluate HealthView with users who are likely to use it frequently, we elected to recruit chronically ill patients, specifically those with cardiovascular disease. Each participant completed nine tasks, or scenarios, in random order (see Table 2) for a test patient.

Table 2.

Tasks completed as part of the study, the rate of participants making errors or giving up on tasks, and the rate of help requests.

Task Description Example Errors/Gave Up Help Requests
1. Find and interpret four lab results Your magnesium level was tested at your last doctor’s appointment. Check to see if your magnesium level is normal. 17.5% of labs 41.3% of labs
2. Make an appointment and reschedule it You would like to make an appointment to see Dr. Paterson for post-surgery follow up one month from now. 31.9% of subtasks 14.5% of subtasks
3. Describe how to take seven medications You have been told by your doctors to take several medications. Find each one on the website and describe how it should be taken [dose, route, frequency]. 13.5% of medications 0.8% of medications
4. Interpret trends in vital signs data You would like to see how your weight has changed in the last year. Describe what your latest weight looks like in comparison with earlier values. 22.4% of subtasks 12.2% of subtasks
5. Determine allergy documentation You’re allergic to Codeine. Check to see whether this allergy is documented in your record. 5.3% of subtasks 5.3% of subtasks
6. Find and print a payment history You’re filling out your annual tax report and would like to know your healthcare expenses for 2009. Find and print this report. 25% of subtasks 0% of subtasks
7. Update personal information Change your home address to 200 Trent Dr., Durham, NC, 27710. 11.9% of subtasks 5.1% of subtasks
8. Add a child to the health record You would like to be able to see your son’s medical information on the website. Add him as your dependent. 5% of subtasks 15% of subtasks
9. Find an introduction video Watch a video that provides an overview of the HealthView features. 5% of subtasks 25% of subtasks
Open in a new tab

Participants were asked to “think aloud”, or describe their thoughts, as they carried out these tasks. Afterwards, they were interviewed about usability problems they encountered, whether they would use HealthView in the future, factors that would prevent them from using it, and additional functionality that they would like to see. Their interactions with the web portal and their interviews were recorded. Participants also completed a background survey, a usability survey eliciting their reactions to HealthView37, and a survey gauging their interest in accessing different types of online health information (based on similar surveys38). A sample of interview and survey questions is presented in Table 1. Quantitative measures collected as part of the study included the number of participants who made errors or gave up on tasks (effectiveness), the number of help requests (efficiency), and participants’ satisfaction with and acceptance of HealthView (satisfaction). Qualitative data – responses to interview questions and observation notes – were analyzed using conventional content analysis39.

Table 1.

Examples of interview and survey questions presented to HealthView study participants.

Interview Questions (open-ended)
What do you think are the most useful features in HealthView that can improve the quality of care you receive?
If you could make one significant change to the HealthView website, what change would you make? Are there other changes you would make?
Would you consider opening a HealthView account? [Or if participant has an account:] Would you recommend HealthView to a friend or colleague? Why or why not?
Survey Questions (rating scales)
How often do you use the internet to look up information about medications?
How interested would you be in using the internet to make a doctor appointment?
Open in a new tab

Results

Twenty patients of the Duke Heart Center took part in the study. In addition, three pilot subjects (with no cardiovascular disease) completed usability testing and their survey data are included in the analysis. The 23 participants ranged in age from 27 to 84 (average, 53). Approximately 48% were males, 70% were white, and 78% had some type of cardiovascular disease. All had graduated high school and 74% completed additional education or training. Participants were relatively inexperienced with computers: 70% claimed beginner or average computer skills and 61% spent less than 10 hours per week using computers. About 61% of participants had previously opened a HealthView account.

In surveys, participants indicated that they performed only a small number of health-related tasks online. Commonly accessed information included medication (81%) and general health (86%) content, as well as information about their own medical conditions (86%). However, many expressed an interest in managing more aspects of their health online, including reviewing their allergies and immunizations, emailing their physicians, accessing medical reports, reading about medications and general and patient-specific health issues, and tracking their health conditions (over 86% of participants were moderately or very interested in performing these tasks through the internet). Activities that were less in demand included participating in online support groups and sharing medical information with family members (only 30% were moderately or very interested in these activities).

Although some participants were not familiar with HealthView features, after interacting with the portal they were very pleased with its functionality. In interviews, one participant said:

I have high blood pressure, a batch of things… I try to take control. HealthView… helps me buy into my health, my wellbeing, better. So I feel really good about it.

And another:

I think it’s a very informative and useful website. I think it can help you a lot… it keeps a lot of information for you.

All participants said they would consider opening a HealthView account or, if they already had one, would recommend it to a friend. They found all its features useful; making appointments and viewing lab test results were particularly well-liked features. A majority of participants agreed with statements such as “Using HealthView could help me manage my health” and “The information on HealthView could improve my interactions with my doctors and nurses” (87% somewhat or strongly agreed with these statements; Figure 2).

Figure 2.

Figure 2.

Open in a new tab

Participant agreement with usability- and usefulness-related statements.

Participants were asked to rate different aspects of HealthView’s usability on a scale of 1 to 5. They gave HealthView an average score of 3.9 on characteristics such as consistency, clarity of messages, learnability, and information organization. There was strong agreement with statements such as, “I can navigate the HealthView website easily,” “I can find information I need easily and quickly in HealthView,” and “I understand health information presented in HealthView” (100%, 96%, and 87% of participants, respectively, somewhat or strongly agreed with these statements). Responses to usability questions are presented in Figures 2 and 3.

Figure 3.

Figure 3.

Open in a new tab

Participant ratings of HealthView usability characteristics.

These findings, however, were in contrast with observations of the think aloud sessions. Navigation was not straightforward for participants. Between 30% and 60% experienced difficulty finding the lab test results, vital signs, allergies, payment history, add children page, and introduction video. Some tasks were difficult or frustrating for users to complete. Trying to find lab results, for example, was a time-consuming task when sorting through long lists of results. Making appointments and interpreting instructions for taking medications were error-prone tasks. Finally, data entry presented challenges to users in several tasks. Table 2 summarizes the observation measures, effectiveness (the error rate or participants giving up on tasks) and efficiency (the rate of help requests).

In interviews, several participants wanted medical terminology explained to them, specifically in the medications and lab results pages. One participant, for example, had this to say about her medical reports:

I received my own report back and was very, very confused by what everything meant. And while it said, no cause for alarm, you’re OK, there were items in this report that I didn’t know what they were.

Many comments reflected the idea that HealthView is not a walk-up-and-use website, for example:

I think, after you get used to it, you know where everything is. It’s just an initial… like with anything, I think I would come around… because I would know where everything is, where all the blocks are and how to fill them in.

The website is very good, it’s very informative… Once you get used to where you’re going… just like anything else, you have to learn it. But once you learn it, it’s a piece of cake.

Several comments were also made with respect to the usability of HealthView, such as:

The intent is there, but the user interface needs a lot of work.

I thought it was fairly user friendly… There could be some improvements. Like as far as how things are laid out.

Discussion

A human factors engineer experienced in human-computer interaction theory and in principles of interface design generated recommendations to address these and other issues encountered during usability testing of HealthView. The design recommendations will be incorporated in future releases of HealthView. Recommendations focused on navigation, consistency, efficiency (e.g., pre-populating forms), functionality, error messages, data entry, conformity with users’ expectations, and text visibility and salience. For example, interpreting abnormal lab values (task 1) presented difficulty for five (of 18) participants, three of whom could not tell which values were out of range. Figure 4 presents the current display of lab test results and a recommended display that enhances the salience of abnormal values.

Figure 4.

Figure 4.

Open in a new tab

(a) Current method of presenting normal (left) and abnormal (right) hemoglobin values in HealthView. (b) Recommended display of lab values.

Navigating forward and back between lab reports (task 1) was also a challenging task for ten (of 18) participants. Here, a suggestion was made to rename the existing buttons (Figure 5). Another recommendation pertained to the use of clinical terminology with respect to medication administration (task 3). “As needed” is preferable to “prn,” for instance, and “by mouth” is better than “orally.” Finally, Figure 6 presents an example of a menu redesign recommendation to prevent confusion when viewing, changing, or making appointments (task 2).

Figure 5.

Figure 5.

Open in a new tab

(a) Current buttons used to navigate between lab reports. (b), (c) Recommended button labels.

Figure 6.

Figure 6.

Open in a new tab

Current (a) and recommended (b) appointment menu structure.

Conclusions

Adopting a HCD approach to system design and evaluation has many advantages. Design efforts that have incorporated a HCD approach have shown significant human-system performance improvements in aviation, military systems, and healthcare environments40–43. In addition, involving users in the design process improves system acceptance29 and enhances usage and user satisfaction with the system44.

Increasing the adoption of PHRs is likely to also increase their effect on patient health. McMahon et al., for example, showed that frequent users of a diabetes management website had greater improvement in HbA1c levels than occasional users, as did users who uploaded more health data2.

In this study, we evaluated the usability and functionality of a PHR using HCD methods. Usability testing, interviews, and surveys were conducted to gain a comprehensive understanding of potential users’ needs and requirements when managing their health online. Although there was widespread satisfaction with the features they discovered through the study, participants were less pleased with HealthView’s usability. In particular, they encountered difficulty navigating the website, long task completion times, misleading interfaces leading to errors, and unclear medical terminology. These findings are in agreement with those of other PHR usability studies45,46. Employing HCD tools throughout the design process, rather than at a summative stage, may have proven useful in this respect. Formative methods such as card sorting to define navigation schemes and taxonomies can guide early design decisions. Further along the design cycle, additional methods are available for assessing low-fidelity prototypes, such as the cognitive walkthrough. Finally, after implementation, tools to track the rate of abandoned tasks can be useful for identifying usability and functionality problems.

It is worthwhile noting the discrepancy between the quantitative (survey) and qualitative (interview and observation) evaluation data. While participants gave high ranks to most usability characteristics of HealthView, they discussed various usability problems in interviews and were observed to experience real difficulty completing several scenarios. This was particularly evident in the gap between the self-assessment of their navigation capabilities (Figure 2) and the extended time they required to find pages, sometimes to the point of giving up. In part, this disparity may be explained by participants’ poor recall and awareness of the quality of their interaction with HealthView. Regardless of the rationale, it points to the need for applying multiple methods to evaluate PHRs and other health information technology. Different tools can produce both subjective and objective performance data and provide diverse perspectives about system usability9.

The more comprehensive the functionality of a PHR, the more useful it will be to patients and clinicians. In other studies, features that were most often requested by potential users included email communication with physicians, medication reconciliation, medical record access (and granting new physicians permission to access these records), and preventive care reminders10,47. In the current study, 87% of participants were also interested in emailing their physicians. This feature may be of particular importance, since provider utilization of PHR functionality (such as electronic communication with patients and the integration of patient-entered data into medical records) is essential for increasing PHR adoption. Other highly desired features included making appointments, reviewing allergies and immunizations, accessing medical reports, reading about medications and general and patient-specific health issues, and tracking personal health conditions. Access to such information and knowledge can be leveraged by patients to manage their health and to better inform their healthcare-related decisions.

References

  • 1.Ralston JD, Hirsch IB, Hoath J, Mullen M, Cheadle A, Goldberg HI. Web-based collaborative care for type 2 diabetes: A pilot randomized trial. Diabetes Care. 2009;32:234–9. doi: 10.2337/dc08-1220. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.McMahon GT, Gomes HE, Hickson Hohne S, Hu TM, Levine BA, Conlin PR. Web-based care management in patients with poorly controlled diabetes. Diabetes Care. 2005;28:1624–9. doi: 10.2337/diacare.28.7.1624. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Cho J, Chang S, Kwon H, et al. Long-term effect of the Internet-Based Glucose Monitoring System on HbA1c reduction and glucose stability: A 30-month follow-up study for diabetes management with a ubiquitous medical care system. Diabetes Care. 2006;29:2625–31. doi: 10.2337/dc05-2371. [DOI] [PubMed] [Google Scholar]
  • 4.Shea S, Weinstock RS, Teresi JA, et al. A randomized trial comparing telemedicine case management with usual care in older, ethnically diverse, medically underserved patients with diabetes mellitus: 5 year results of the IDEATel study. Journal of the American Medical Informatics Association. 2009;16:446–56. doi: 10.1197/jamia.M3157. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Martínez A, Everss E, Rojo-Alvarez JL, Figal DP, García-Alberola A. A systematic review of the literature on home monitoring for patients with heart failure. Journal of telemedicine and telecare. 2006;12:234–41. doi: 10.1258/135763306777889109. [DOI] [PubMed] [Google Scholar]
  • 6.Gysels M, Richardson A, Higginson I. Does the patient-held record improve continuity and related outcomes in cancer care: A systematic review. Health Expectations. 2007;10:75–91. doi: 10.1111/j.1369-7625.2006.00415.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Tuil W, Verhaak C, Braat D, de Vries Robbé P, Kremer J. Empowering patients undergoing in vitro fertilization by providing Internet access to medical data. Fertility and sterility. 2007;88:361–8. doi: 10.1016/j.fertnstert.2006.11.197. [DOI] [PubMed] [Google Scholar]
  • 8.Grant RW, Wald JS, Schnipper JL, et al. Practice-linked online personal health records for type 2 diabetes mellitus: A randomized controlled trial. Archives of Internal Medicine. 2008;168:1776–82. doi: 10.1001/archinte.168.16.1776. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Kushniruk AW, Patel VL. Cognitive and usability engineering methods for the evaluation of clinical information systems. Journal of Biomedical Informatics. 2004;37:56–76. doi: 10.1016/j.jbi.2004.01.003. [DOI] [PubMed] [Google Scholar]
  • 10.The personal health working group final report. Author, 2003. (Accessed August 21, 2009, at http://www.connectingforhealth.org/resources/final_phwg_report1.pdf.)
  • 11.Kaufman D, Patel V, Hilliman C, et al. Usability in the real world: Assessing medical information technologies in patients’ homes. Journal of Biomedical Informatics. 2003;36:45–60. doi: 10.1016/s1532-0464(03)00056-x. [DOI] [PubMed] [Google Scholar]
  • 12.Tang PC, Ash JS, Bates DW, Overhage JM, Sands DZ. Personal health records: Definitions, benefits, and strategies for overcoming barriers to adoption. Journal of the American Medical Informatics Association. 2006;13:121–6. doi: 10.1197/jamia.M2025. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Hess R, Bryce C, Paone S, et al. Exploring challenges and potentials of personal health records in diabetes self-management: Implementation and initial assessment. Telemedicine and e-Health. 2007;13:509–18. doi: 10.1089/tmj.2006.0089. [DOI] [PubMed] [Google Scholar]
  • 14.Lafky DB, Horan TA. Prospective personal health record use among different user groups: Results of a multi-wave study. 41st Hawaii International Conference on System Sciences; Big Island, HI: Waikoloa; 2008. 2008. pp. 233–41. [Google Scholar]
  • 15.Detmer D, Bloomrosen M, Raymond B, Tang P. Integrated personal health records: Transformative tools for consumer-centric care. BMC Medical Informatics and Decision Making. 2008;8:45–58. doi: 10.1186/1472-6947-8-45. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Stead WW. Rethinking electronic health records to better achieve quality and safety goals. Annual Review of Medicine. 2007;58:35–47. doi: 10.1146/annurev.med.58.061705.144942. [DOI] [PubMed] [Google Scholar]
  • 17.Steinbrook R. Personally controlled online health data--The next big thing in medical care? The New England Journal of Medicine. 2008;358:1653–6. doi: 10.1056/NEJMp0801736. [DOI] [PubMed] [Google Scholar]
  • 18.Kaelber DC, Jha AK, Johnston D, Middleton B, Bates DW. A research agenda for personal health records (PHRs) Journal of the American Medical Informatics Association. 2008;15:729–36. doi: 10.1197/jamia.M2547. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Dorr D, Bonner LM, Cohen AN, et al. Informatics systems to promote improved care for chronic illness: A literature review. Journal of the American Medical Informatics Association. 2007;14:156–63. doi: 10.1197/jamia.M2255. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Dabbs ADV, Myers BA, Mc Curry KR, et al. User-centered design and interactive health technologies for patients. Computers, Informatics, Nursing. 2009;27:175–83. doi: 10.1097/NCN.0b013e31819f7c7c. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Pagliari C, Detmer D, Singleton P. Potential of electronic personal health records. BMJ. 2007;335:330–3. doi: 10.1136/bmj.39279.482963.AD. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Zhang J, Johnson T, Patel V, Paige D, Kubose T. Using usability heuristics to evaluate patient safety of medical devices. Journal of Biomedical Informatics. 2003;36:23–30. doi: 10.1016/s1532-0464(03)00060-1. [DOI] [PubMed] [Google Scholar]
  • 23.Jaspers MWM. A comparison of usability methods for testing interactive health technologies: Methodological aspects and empirical evidence. International Journal of Medical Informatics. 2009;78:340–53. doi: 10.1016/j.ijmedinf.2008.10.002. [DOI] [PubMed] [Google Scholar]
  • 24.Rouse WB. Design for Success: A Human-Centered Approach to Designing Successful Products and Systems. New York: John Wiley & Sons, Inc; 1991. [Google Scholar]
  • 25.Bannon LJ. Issues in design: Some notes. In: Norman DA, Draper SW, editors. User centered system design : New perspectives on human-computer interaction. Hillsdale, NJ: Lawrence Erlbaum Associates; 1986. pp. 25–30. [Google Scholar]
  • 26.Gould JD. How to design usable systems. In: Helander M, Landauer TK, Prabhu PV, editors. Handbook of Human-Computer Interaction. 2 ed. Amsterdam: Elsevier; 1997. pp. 231–54. [Google Scholar]
  • 27.Johnson CM, Johnson TR, Zhang J. A user-centered framework for redesigning health care interfaces. Journal of Biomedical Informatics. 2005;38:75–87. doi: 10.1016/j.jbi.2004.11.005. [DOI] [PubMed] [Google Scholar]
  • 28.Abras C, Maloney-Krichmar D, Preece J. User-centered design. In: Bainbridge W, editor. Berkshire Encyclopedia of Human-Computer Interaction. Great Barrington, MA: Berkshire Publishing Group; 2004. pp. 463–8. [Google Scholar]
  • 29.Kujala S. User involvement: A review of the benefits and challenges. Behaviour & Information Technology. 2003;22:1–16. [Google Scholar]
  • 30.Wixon D, Wilson C. The usability engineering framework for product design and evaluation. In: Helander MG, Landauer TK, Prabhu PV, editors. Handbook of Human-Computer Interaction. Amsterdam: Elsevier; 1997. pp. 653–88. [Google Scholar]
  • 31.Kupchunas WR. Personal health record: New opportunity for patient education. Orthopaedic Nursing. 2007;26:185–91. doi: 10.1097/01.NOR.0000276971.86937.c4. [DOI] [PubMed] [Google Scholar]
  • 32.Kahn JS, Aulakh V, Bosworth A. What it takes: Characteristics of the ideal personal health record. Health Affairs. 2009;28:369–76. doi: 10.1377/hlthaff.28.2.369. [DOI] [PubMed] [Google Scholar]
  • 33.Lober WB, Zierler B, Herbaugh A, et al. AMIA Annual Symposium Proceedings. 2006. Barriers to the use of a personal health record by an elderly population; pp. 514–8. [PMC free article] [PubMed] [Google Scholar]
  • 34.Wolf M, Davis T, Shrank W, et al. To err is human: Patient misinterpretations of prescription drug label instructions. Patient Education and Counseling. 2007;67:293–300. doi: 10.1016/j.pec.2007.03.024. [DOI] [PubMed] [Google Scholar]
  • 35.Brodie M, Flournoy RE, Altman DE, Blendon RJ, Benson JM, Rosenbaum MD. Health information, the Internet, and the digital divide. Health affairs. 2000;19:255–65. doi: 10.1377/hlthaff.19.6.255. [DOI] [PubMed] [Google Scholar]
  • 36.Nielsen J. Usability Engineering. San Diego, CA: Academic Press; 1993. [Google Scholar]
  • 37.Chin JP, Diehl VA, Norman KL. Development of a tool measuring user satisfaction of the human-computer interface. Proceedings of ACM CHI’88 Conference on Human Factors in Computing Systems; New York, NY: Association for Computing Machinery; 1988. pp. 213–8. [Google Scholar]
  • 38.Health IT Survey Compendium 2011. (Accessed July 11, 2011, at http://healthit.ahrq.gov/portal/server.pt/community/health_it_tools_and_resources/919/health_it_survey_compendium/27874.)
  • 39.Hsieh H. Three Approaches to Qualitative Content Analysis. Qualitative Health Research. 2005;15:1277–88. doi: 10.1177/1049732305276687. [DOI] [PubMed] [Google Scholar]
  • 40.Endsley MR, Sollenberger R, Nakata A, Stein ES. Atlantic City, NJ: Federal Aviation Administration William J. Hughes Technical Center; 1999. Situation awareness in air traffic control: Enhanced displays for advanced operations. [Google Scholar]
  • 41.Klinger DW, Andriole SJ, Militello LG, Adelman L, Klein G, Gomes ME. Wright-Patterson AFB, OH: Department of the Air Force, Armstrong Laboratory, Air Force Materiel Command; 1993. Designing for performance: A cognitive systems engineering approach to modifying an AWACS human-computer interface. AL/CF-TR-1993-0093. [Google Scholar]
  • 42.Jungk A, Thull B, Hoeft A, Rau G. Evaluation of two new ecological interface approaches for the anesthesia workplace. Journal of Clinical Monitoring and Computing. 2000;16:243–58. doi: 10.1023/a:1011462726040. [DOI] [PubMed] [Google Scholar]
  • 43.Jalote-Parmar A, Badke-Schaub P, Ali W, Samset E. Cognitive processes as integrative component for developing expert decision-making systems: A workflow centered framework. Journal of Biomedical Informatics. 2010;43:60–74. doi: 10.1016/j.jbi.2009.07.001. [DOI] [PubMed] [Google Scholar]
  • 44.Baroudi JJ, Olson MH, Ives B. An empirical study of the impact of user involvement on system usage and information satisfaction. Communications of the ACM. 1986;29:232–8. [Google Scholar]
  • 45.Britto MT, Jimison HB, Munafo JK, Wissman J, Rogers ML, Hersh W. Usability testing finds problems for novice users of pediatric portals. Journal of the American Medical Informatics Association. 2009;16:660–9. doi: 10.1197/jamia.M3154. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 46.Tran DT, Zhang X, Stolyar A, Lober WB. Patient-centered design for a personal health record system. AMIA Annual Symposium; Washington, DC. 2005. p. 1140. [PMC free article] [PubMed] [Google Scholar]
  • 47.Fricton JR, Davies D. Advances in Patient Safety: New Directions and Alternative Approaches. Rockville, MD: Agency for Healthcare Research and Quality; 2008. Personal health records to improve health information exchange and patient safety. [PubMed] [Google Scholar]

Articles from AMIA Annual Symposium Proceedings are provided here courtesy of American Medical Informatics Association

RESOURCES