Abstract
Objectives
(1) To develop a prototype Continuity of Care Record (CCR) with context-specific links to electronic HIV information resources; and (2) to assess case managers’ perceptions regarding the usability of the prototype.
Methods
We integrated context-specific links to HIV case management information resources into a prototype CCR using the Infobutton Manager and Librarian Infobutton Tailoring Environment (LITE). Case managers (N=9) completed a think-aloud protocol and the Computer System Usability Questionnaire (CSUQ) to evaluate the usability of the prototype. Verbalizations from the think-aloud protocol were summarized using thematic analysis. CSUQ data were analyzed with descriptive statistics.
Results
Although participants expressed positive comments regarding the usability of the prototype, the think-aloud protocol also identified the need for improvement in resource labels and for additional resources. On a scale ranging from 1 (strongly agree) to 7 (strongly disagree), the average CSUQ overall satisfaction was 2.25 indicating that users (n=9) were generally satisfied with the system. Mean CSUQ factor scores were: System Usefulness (M=2.13), Information Quality (M=2.46), and Interface Quality (M=2.26).
Conclusion
Our novel application of the Infobutton Manager and LITE in the context of case management for persons living with HIV in community-based settings resulted in a prototype CCR with infobuttons that met the majority of case managers’ information needs and received relatively positive usability ratings. Findings from this study inform future integration of context-specific links into CCRs and electronic health records and support their use for meeting end-users information needs.
Healthcare information has been defined as "the commodity used to help make patient care decisions” [1] and the quality of healthcare has been shown to depend on the quality of a clinician’s domain knowledge [2]. Nonetheless, many healthcare workers often have questions about how to care for their patients, and most questions go unanswered [3–5]. Previous studies have identified many obstacles to answering clinical questions [3, 6] and found that computer resources are infrequently used [7]. To address some of the obstacles related to meeting information needs, a number of investigators have designed context-specific strategies (e.g., infobuttons, InfoRetriever) for integrating information resources into existing information systems such as electronic health records (EHRs) or clinical information systems (CISs)[8–10].
The aims of this study were to develop a prototype Continuity of Care Record (CCR) with context-specific links to electronic HIV information resources and to assess case managers’ perceptions regarding the usability of the prototype. This research study differs from prior research in four ways. First, most research related to meeting information needs has focused on physicians or nurses. The sample in our study is case managers who care for Persons Living with HIV (PLWH). Because case management tasks differ from other types of clinical tasks, they are associated with different unmet information needs[11]. Second, the type of information system of focus is a CCR rather than the more information rich EHR or CIS. Third, much of case management for PLWH occurs in community-based settings and little is known about unmet information needs that arise in this context or solutions for meeting the information needs. Such research is a prerequisite to efficacy studies that evaluate the impact of the solution on clinician behaviors and patient outcomes. Fourth, our prototype development illustrates application of the Infobutton Manager and Librarian Information Tailoring Environment (LITE) [12, 13], resources that are available to others wishing to create context-specific solutions to meet information needs within information systems.
Background
Task of Case Management
Case management has been a frequently used strategy for targeting and coordinating care. More recently, case management was adopted for work with PLWH [14]. In contrast with diagnostic and treatment services, case management programs are designed to provide continuity of care for PLWH by coordination of resources and referrals to community-based social services and medical care [15]. HIV case managers address their clients’ medical needs, as well as spend considerable time addressing client needs related to social problems, including homelessness and substance abuse [16]. Since many services are available for PLWH, centralized resource coordination and referral to services are increasingly important [17, 18]. In addition, case managers can play a valuable role in promoting client adherence to highly active antiretroviral treatment (HAART), particularly since physicians do not always have adequate time to provide adherence counseling [19].
Continuity of Care Record
Unlike a comprehensive EHR or CIS, a CCR provides a limited set of data elements deemed to be essential for continuity of care across settings and its contents were initially specified by an ASTM International Standard [20]. Subsequently, the Continuity of Care Document (CCD) standard provided a specification for CCR implementation that incorporated aspects of the Health Level Seven Clinical Document Architecture [21]. CCRs have the potential to promote uncomplicated access to critical patient data and allow for care to be more complete by providing healthcare services that are less redundant, time intensive, and prone to error [22]. Improving case managers’ access to patient data through a CCR has the potential to generate information needs that were not previously explicated. Thus, information access solutions are essential.
Information Access Solutions
Information access may take many forms, ranging from looking up information in a textbook or journal or on a computer, to consults with other healthcare professionals [23]. With the advance of computer capabilities, electronic information resources have been developed and integrated into CISs [24, 25]. Electronic access to information resources at the point of care has the potential to improve information seeking within a busy clinical setting [26].
One approach to helping clinicians resolve their information needs is to create links within CISs to other on-line information resources [8]. If clinical data trigger the need for additional information, the same data can be co-opted into the retrieval process to reduce the effort needed on the part of the clinician [27] to resolve the information need. For example, infobuttons link clinical information, such as laboratory results, to information resources such as Up-to-Date[7]. Studies of infobutton use have shown that context-specific links to information resources are easy to use and helpful [28]. Timing studies have shown that users can get information quickly with a minimal amount of time spent on navigation [12]. Ebell and colleagues have developed InfoRetriever, which is a resource for handheld computers that provides up-to date evidence to guide patient care decisions [9, 10]. Infobuttons and InfoRetriever were both developed to help clinicians in their daily clinical practice by providing them with easy access to information resources. However, adoption of these systems has been slow, and most clinicians continue to practice without them. Lack of knowledge about the information needs of clinicians has been identified as one of the major reasons for slow uptake of systems [29]. Moreover, prior research has not focused on application of IT strategies designed to meet information needs for the task of case management or that occurred within the context of a CCR.
Methods
The CCR
This study was done within the context of SelectHealth, an HIV Special Needs Plan that provides comprehensive health services to its members through a fully-contracted network of providers. Highly coordinated services are necessary to provide comprehensive care and achieve desired outcomes for the management of PLWH [30]. Toward this goal, SelectHealth implemented a CCR, the SelectHealth Continuity of Care Document (SH-CCD), which is based upon the CCD standard, but more limited in content. The SH-CCD provides laboratory results and medication refills, which is critical information for the provision of HIV care.
The SH-CCD is available via a secure Internet connection to PLWH, clinicians, and case managers [31]. Case managers can access their client’s health information in the CCR if the case manager is part of the SelectHealth network or if the PLWH provides the case manager with access.
Development of the Prototype CCR with Infobuttons
The goal of embedding infobuttons into a CCR is to decrease the number of steps required for users to find resources to meet their information needs.[8] Development of the prototype SH-CCDwith infobuttons comprised three steps: 1) selection of electronic information resources to meet information needs; 2) adding links to electronic information resources to the Infobutton Manager knowledge base using the Librarian Infobutton Tailoring Environment (LITE)[12]; and 3) defining associations between information needs, electronic information resources, and user contexts.
The case manager information needs identified in our prior research served as the focus for selection of electronic information resources in our current study [11]. Case managers have unique information needs that differ from those of physicians and nurses. For instance in our previous work, we found that case managers had three major areas of information needs. Case managers described their need to access pertinent patient data. In contrast to clinicians who often have access to patient’s paper or electronic charts, many of our study participants work in freestanding case management offices and do not have access to this information. In addition, case managers had information needs related to educational resources about drug information adherence and substance abuse since, unlike clinicians, they do not have specific training in these areas. Third, case managers need specific information resources related to referral needs (e.g. housing, food) that are uniquely related to their role.
One author (RS) collected resources to address these information needs. The resources were reviewed by the authors and an additional nurse scientist with expertise in HIV care. Resources were added based on this review resulting in a set of electronic information resources which matched with information needs related to medications, laboratory tests, patient education and referrals.
We added links for the selected electronic information resources to the Information Manager knowledge base using LITE. LITE is designed for use by medical librarians and others who provide access to information resources (infobuttons.org). We used the Infobutton Manager [13] to create infobuttons as the mechanism for providing context-specific access to the selected resources. The Infobutton Manager uses a knowledge base of information needs and electronic information resources along with user context to predict information needs and create appropriate links to resources. For example, given the user context of viewing a medication list, the Infobutton Manager might predict that the user would want to know the side effects of a drug and consequently, create a link to a spot in an electronic information resource that displays side effect information for the particular drug of interest.
Usability Testing
We applied qualitative (think-aloud protocol) and quantitative (survey) data collection and analysis techniques to examine case managers’ perceptions of the usability of the prototype CCR with context-specific links.
Sample and Setting
Participants were recruited via personal contact from a single case management agency in New York City. Participants included Case Managers (N=5), Assistant Case Managers (N=2), and Senior Case Managers (N=2). The average age of participants was 27 years old and all were female. Four participants identified themselves as White, 2 as Black, 2 as Asian and 1 as multi-racial. All case managers had Bachelor’s degrees, and experience as a case manager ranged from 4.5 months – 11 years (M=2.01). All participants reported that they use a computer more than once a day and that they have at least 2 years of computer experience.
Procedures
Following informed consent, participants were asked to think-aloud as they used the CCR with context-specific links to complete a representative set of 10 tasks that are commonly performed by case managers while caring for PLWH. These included identifying a client’s CD4 count and most recent medication refills. Case managers’ utterances and screenshots were recorded using Morae™ software. One investigator (RS) also probed participants to gather additional information regarding functionality, features, processes, user interface, user-system interactions and manager-staff communication. We categorized the verbalizations from the think-aloud protocol and from the additional probes to identify key aspects of case managers’ positive and negative perceptions of the usability of the system.
The Computer System Usability Questionnaire (CSUQ) is a 19-item survey instrument developed at IBM to assess user satisfaction with system usability on a scale ranging from 1 (strongly agree) to 7 (strongly disagree). Items relate to efficiency, ease of use and likability of the system interface and an overall score and three factor scores are computed: System Usefulness , Information Quality , and Interface Quality. The CSUQ has strong evidence of reliability (α = .95), and content and construct validity. [32]A lower score on the CSUQ indicates a relatively more usable system. We administered the CSUQ after participants completed their ten assigned tasks. Data were summarized using descriptive statistics.
Results
CCR with context-specific links development
Examples of resources and associated information needs are shown in Table 1.Infobutton icons indicating context-specific links for three medications are displayed in Figure 2.For example, if a case manager is reviewing a patient’s current medication regimen, when the case manager clicks on the infobutton next to Norvir, she is taken to the Infobutton Manager which provides links to various references about the drug.
Table 1.
Information Needs and Sample Resources
| Generic Question1 | Case Manager Information Need | Resource |
|---|---|---|
|
|
UCSD Drug Guide, Rx List, AIDS Info |
|
|
Lab Tests Online, PubMed |
|
|
Lab Tests Online, PubMed, National Guideline Clearinghouse |
|
|
Hopkins HIV Guide, VA HIV Guide |
|
|
NYP Provider Lookup, yahoomaps.com, mapquest.com |
Information needs are categorized according to generic questions from the Information Needs Event Taxonomy[32]
Figure 2.
Box and Whisker Plot of CSUQ Satisfaction Scores (N=9)
Usability Testing
Case managers used the system for 30–60 minutes in a private office at their workplace.
Qualitative Findings
We summarized verbalizations from the think-aloud protocol and in response to probing into three categories: system components that worked well, system components that did not work well and desired system enhancements.
Components that worked well
There were many positive comments, especially concerning convenience, ease of use, and understandability of the information. In addition, several participants commented that the infobuttons would be particularly useful for finding drug information such as side effects, dosages and pictures of medications. Others commented that the resources linked to laboratory values would be helpful in understanding laboratory value ranges and explanations of laboratory tests.
Components that did not work well
Case managers found the labels for the resources to be confusing and cumbersome. For example, a case manager was looking for a picture of a medication and could not figure out from the names of the links if they were the correct resources to meet her information need.
Desired enhancements
Many case managers commented that they sought additional transportation resources. We included mapquest.com and maps.yahoo.com. However, case managers, suggested that maps.google.com and hopstop.com were more appropriate for their clients who travel by foot or public transportation, rather than drive to their appointments. In addition, case managers responded that too many clicks were often required to access a resource, and that infobuttons should more directly link to the appropriate information.
Quantitative Analysis
Mean CSUQ factor scores were: System Usefulness (M=2.13), Information Quality (M=2.46), and Interface Quality (M=2.26) (Figure 2). “Error messages” had the highest CSUQ score, indicating that users were not satisfied with the availability and comprehensibility of the error messages that they received. “Easy to learn the system” and “overall satisfaction” with the system had the lowest mean scores indicating higher usability in these areas. The average overall satisfaction was 2.25 (Table 2) indicating that users (n=9) were generally satisfied with the system.
Table 2.
CSUQ Satisfaction Scores (N=9)
| Mean (S.D.) | Range | |
|---|---|---|
| Overall | 2.25 (1.25) | [1.11, 4.95] |
| System Usefulness | 2.13 (1.33) | [1.00, 5.13] |
| Information Quality | 2.46 (1.34) | [1.14, 5.43] |
| Interface Quality | 2.26 (1.24) | [1.00, 4.33] |
Discussion
This study illustrates the application of the Infobutton Manager and LITE to develop a prototype CCR with infobuttons. The development of a CCR with infobuttons may be useful to others wishing to create similar context-specific links within electronic health information systems. Our study not only evaluated the system’s usability but we also found specific resources that would be appropriate to add, such as the inclusion of maps. Some information sources were not included due to limitations of resources for programmer support for prototype development. The architecture of Infobutton Manager and LITE allow for improvements to the system with very little effort or additional testing as long as the appropriate resources are identified and integrated.
Prior research identified lack of knowledge about the information needs as a barrier to uptake of systems designed to meet information needs [29]. In this study, infobuttons were created based upon explicitly identified information needs of case managers [11]. This likely contributed to relatively small number of missing content areas reported by case managers (e.g. information on when medications should be started, drug interactions, allergies and secondary diagnoses). This finding suggests the importance of first identifying the end-users’ information needs prior to system design.
The CSUQ usability scores indicated high satisfaction with the usability of the system. The Information Quality mean score although good, was less positive the other two factors. This can be explained in part because the Information Quality score includes the item about Error Messages. Overall, users did not feel like the system provided them with error messages that told them how to fix the issue they were experiencing.
The usability scores from our study were extremely encouraging. Even so, some case managers commented negatively on the efficiency of the infobuttons. For example, it took one case manager five or six clicks to find the information she was seeking and she explained that she would rather open a new browser window and use a search engine than the infobutton. These findings indicate that revisions to the system need to address the ease of accessing information resources [33]. Refinements to the system may include finding better resources or finding a better way to exploit the resource to get the user to the desired information more quickly.
One of the limitations of this study was the use of a small convenience sample of case managers from a single agency, but our previous research suggest that the demographics of the case managers at this agency were very similar to those at other HIV case management agencies. A selection bias may be present since the respondents were all computer and Internet users and were willing to use a computer-based system. Another limitation of our study was the use of a prototype, rather than a system used in the context of an actual patient setting. Although our study evaluated the use of context-specific links within a CCR, the SH-CCD differs from an EHR only in that is has less information in it and perhaps different types of information. As result, the lessons learned in this study are relevant to inclusion of infobuttons with EHRs and CISs. We therefore believe that the findings are generalizable outside of the context of a proprietary case management electronic record (SH-CCD).
Conclusions
Solutions are needed to address information needs at the point of care. The Infobutton Manager and LITE are tools for addressing such needs in the context of information systems. Our novel application of these tools in the context of case management for PLWH in community-based settings resulted in a prototype CCR with infobuttons that met the majority of case managers’ information needs and received relatively positive usability ratings. Iterative refinement based upon the usability evaluation is necessary prior to testing the efficacy in clinical settings.
The SH-CCD is currently being updated and a key enhancement to the current system is the addition of infobuttons. Findings from this study inform the inclusion of context-specific links in a CCR to meet the information needs of healthcare workers. In addition, this study provides system developers with a method for understanding and implementing context-specific links within information systems such as CCRs, EHRs, and CISs to meet usability principles and end-user acceptance.
Figure 1.
Prototype CCR with Context-specific Links
Summary points.
What is already known on the topic?
Many healthcare workers often have questions about how to care for their patients, and most questions go unanswered
There are many obstacles to answering clinical questions. Clinicians have documented that the most frequent barrier to having desired information sources is a lack of time to look up information
What this study added to our knowledge?
Our application of infobuttons in the context of case management for PLWH in community-based settings resulted in a prototype CCR with infobuttons that met the majority of case managers’ information needs and received relatively positive usability ratings.
Users of a CCR for PLWH have information needs that differ from those of users of other types of Clinical Information Systems.
Preliminary evidence suggests that infobuttons will be useful in addressing information needs, with current inadequacies easily addressed using available technology
Highlights.
-
○
Our study is the first to focus on creating infobuttons to meet the information needs of case managers in the clinical domain of HIV.
-
○
Context specific links to HIV resources have not been evaluated within a clinical information system.
-
○
Preliminary evidence suggests that infobuttons will be useful in addressing the information needs of HIV case managers.
-
○
The results of this study can be used to help meet information needs and can contribute to the overall care of PLWH.
Acknowledgments
The authors thank Peter Gordon, MD and Eli Camhi, MSSW, Principal Investigators of the parent project (NewYork-Presbyterian Hospital/Select Health CCD Demonstration Project, H97HA08483), Martha Rodriguez for her assistance in subject recruitment, and the case managers who participated in the study. The study was supported by the National Institute of Nursing Research (P30NR010677) and the Health Resources and Services Administration Grant (D11HP07346).
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Author Contributions
Drs. Schnall, Cimino and Bakken have all made substantial contributions to all of the following: (1) the conception and design of the study, or acquisition of data, or analysis and interpretation of data, (2) drafting the article or revising it critically for important intellectual content, (3) final approval of the version to be submitted.
Conflict of interest
The authors do not have any financial and personal relationships with other people or organisations that could inappropriately influence (bias) their work. In addition the authors do not have any other relations that could be a potential conflict of interest.
Contributor Information
Rebecca Schnall, Email: rb897@columbia.edu, Columbia University School of Nursing, 617 W. 168th Street, NY, NY 10032 USA.
James J. Cimino, Laboratory for Informatics Development, NIH Clinical Center, Bethesda, MD
Suzanne Bakken, Columbia University School of Nursing and Department of Biomedical Informatics, College of Physicians & Surgeons, Columbia University.
References
- 1.Wyatt J. Medical informatics, artefacts or science? Methods of Information in Medicine. 1996;35(3):197–200. [PubMed] [Google Scholar]
- 2.Braun LM, et al. Towards patient-related information needs. International Jounrnal of Medical Informatics. 2007;76(2–3):246–251. doi: 10.1016/j.ijmedinf.2006.03.004. [DOI] [PubMed] [Google Scholar]
- 3.Ely JW, et al. Answering physicians' clinical questions: obstacles and potential solutions. Journal of the American Medical Informatics Association. 2005;12(2):217–224. doi: 10.1197/jamia.M1608. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Allen M, et al. The Classification of Clinicians’ Information Needs While Using a Clinical Information System; AMIA Annual Symposium Proceedings; 2003. pp. 26–30. [PMC free article] [PubMed] [Google Scholar]
- 5.Currie LM, et al. Clinical information needs in context: an observational study of clinicians while using a clinical information system; AMIA Annu Symp Proc; 2003. pp. 190–194. [PMC free article] [PubMed] [Google Scholar]
- 6.Haug JD. Physicians' preferences for information sources: a meta-analytic study. Bulletin of the Medical Library Association. 1997;85(3):223–232. [PMC free article] [PubMed] [Google Scholar]
- 7.Cimino JJ, et al. Use of Online Resources While Using a Clinical Information System; AMIA Annual Symposium Proceedings; 2003. [PMC free article] [PubMed] [Google Scholar]
- 8.Cimino JJ, Li J. Sharing infobuttons to resolve clinicians' information needs; Proceedings AMIA Annual Fall Symposium; 2003. [PMC free article] [PubMed] [Google Scholar]
- 9.Ebell MH, et al. Update on InfoRetriever software. Journal of the Medical Library Association. 2002;90(3):343. [PMC free article] [PubMed] [Google Scholar]
- 10.Ebell MH, Barry HC. InfoRetriever: rapid access to evidence-based information on a hand-held computer. MD Comput. 1998;15(5):289, 292–297. [PubMed] [Google Scholar]
- 11.Schnall R, et al. Information needs of case managers caring for persons living with HIV. J Am Med Inform Assoc. 2011;18(3):305–308. doi: 10.1136/jamia.2010.006668. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Cimino JJ, et al. Redesign of the Columbia University Infobutton Manager; AMIA Annu Symp Proc; 2007. pp. 135–139. [PMC free article] [PubMed] [Google Scholar]
- 13.Cimino JJ. Infobuttons: anticipatory passive decision support; AMIA Annu Symp Proc; 2008. pp. 1203–1204. [PubMed] [Google Scholar]
- 14.Chernesky RH, Grube B. Examining the HIV/AIDS case management process. Health & Social Work. 2000;25(4):243–253. doi: 10.1093/hsw/25.4.243. [DOI] [PubMed] [Google Scholar]
- 15.Merrill EB. HIV/AIDS case management: a learning experience for undergraduate nursing students. The ABNF journal : official journal of the Association of Black Nursing Faculty in Higher Education, Inc. 1996;7(2):47–53. [PubMed] [Google Scholar]
- 16.Abramowitz S, Obten N, Cohen H. Measuring case management for families with HIV. Social Work in Health Care. 1998;27(3):29–41. doi: 10.1300/J010v27n03_02. [DOI] [PubMed] [Google Scholar]
- 17.Merithew MA, Davis-Satterla L. Protease inhibitors: changing the way AIDS case management does business. Qual Health Res. 2000;10(5):632–645. doi: 10.1177/104973200129118697. [DOI] [PubMed] [Google Scholar]
- 18.Ungvarski PJ. Challenges for the urban home health care provider. The New York City experience. The Nursing Clinics of North America. 1996;31(1):81–95. [PubMed] [Google Scholar]
- 19.Shelton RC, et al. Role of the HIV/AIDS case manager: analysis of a case management adherence training and coordination program in North Carolina. AIDS Patient Care and STDS. 2006;20(3):193–204. doi: 10.1089/apc.2006.20.193. [DOI] [PubMed] [Google Scholar]
- 20.ASTM International. [cited 2009 December 14];ASTM E31.28 E-. Standard Specification for Continuity of Care Record (CCR) 2005 Available from: www.astm.org/COMMIT/E31_ConceptPaper.doc < http://www.astm.org/COMMIT/E31_ConceptPaper.doc>.
- 21.Simonaitis L, et al. Continuity of Care Document (CCD) Enables Delivery of Medication Histories to the Primary Care Clinician. AMIA Annu Symp Proc. 2010;2010:747–751. [PMC free article] [PubMed] [Google Scholar]
- 22.Siika AM, et al. An electronic medical record system for ambulatory care of HIV-infected patients in Kenya. International Jounrnal of Medical Informatics. 2005;74(5):345–355. doi: 10.1016/j.ijmedinf.2005.03.002. [DOI] [PubMed] [Google Scholar]
- 23.McKnight L, et al. Perceived information needs and communication difficulties of inpatient physicians and nurses. Journal of the American Medical Informatics Association. 2002;9(Nov–Dec suppl):S64–S69. [PMC free article] [PubMed] [Google Scholar]
- 24.Haynes RB. Of studies, syntheses, synopses, summaries, and systems: the "5S" evolution of information services for evidence-based healthcare decisions. Evidence Based Medicine. 2006;11(6):162–164. doi: 10.1136/ebm.11.6.162-a. [DOI] [PubMed] [Google Scholar]
- 25.McKibbon KA, Fridsma DB. Effectiveness of clinician-selected electronic information resources for answering primary care physicians' information needs. Journal of the American Medical Informatics Association. 2006;13(6):653–659. doi: 10.1197/jamia.M2087. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Magrabi F, Westbrook JI, Coiera EW. What factors are associated with the integration of evidence retrieval technology into routine general practice settings? International Jounrnal of Medical Informatics. 2007;76(10):701–709. doi: 10.1016/j.ijmedinf.2006.06.009. [DOI] [PubMed] [Google Scholar]
- 27.Cimino JJ. Linking patient information systems to bibliographic resources. Methods of Information in Medicine. 1996;35(2):122–126. [PubMed] [Google Scholar]
- 28.Maviglia SM, et al. KnowledgeLink: impact of context-sensitive information retrieval on clinicians' information needs. Journal of the American Medical Informatics Association. 2006;13(1):67–73. doi: 10.1197/jamia.M1861. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Sintchenko V, Usherwood T, Coiera E. Are clinicians' information needs and decision support affected by different models of care? Experimental study. Studies in Health Technology and Informatics. 2007;129(Pt 2):895–899. [PubMed] [Google Scholar]
- 30.Hecht FM, et al. Optimizing care for persons with HIV infection. Society of General Internal Medicine AIDS Task Force. Annals of Internal Medicine. 1999;131(2):136–143. doi: 10.7326/0003-4819-131-2-199907200-00011. [DOI] [PubMed] [Google Scholar]
- 31.Jerant A, Hill D. Does the use of electronic medical records improve surrogate patient outcomes in outpatient settings? Journal of Family Practice. 2000;49:349–357. [PubMed] [Google Scholar]
- 32.Lewis JR. IBM Computer Usability Satisfaction Questionnaires: Psychometric Evaluation and Instructions for Use. International Journal of Human-Computer Interaction. 1995;7(1):57. [Google Scholar]
- 33.Cimino JJ, et al. Practical considerations for exploiting the World Wide Web to create infobuttons. Stud Health Technol Inform. 2004;107(Pt 1):277–281. [PubMed] [Google Scholar]