Distribution of a Primary Care Office Information System

Abstract

Primary Care Office InSite (PCOI) is a Web-based intranet application that provides ready access to a collection of information useful in primary care. The PCOI Web site was developed by, and is widely used within, the Massachusetts General Hospital (MGH) and its affiliated community practices. Over 1600 users logged 60,000 separate sessions in the past year. The site contains clinical practice guidelines, patient educational material, drug prescription and cost information and referral information, all designed for use during routine patient care activity. This paper discusses the problems encountered and the lessons learned during an ongoing experiment to disseminate PCOI via the Internet to four distant and very different ambulatory care sites. None of these sites (a rural community hospital, a city-wide health care network, an inner -city general hospital, and an Indian Health Service hospital have the resources to develop such an application internally.

INTRODUCTION

One of the recommendations from the Institute of Medicine (IOM) report on improving the delivery of medical care was to utilize information technology in daily practice to increase the use of scientifically valid decision-making.¹ Studies have repeatedly shown that the information needs of primary care physicians (PCPs) are not being met^{2, 3}. While there is a wealth of information available on the Internet, this information is not readily available and not optimally organized for use during daily patient care.

The information needs of primary care physicians and their information-seeking behaviors have been widely studied. The findings have been remarkably similar among clinicians at different levels of training and practice settings and across different cultures. An excellent review by Smith⁴ summarized a group of studies showing that the questions are often complex and multidimensional, and that doctors are most likely to seek answers to their questions from other doctors. Most of the questions generated by doctors can be answered, using electronic sources, but this is time consuming and expensive—and demands information skills that many doctors lack.

The typical primary care patient visit lasts 10–15 minutes. During that brief time, the PCP must review the patient’s chart, take a history, perform the necessary physical exam, discuss his/her findings, write prescriptions, fill out a billing/encounter form and write a note in the chart. At the same time, he or she must try to fulfill the needs of the patient for emotional support and understanding. It is not surprising that PCPs rarely search external sources for information. Even though many would agree that it is not optimal, mo st physicians rely on their personal knowledge and memory rather than published guidelines or the medical literature⁵. As McKibbon et al note, “Because of other demands on their time, clinicians preferred to take less than two minutes per question to find answers⁶.”

Many of the clinical questions that go unanswered during daily practice can be answered from the literature. In an Oregon study, medical librarians were asked to use only online resources to answer 60 PCP-generated questions. Time or cost res trictions were not applied to the searches. These “gold standard” searches averaged 43 minutes in length and $27.37 in cost. The clinicians judged the answers relevant in 56% of cases, and said that the information provided a “clear answer” to 46% of their questions⁷. The value of the trained searcher is not under question; however, the time and cost involved makes this approach impractical. There has been increasing attention to developing guidelines that promote evidence-based medicine, but less attention to mechanisms to transfer evidence into clinical practice and to keep up with the ever-increasing amount of clinical trial data. The reality is that it is nearly impossible for any human-based information service to provide cost-efficient, timely information under the two-minute deadline needed to satisfy a PCP. Shifting the work of answering questions that arise in daily practice to a computer-based information resource may provide the Just-in-Time information⁸ desired at the point of care.

BACKGROUND

In 1996, the Massachusetts General Hospital (MGH) began a number of operation improvement initiatives to identify and implement changes that would improve patient care, increase patient satisfaction, and promote cost-effective decision-making. One initiative of the General Medicine Unit was the development of evidence-based clinical guidelines focused on clinical relevance and potential to improve medical care efficiency and effectiveness. The guidelines, developed by a working group of the MGH adult medicine primary care practices, were not meant to represent hard and fast rules; clinical judgment is required in applying these guidelines to any individual patient. The philosophy of the work group was that time and resources are limited in primary care practice; therefore, selection and prioritization of recommended care guidelines should be based on the strength of direct evidence of benefit from clinical research. Individual guidelines were sent to MGH primary care and specialty content experts for review and comment; however, no clinicians outside the work group were asked to approve or endorse these guidelines. The work group considers it to be an ongoing responsibility to review and revise the existing guidelines as new evidence becomes available.

The guidelines were initially distributed to MGH adult primary care clinicians in hard-copy form but we soon realized that the size of the notebook would inhibit physicians from accessing this material in their routine patient care practice and that it would be difficult to distribute new or revised material in a timely fashion. Therefore we shifted the distribution from a printed binder to the MGH intranet taking advantage of Web technology to facilitate indexed searching, ease of navigation, and linking of related patient instruction materials to clinical guidelines to better respond to the point-of-care physician information needs. There are three different approaches to provide Web-based clinical information: providing pointers to information resources available over the Internet; providing electronic versions of commercially available standard medical reference sources; and creating a local information repository which combines focused, patient-centered knowledge and patient-specific recommendations and resources that facilitate the information management and workflow of routine practice. Each of these three approaches has advantages and disadvantages. The PCOI Web site is an example of the third approach; it was developed with the following design objectives:

1. The system must be easy to use; the content must be appropriately indexed so relevant information is easy to identify and access.

2. The system content must include the relevant information required to respond to many of the questions and issues encountered in primary patient care. The information must be focused, short, and relevant to specific patient-care issues. The application must provide patient information documents that contain relevant information that is consistent with the clinician guidelines.

3. The information source must be reliable, and must be associated with references to the evidence on which it is based. The authors must commit to regular review and update of the material.

4. The system must be supported and enthusiastically promoted by respected clinical leaders. A continuing effort should be made to recruit new users.

5. The technology supporting the system must be readily available (in the office or examining room of the physician), reliable, and not impose undue barriers of time -consuming login procedures. It should require fewer than five seconds (and three clicks) to access desired information, and an average of fewer than two minutes to read the relevant information. (These time intervals seem small, but in the present circumstances of primary care, access time is of critical importance.)

6. Wherever possible, the knowledge access system should be integrated with applications that support clinical care work flow such as drug and prescribing information, specialist referral names and phone numbers, printable copies of forms frequently used in primary care, information about how to access and use other hospital specific resources, etc.

7. The system designers should encourage user feedback about how the system can be improved, and what additional content should be added.

8. The marginal cost of supporting and distributing the system should be small, on the order of less than $5/physician/month.

The PCOI web site has largely met these design goals. Its on-going and steadily increasing use by PCPs and the hospital’s continuing financial support provide testimony to this success. MGH is committed to the continuing support of the MGH PCOI site, and has recently provided additional financial support to increase the programming and technical support, and to recruit a Clinical Director and a Medical Editor for PCOI. Over 1600 clinical and administrative staff of MGH and other hospitals in the Partners Health Care System have used PCOI in the past year with 60,000 sessions (defining a ‘session’ as a new interaction with PCOI). Each year, we evaluate PCOI with a survey that is distributed to all the MGH PCPs who are potential users of PCOI. Results of the 2002 survey of 254 such physicians (with a return of 180 surveys – a 71% response rate) showed that 53% of the users chose the highest ranking of importance of the web site (“The site is very useful. It helps me give much better patient care.”). An additional 25% of the respondents stated that the site was ‘useful.’ Over half the clinicians stated that they used the site between 10–20 times/week and almost half the respondents stated that use of the site saved over 10 minutes per half day patient care session.

We believe that the acceptance of the PCOI Web site is related to two important characteristics: (1) it provides ‘one-stop shopping’ -- a portal to both educational resources and workflow support; and (2) it has a high ranking according to the algorithm proposed by Shaughnessy⁹ for assessing the usefulness of such web based techniques.

Relevance is based on the frequency of exposure to the problem being addressed and the type of information being presented. Validity is the likelihood of the information being true, and work to access is the time and effort that must be spent extracting the information. Using this measure, PCOI achieves a high ranking in that the material in PCOI is selected and created by the physicians who are intimately familiar with the problems and the need for specific information. The authors of the material in PCOI have a strong commitment to evidence-based research and attempt to create material that has a documented evidence base. We have spent considerable effort in designing and indexing the content so that access is relatively simple and fast. The primary design philosophy of PCOI is that the content will be directly relevant, valid, and be accessed with a minimal amount of time and effort.

RESEARCH DESIGN

Our hypothesis was that the PCOI application developed to serve primary care physicians at MGH could be successfully transferred to sites quite different from MGH. The research objectives focus on the identification of the important factors that would facilitate or hamper the transfer. In addition, a key component of the research is to identify those changes in the content and organization of the original MGH PCOI that would be required for institutions with different cultures and staffing patterns and serving quite different populations. Other objectives were: to develop a “template” approach that will allow each participating institution to locally create and maintain site-specific data; to quantify the amount of effort required to make the necessary changes that each site requires; and to evaluate the level of user support needed for the remote sites. We also need to assess the level of computer, network, and Internet support required at each site in order to achieve a high level of clinician use.

The four experimental sites now operational are: Franklin Community Health Network (FCHN) associated with a small, rural community hospital—the Farmington Memorial Hospital in Farmington, Maine; the ambulatory clinics in Nashville General Hospital (an inner-city hospital staffed by Meharry Medical School) in Tennessee; the Boston Health Care for the Homeless Program (BHCHP) in Massachusetts (an organization responsible for the health care of 7000 homeless individuals, serving these individuals both in shelters and in the streets); and the Tuba City Indian Medical Center (a small hospital in an isolated part of a Navajo Indian reservation in Arizona.) These four sites share a variety of disparities in health outcomes attributable to race, ethnicity, and lack of access to health care. However, the sites differ in significant ways. They have different staffing patterns, organizational forms and facilities, serve different populations,, and have presented different opportunities and challenges.

We identified local project leaders at each site and created fully functional web sites for each location in less than a month. Access to each site is restricted by site-specific IP addresses. We request initial registration by each user and subsequent login using the same identifier (allowing us to track individual user activity). IRB approval has been obtained by MGH for this experiment. The clinical staff and administration at the different sites have demonstrated strong enthusiasm for the collaboration and are committed to working with t he Laboratory of Computer Science (LCS) to use and modify the MGH PCOI content and to provide the site-specific material. In each case we spent considerable effort in discussions with the local project leaders, learning the institutional characteristics and the needs and desires at each site, and maintaining frequent email and telephone communication. The only technical issues encountered were minor and related either to identifying all the IP addresses of the multiple locations at each site or to advising the technical staff at each location in the creation of links to the site-specific PCOI version. Each remote site has a separate PCOI website on an MGH server with content tailored to the site.

The four test sites varied in how quickly they implemented the initial system, introduced and added local material, and in the amount of usage at each site. A common experience at all the sites was significant startup costs (time, energy, and resources) for creating and customizing the site-specific content. The following factors have proved most important to a rapid implementation and heavy clinical use:

1. The presence of an influential and technically competent champion of the system to advocate for introduction at the local site. The most extensive modification of content, introduction of locally derived content and initial local usage has occurred at the site where the CEO combines strong commitment and leadership with technical and medical knowledge. There is strong leadership at the other sites, but due to time limitations and lack of resources, they have begun to add local material only slowly. In one location, the PCOI implementation was delayed because a new electronic medical record system was being installed. A common problem is the trade off in commitment and use between the PCOI application and a local EMR application (a different EMR at each site). Obviously an integration of these two applications would be ideal, but this is a complicated and difficult task, and cannot be addressed within the objectives, budget, and time constraints of the present grant.

2. The presence of high-speed Internet access, reliable networking, and computer terminals readily available for use by the clinicians to access the local PCOI. At one site, this has been particularly problematic in that there are only two computers in the ambulatory area and these are heavily used to access laboratory results. We are using a limited amount of grant funds to provide extra computers at this site to facilitate clinical access to PCOI. In contrast, the Indian Health Service site has extensive Internet access and is placing numerous computers in the patient care areas.

3. Availability of non-clinical personnel to collect, modify and add locally relevant data to the remote PCOI site. We recognized early in the planning phase of writing this grant that the MGH specific content would have to be modified or replaced, especially in the applications relevant to drug insurance coverage, and referral options. Our collaborating sites are not blessed with abundant personnel resources, but we have been pleased at the efforts, particularly of pharmacy personnel, at each site to create locally relevant drug formularies for PCOI. However, limited availability of medical and clerical resources does have negative impact on how rapidly each site can create and individualize their own content. We are providing small grants to two of the sites to allow them to hire extra personnel to gather locally relevant information that will enrich their PCOI site.

4. The degree to which the MGH-developed guidelines and patient instructions are appropriate for the population and context of the individual site. It became obvious that some of the guidelines, although based on national standards and evidence-based, were not applicable to the particular site. For example, the recommendation for using colonoscopy for cancer screening was not appropriate for the IHS site since they do not have the resources to offer screening colonoscopy. We have also found that the MGH-prepared patient instructions sometimes need to be modified or replaced with content appropriate to the geographical location and reading level of the population served. It should also be noted that some of the material developed at the test sites was of interest to PCP’s at MGH, since many patients seen at the MGH have low reading skills, while the MGH-developed material was not always easy to read.

SIGNIFICANCE

1. The basic premise that the PCOI concept and protocol can be disseminated seems to be supported. At one of the external sites, the use of the PCOI web site is now similar to that at the MGH. Historically there are few examples of acceptance and use at smaller and dissimilar medical centers of an application developed at a tertiary care center like the Massachusetts General Hospital. While it is premature to declare success, we are encouraged by the enthusiasm and initial implementations that have occurred thus far in the four sites.

2. We have identified some of the important impediments to easy adaptation and are developing strategies to overcome these. The most critical observation is that the degree of acceptance and usage is dependent not so much on the characteristics of the application, but more on the commitment and technical awareness of the leadership at the local site. Our experience so far is that the remote sites cannot carry extensive independent development but require continuing LCS support by our development team to update, customize and enhance the site.

3. There is unqualified and continued enthusiasm on the part of each of the sites to participate in the experiment and to devote resources to make it successful. It is clear that patience and continuing personal interaction are required. Two of the sites have expressed interest to extend and expand the application to a broader level (in Farmington to the state of Maine, in Tuba City to the Indian Health Service). We have made only a restrained response to these requests because we believe our priority should be to demonstrate viable implementations at the single local sites, hoping to achieve equivalent patterns of usage and user support that exists at MGH.

4. We have been unsuccessful in having the clinicians at the local sites participate in a formal evaluation effort documenting their pre-imple mentation customs of clinical knowledge access. We are able to analyze post-implementation patterns of clinical knowledge access since we track in detail the specific component usage by each individual at each site.

PLANS

1. We continue to work intensively with the leadership at each local site. Personal contact and continuing expression of interest and awareness of the local needs and problems is of the highest importance. To the extent possible, the Principal Investigators will make local site visits in an effort to understand the problems and the opportunities and to encourage and educate the leadership and the local clinicians.

2. We will continue to encourage the project leaders at each site to collect information about their clinical and formulary needs, their forms, ‘How To” information, and referral patterns for inclusion in their PCOI site. The more the remote sites provide useful information that facilitates workflow unique to the needs of the local site, the more useful and used will be the PCOI application.

3. We are acquiring licenses (at a reduced rate) for a few of the more commercially available educational resources (e.g., Scientific American Medicine, Lexicomp CRL OnLine) for each of the remote sites.

4. We are developing a common repository of the valuable ideas and content suggested by each local site so that all sites can profit by the experience and contributions of the other sites, and, at the same time foster a sense of community among the different collaborators.

5. We are continuing to collect data on the evaluation of the impact of the experiment at each site.