Abstract
Practice guidelines are an integral part of evidence-based health care delivery. When the authors decided to install the clinical documentation component of an electronic health record in a nurse practitioner faculty practice, however, they found that they lacked the resources to integrate it immediately with other systems and components that would support the processing of clinical rules. They were thus challenged to devise an initial approach for decision support related to clinical practice guidelines that did not include interfacing with an inference engine and set of decision rules. The authors developed a prototypic application within the WAVE electronic health record that demonstrates the feasibility of representing a guideline as structured encoded text organized into an online patient-encounter template. Although this approach may be more broadly applicable, it is described within the context of the management of diabetes mellitus by nurse practitioners. The advantages of the approach relate primarily to the integration of the guideline recommendations with the encounter form, the online interaction of the clinician with the system, and the ease of creation and modification of the guideline-based encounter form. However, there are several limitations of the current approach as a result of the inability to do inference and the lack of integration with patient-specific data to trigger specific rules.
Although heightened interest in evidence-based health care is more recent in origin, for more than two decades computers have assisted in the provision of reminders to clinicians regarding standardized protocols or guidelines.1,2,3 A number of randomized clinical trials have demonstrated that computer-based reminders decrease errors of omission1,4,5 and increase compliance with preventive care guidelines.6,7,8 These have recently been reviewed in detail by others.9,10,11 The literature indicates that the impact of these systems is best realized through an integrated set of applications with access to a broad array of patient data and well-specified decision rules.3,11 However, many clinical settings, particularly small ones, are lacking one or more of the components needed to implement a fully integrated decision-support system. Such was the case in our small nurse practitioner (NP) faculty practice when we decided to install the clinical documentation component of an electronic health record but lacked the resources to integrate it immediately with other systems and components. Thus, we were challenged to devise an initial approach for decision support related to clinical practice guidelines that did not include interfacing with an inference engine and set of decision rules. The resulting approach has a number of limitations, but it may provide guidance for others facing a similar challenge.
Although our approach may be more broadly applicable, we describe it within a context relevant to our faculty practice—i.e., the management of diabetes mellitus by NPs. As background, we review selected literature related to NPs and to computer-based support for diabetes mellitus management. We illustrate our approach using recommendations of the American Diabetes Association (ADA) and summarize its strengths and limitations.
Background
Nurse Practitioners
Nurse practitioners engage in diagnostic and treatment activities aimed at a broad range of human responses to actual and potential health problems. These include preventive services12,13; treatments related to nursing diagnoses, such as self-care deficit, impaired coping, and knowledge deficit14; and the management of acute and chronic illnesses.15,16 A meta-analysis revealed that, in comparison with physicians, NPs achieved equivalent or higher scores on process and outcome measures such as the provision of health promotion activities, rate of drug prescription, resolution of pathologic conditions, patient functional status, patient compliance, and patient knowledge.17 Historically, the predominant setting for NP care has been ambulatory care, particularly in medically underserved areas. More recently, however, models for acute care NPs have been implemented.18
Few studies have examined the impact of computer-based decision support on NP decision-making. Recently, Lange et al.19 demonstrated the impact of Iliad, a decision-support system designed to provide expert diagnostic consultations, in improving the diagnostic performance of NP students.19 In another instance, NPs were included as subjects along with other primary care providers in a study that demonstrated the impact of computer-based reminders on improved compliance with HIV/AIDS guidelines.20
Although we found no studies that specifically evaluate the impact of computer-based decision support on NP compliance with protocols or guidelines, it is known that NP use of paper-based protocols and guidelines is extensive.21,22,23 Moreover, nursing leadership groups have embraced the development and dissemination of clinical practice guidelines.24,25
Computer-based Support for Management of Diabetes Mellitus
A number of investigators have studied the provision of computer-based support for the management of diabetes mellitus. These include evaluating the terminology requirements for multidisciplinary care,26 achieving consensus related to the content of guidelines,27,28 engaging the patient in providing relevant data by means of an automated self-administered questionnaire,29 and Web-based architectures for therapy planning and revision.30 In addition, several recent studies31,32 have described approaches for temporal abstraction using the domain of diabetes mellitus.
McDonald et al.33 summarized the promise of computerized feedback systems for diabetes care and suggested that many facets of diabetes management could be facilitated by the adoption of open-loop control systems. Several studies have focused upon the provision of patient-specific advice related to a particular diabetes management guideline or standard of care. For example, in a randomized trial Overhage et al.5 demonstrated the impact of reminders for corollary orders associated with medication orders on the prevention of physicians' errors of omission. Orders for insulin and oral hypoglycemics were among the orders triggering corollary orders significant to diabetes management, such as glycosylated hemoglobin and capillary glucose. The compliance rate for the intervention group compared with that of the control group was 16.32 percent greater for glycosylated hemoglobin and 26.36 percent greater for capillary glucose.
Of particular relevance are the investigations that utilized computer-generated reminders for diabetic care delivered via a patient encounter form. Lobach and Hammond34 evaluated the effect of a computer-assisted management protocol (CAMP) on guideline compliance. Fifty-eight physicians were randomly assigned to receive or not receive the CAMP. Customized diabetes guideline recommendations were included on the encounter forms of patients receiving care from providers in the group receiving the CAMP. Based on minimal criteria for contact with diabetic patients, 16 physicians were assigned to the CAMP group and 14 to the control group. The CAMP group had a statistically significant greater median level of compliance (32.0 versus 15.6 percent, p =.02).
Nilasena and Lincoln35 reported the impact of a computer-generated reminder system on physicians' compliance with diabetes preventive care guidelines. Thirty-five resident physicians were randomized to the intervention or control group. Physicians in both groups completed the encounter forms used to collect patient-specific data. Health maintenance reports for diabetes mellitus based on patient-specific data were placed on the charts of patients being cared for by physicians in the experimental group. Guideline compliance significantly increased in both the intervention (38.0 versus 54.9 percent) and control (34.6 versus 51.0 percent) groups. There were no significant differences between the two groups. The investigators concluded that the system improved compliance with recommended care through the facilitation of the documentation of clinical findings and ordering of recommended procedures rather than through the provision of patient-specific reminders about guideline compliance status.
The review of the literature provides support for our approach in a number of areas. First, the findings of several studies suggest that strategies focused on the prevention of errors of omission have utility in improving guideline compliance.1,5,35 Second, the importance of delivering the reminders at the point of care—e.g., during the patient encounter or at the time of order entry—was noted by several investigators.5,34,35,36 Third, the findings of one study suggested that it was the facilitation of documentation and ordering of recommended procedures that improved guideline compliance in a small sample of resident physicians.35
System Description
Our approach builds on seminal work in the development of structured patient encounter forms that include reminders regarding protocols or guidelines.2,3,20,37,38 Our objective was to develop an online encounter form that incorporated guidelines related to assessments and interventions for an initial visit for diabetes mellitus. While our approach is based on structured encoded text organized into a documentation form or template like the one described earlier, it differs in three significant ways from the majority of guideline-related systems that have been tested in randomized controlled trials. First, the approach is centered on the facilitation of clinical documentation and the ordering of procedures according to a particular set of guidelines rather than to the provision of patient-specific reminders about guideline compliance. Second, the clinician documents the encounter directly on the computer system rather than annotating a printed encounter form that is subsequently processed. Third (and a significant limitation, as noted earlier), the approach does not include integration with a system that links an algorithm based on a specific guideline to patient-specific data.
In this section, we first provide a brief overview of the electronic health record in which our approach was implemented. Second, the approach is illustrated through the incorporation of ADA recommendations into a patient encounter form.
WAVE
WAVE is an object-oriented electronic health record.39 The version of WAVE installed in the faculty practice runs on the Microsoft Windows NT operating system and a Sybase database and is implemented in a networked environment consisting of a server and three clinical workstations. Patients are registered through a WAVE registration application.
The two general components of WAVE are the provider desktop and the patient chart. The provider desktop is the clinician's personal work area and is used to perform administrative functions related to patient charts (e.g., development of templates) and to tailor the environment to the individual's working style. The patient chart is the patient work area. It is used to review, add, and modify patient documents and to view and utilize information about specific patient topics. The patient chart for each patient includes five tabbed folders: profile, chart review, notes, results, and flowsheet.
Our approach for decision support related to clinical practice guidelines primarily utilizes the profile and notes components of the patient chart and the template manager components of the provider desktop. The profile summarizes clinically pertinent, patient-specific information derived from signed documents or obtained through interfaces to other systems. Two types of lists or views display information. The health summary comprises problems, procedures, medications, adverse reactions, and encounters. The health maintenance profile provides information such as preventive-care screening, the risk category of the patient, and patient education. The profile can be tailored by the user to suit the user's display preferences. As noted earlier, profile items are automatically posted from the note to the profile. In addition, items from the profile can be autocited from the profile into a specific patient document. For instance, one might choose to always autocite health maintenance items, medication allergies, and active medications into the current note. An additional functionality is the ability to easily access additional information about the profile item. When the user double-clicks on an item in the profile, the chart review folder appears. The chart review displays the item in full detail and provides a link to the original document or set of documents that contain references to the item.
Notes are created using structured, encoded text. Subsequently, each word or text string can be queried. The structured text within WAVE is built primarily on atomic-level terms and codes from SNOMED International.40 Additional structured text in WAVE is developed by content authors using proprietary content-authoring tools.
The structured text is organized into libraries that are linked with relevant types of notes (e.g., outpatient progress note, communication note) and sections of notes (e.g., review of systems, plan of care). The provider interacts with the structured text by using a hierarchic browser or a template that has been preconstructed from the browser. Browsers consist of terms organized into columns that progress from the more general to the more specific terms. Only terms relevant to the context of the current note type or note-section type appear in the browser.
At the most basic level, a word-processed document that is copied and subsequently edited could be considered a template, but in the context of WAVE we are referring to templates as documents built from structured text. A template can be created to represent an entire type of patient encounter (e.g., well-baby visit) or a particular section of a note (e.g., history and physical). Studies of WAVE have demonstrated that high-volume, well-structured activities are highly suitable to templates.41,42 Patient management according to specific clinical practice guidelines meets these criteria.
Templates can be created and edited within the provider desktop using the template manager. To use an existing template, the clinician selects the relevant template from the browser in a new document setup window in the patient chart area or utilizes a user-defined fast-path feature. The template manager also facilitates the location, copying, and renaming of templates. In addition, it allows the user to specify the location of the template within the set of browsers from which it will be accessed for future use.
Example of Diabetes Mellitus Template
Figures ▶ to ▶ demonstrate selections from an outpatient progress-note document template created in the WAVE template manager and based on the ADA recommendations for the components of the initial visit for diabetes mellitus.43 The figures illustrate three strategies we have found useful in template creation to support practice guidelines: exception editing, check boxes and blanks, and judicious use of free text. These three—along with another useful strategy, autociting from the profile—are described in the following paragraphs.
Exception Editing
▶ illustrates the use of exception editing for a portion of the diabetes management physical exam. The user-defined defaults for subjects (e.g., cardiovascular inspection), properties (e.g., clubbing), and values (e.g., none) are included in the template at the time of creation. When the provider wants to change a portion of the structured text, the relevant part of the hierarchic browser is opened by clicking on the text in the document. Notice that the column on the far right of the browser lists alternative values to “regular,” which is highlighted, for the purposes of exception editing. This strategy allows the pre-specification of guideline-recommended areas of examination along with the most probable values listed as defaults while making it extremely easy to edit the value when necessary.
Check Boxes and Blanks
A second strategy is the use of check boxes and blanks, as shown in the plan-of-care portion of the progress note (▶). Completion of the blanks is defined as required or not required when the template is created. The blanks associated with the dose of regular insulin and fasting blood sugar are examples of types of blanks that the template developer might choose to specify. When closing a note documented by means of the template, the provider has the option of leaving all the check boxes and related text visible or removing them so that only selected ones can be viewed. For example, in a physical exam or family history the user might choose to leave unchecked check boxes visible if they are associated with pertinent negatives, whereas in a plan of care unchecked procedures or medications left in the note might prove distracting to a clinician reviewing the document.
Free Text
▶ illustrates the judicious use of free text. On the WAVE screen such text is displayed using the color pink to differentiate it from structured text, but in the printed example shown here, an italicized font is used. The free text is used in three ways. First, the source of the guideline is noted. Because it is possible to access Web-based resources from within WAVE, the Web site address of the source is included so that the clinician can access the source by selecting it from the menu. Second, free text is used as a placeholder and reminder to document short-term and long-term goals for the patient. The goals can be typed in directly by the clinician or selected from a pre-specified set of goals. Third, free text is used to insert simple “if... then” statements in the plan of care according to ADA recommendations.
Free text may be added directly to the template or note within the WAVE application or pasted from another application such as a word-processing file. For instance, patient education materials or instructions from other sources may be included as part of the note.
Autociting from the Profile
The preventive care category of the health maintenance profile contains items that require interventions at regular intervals. Specific screening items can be added to the profile. The list of preventive care items reflects the status of each item, when an intervention was last done, and whether the finding was normal or abnormal. By using the autocite control in a template, the developer can designate that a particular section of the profile be autocited when a note is created. In our example (▶) autociting an ophthalmology consult in the diabetes management template would result in a listing of the occurrences and results of the consult for a specific patient when the template was opened to create a document for that patient. Thus, while a specific item would not be labeled as not compliant with a particular guideline, the findings of other investigators suggest that it would be relatively easy for the clinician to note the date of the last examination in relationship to the plan-of-care item that specifies that an ophthalmology consult should be ordered if there has been no eye exam in the last 12 months.35
Status Report
In this article we have provided evidence to support the feasibility of our approach and illustrated its implementation within the WAVE electronic health record. However, the specific template for the initial diabetes management visit has not yet been formally tested in the NP faculty practice. Earlier evaluations of WAVE, primarily related to the reduction of errors of omission, provide support for our approach. Douglas and Nubie42 reported that users provided more complete documentation of history and physicals when prompted by structured text terms within WAVE.42 Another investigation41 demonstrated the improvement in compliance with an immunization documentation standard. A prior study also supports the judicious use of free text in combination with structured text to afford the clinicians flexibility of expression and to provide an easy method for incorporating text that does not need to be easily retrieved for analytic purposes but does warrant documentation in the record.44
Our next steps will include activities focused on the evaluation of the diabetes mellitus template in the faculty practice as well as broader activities related to incorporation of guidelines into the practice. These activities include:
Working with clinicians to achieve consensus regarding the set of guidelines to be used in the practice;
Constructing templates based on the guidelines;
Selecting the most appropriate Web-based information resources to be linked with WAVE;
Evaluating the impact of the approach on selected process and outcome variables relevant for a particular guideline; and
Exploring funding opportunities to develop or purchase tools for inference that will support both the clinical process management and decision-support functions such as alerts and reminders.
Discussion
There are a number of advantages to the approach we have described:
It is simple and amenable to implementation in the absence of an inference engine and set of decision rules.
Appropriate practice guidelines can be totally integrated into a template to be used by the clinician at the point of care.
The time required to create a template is short. Simple templates can be created in less than five minutes, whereas more complex templates may require up to an hour.
The skill needed to develop a template is that of a clinician user; no specialized skills are required.
The guidelines can be easily updated within the templates in the template manager.
The source of the guideline can be clearly delineated within the template and a link provided to the Web address.
The clinician interaction with the template occurs online. Thus, the information is immediately available to others and there is no opportunity for errors to be introduced because of inaccurate data entry by clerical personnel.
It is simple for the clinician to personalize or tailor the guideline template by adding more structured or unstructured text (e.g., specific patient education material).
However, there are several limitations to our current approach relating primarily to the inability to do inference and the lack of integration with patient-specific data to trigger specific rules. These limitations relate to our approach, not the the WAVE electronic health record, which has been linked with clinical and business processing rules in other settings. Our approach supports only very simple “if... then” statements, so the complex branching algorithms associated with some practice guidelines cannot be represented. The difficulty of representing complex algorithms for computer-based systems has been identified by a number of authors,28,45,46 and our approach offers no solution to this problem or to the problems associated with maintaining complex rules. In addition, because of the lack of a rules engine, our approach requires the clinician to take extra cognitive steps in applying a particular guideline or standard of care, as noted in the previous discussion of autociting profile items. Our approach is relevant only for guidelines that are appropriately linked to patient encounters. Moreover, it does not address significant issues such as the consensus-building activities that precede implementation of a guideline28 or strategies for making generic guidelines site-specific.47
Practice guidelines are an integral part of evidence-based health care delivery. Multiple strategies are needed to increase the provision of care that complies with guidelines.6,36,38,47,48 In this article, we have focused on only one component of what is needed— the provision of guidelines to the clinician at the point of care. In particular, we have shared an approach devised for a computing environment in which online clinical documentation is not linked to other systems. Clinical settings are disparate in the types of information technology available. The strategies designed for guideline implementation must be well matched to the level of technology available in the environment. Thus, others with a similar computing environment may find our preliminary work useful in designing approaches and strategies for their clinical settings.
Acknowledgments
The authors thank Keshia Barnes and Angela Ornelas for their technical assistance in system implementation and David Goldschmid, MD, who authored the template that was adapted for this manuscript.
This work was supported by grant NR03874 from the National Institute of Nursing Research.
This paper was presented in an earlier version at the International Medical Informatics Association (IMIA) Nursing Informatics '97 Working Conference, “Informatics and Patient and Clinical Guidelines: The State of Our Knowledge and a Vision,” Stockholm, Sweden, October 2-4, 1997.
References
- 1.McDonald CJ. Protocol-based reminders, the quality of care, and the non-perfectibility of man. N Engl J Med. 1976;295: 1351-5. [DOI] [PubMed] [Google Scholar]
- 2.McDonald CJ, Hui SL, Smith DM, et al. Reminders to physicians from an introspective computer medical record: a two-year randomized trial. Ann Intern Med. 1984;100: 130-8. [DOI] [PubMed] [Google Scholar]
- 3.Barnett GO, Winickoff RN, Morgan MM, et al. A computer-based monitoring system for follow-up of elevated blood pressure. Med Care. 1983;21(4): 400-9. [DOI] [PubMed] [Google Scholar]
- 4.Burack RC, Gimotty PA, George J, et al. Promoting screening mammography in inner-city settings: a randomized controlled trial of computerized reminders as a component of a program to facilitate mammography. Med Care. 1994;32(6): 609-24. [DOI] [PubMed] [Google Scholar]
- 5.Overhage JM, Tierney WM, Zhou X, et al. A randomized trial of “corollary orders” to prevent errors of omission. J Am Med Inform Assoc. 1997;4(5): 364-75. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.McPhee SJ, Bird JA, Jenkins CNH, et al. Promoting cancer screening: a randomized, controlled trial of three interventions. Arch Intern Med. 1989;149: 1866-72. [DOI] [PubMed] [Google Scholar]
- 7.McDonald C, Hui S, Tierney WM. Effects of computer reminders for influenza vaccination on morbidity during influenza epidemics. MD Comput. 1992;9: 304-12. [PubMed] [Google Scholar]
- 8.Chambers CV, Balaban DJ, Carlson BL, et al. Microcomputer-generated reminders: improving compliance of primary care physicians with mammography screening guidelines. J Fam Pract. 1989;29(3): 273-80. [PubMed] [Google Scholar]
- 9.Johnson ME, Langton KB, Haynes RB, et al. Effects of computer-based clinical decision support systems on clinician performance and patient outcome: a critical appraisal of research. Ann Intern Med. 1996;120(2): 135-42. [DOI] [PubMed] [Google Scholar]
- 10.Shea S, DuMouchel W, Bahamonde L. A meta-analysis of 16 randomized controlled trials to evaluate computer-based clinical reminder systems for preventive care in the ambulatory setting. J Am Med Inform Assoc. 1996;3(6): 399-409. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Zielstorff RD. On line practice guidelines: issues, obstacles, and future prospects. J Am Med Inform Assoc. 1998;3: 227-36. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Lemley KB, O'Grady ET, Rauckhorst L, et al. Baseline data on the delivery of clinical preventive services provided by nurse practitioners. Nurse Pract. 1994;19(5): 59-61. [PubMed] [Google Scholar]
- 13.Griffith HM, Rahman MI. Implementing “put prevention into practice” program. Nurse Pract. 1994;19(10): 12-9. [DOI] [PubMed] [Google Scholar]
- 14.Martin K. Nurse practitioners' use of nursing diagnosis. Nurs Diagn. 1995;6(1): 9-15. [DOI] [PubMed] [Google Scholar]
- 15.Aiken LH, Lake ET, Semaan S, et al. Nurse practitioner managed care for persons with HIV infection. Image J Nurs Sch. 1993;25: 172-7. [DOI] [PubMed] [Google Scholar]
- 16.Brykczynski KA. An interpretive study describing the clinical judgment of nurse practitioners. Sch Inq Nurs Pract. 1989;3(2): 75-104. [PubMed] [Google Scholar]
- 17.Brown SA, Grimes DE. Nurse Practitioners and Certified Nurse-Midwives: A Meta-analysis of Studies on Nurses in Primary Care Roles. Washington, DC: American Nurses Association, 1993. [PubMed]
- 18.Genet CA, Brennan PF, Ibbotson-Wolff S, et al. Nurse practitioners in a teaching hospital. Nurse Pract. 1995;20(9): 46-54. [PubMed] [Google Scholar]
- 19.Lange LL, Haak SW, Lincoln MJ, et al. Use of Iliad to improve diagnostic performance of nurse practitioner students. J Nurs Educ. 1997;36(1): 36-44. [DOI] [PubMed] [Google Scholar]
- 20.Safran C, Rind DM, Sands DZ, et al. Development of a knowledge-based electronic patient record. MD Comput. 1996;13(1): 46-54, 63. [PubMed] [Google Scholar]
- 21.Star W, Shannon MT, Sammons LN, et al. Ambulatory Obstetrics: Protocols for Nurse Practitioners/Nurse-Midwives. San Francisco, Calif: UCSF Nursing Press, 1990.
- 22.Marshall J. Protocols and emergency nurse practitioners. Nurs Times. 1997;93(14): 58-9. [PubMed] [Google Scholar]
- 23.Robinson ME, Goodrich M. Clinical acute sinusitis. J Pediatr Health Care. 1995;9(3): 136-8. [DOI] [PubMed] [Google Scholar]
- 24.Porter CP, Pender NJ, Hayman LL, et al. Educating APNs for implementing the guidelines for adolescents in Bright Futures: guidelines of health supervision of infants, children, and adolescents. Nurs Outlook. 1997;45(6): 252-7. [DOI] [PubMed] [Google Scholar]
- 25.Marek KD. Manual to Develop Guidelines. Washington, DC: American Nurses Publishing, 1995.
- 26.Brown PJB, Price C, Sonksen PH. Evaluating the terminology requirements to support multi-disciplinary diabetes care. Proc AMIA Annu Fall Symp. 1997: 645-9. [PMC free article] [PubMed]
- 27.Nilasena DS, Lincoln MJ, Turner CW, et al. Development and implementation of a computer-generated reminder system for diabetes preventive care. Proc 18th Annu Symp Comput Appl Med Care. 1994: 831-5. [PMC free article] [PubMed]
- 28.Lobach DF. A model for adapting clinical guidelines for electronic implementation in primary care. Proc 19th Annu Symp Comput Appl Med Care. 1995: 581-5. [PMC free article] [PubMed]
- 29.Hunt DL, Haynes RB, Hayward RSA, et al. Automated direct-from-patient information collection for evidence-based diabetes care. Proc AMIA Annu Fall Symp. 1997: 81-5. [PMC free article] [PubMed]
- 30.Riva A. A Web-based architecture for intelligent management of chronic patients. Proc AMIA Annu Fall Symp. 1996: 179-83. [PMC free article] [PubMed]
- 31.Bellazzi R, Larizza A, Riva A, et al. Distributed intelligent data analysis in diabetic patient management. Proc AMIA Annu Fall Symp. 1996: 194-8. [PMC free article] [PubMed]
- 32.Shahar Y, Das AK, Tu SW, et al. Knowledge-based temporal abstraction for diabetes monitoring. Proc 18th Annu Symp Comput Appl Med Care. 1994: 697-701. [PMC free article] [PubMed]
- 33.McDonald CJ, Overhage JM, Tierney WM, et al. The promise of computerized feedback systems for diabetes care. Ann Intern Med. 1996;124(1): 170-4. [DOI] [PubMed] [Google Scholar]
- 34.Lobach DF, Hammond WE. Development and evaluation of a computer-assisted management protocol (CAMP): improved compliance with care guidelines for diabetes mellitus. Proc 18th Annu Symp Comput Appl Med Care. 1994: 787-91. [PMC free article] [PubMed]
- 35.Nilasena DS, Lincoln MJ. A computer-based reminder system improved physician compliance with diabetes preventive care guidelines. Proc 19th Annu Symp Comput Appl Med Care. 1995: 640-5. [PMC free article] [PubMed]
- 36.Tierney WM, Hui SL, McDonald CJ. Delayed feedback of physician performance versus immediate reminders to perform preventive care: effects on physician compliance. Med Care. 1986;24(8): 659-66. [DOI] [PubMed] [Google Scholar]
- 37.McPhee SJ, Bird JA, Fordham D, et al. Promoting cancer prevention activities by primary care physicians: results of a randomized, controlled trial. JAMA. 1991;266(4): 538-44. [PubMed] [Google Scholar]
- 38.Turner BJ, Day SC, Borenstein B. A controlled trial to improve delivery of preventive care: physician or patient reminders? J Gen Intern Med. 1989;4: 403-9. [DOI] [PubMed] [Google Scholar]
- 39.Billings E, Kushinka S, Douglas K, et al. An object-oriented record: maximizing benefits to support the use of controlled terminology. Proc AMIA Annu Fall Symp. 1996: 972.
- 40.Côté RA, Rothwell DJ, Palotay JL, Beckett RS, Brochu L. The Systematized Nomenclature of Human and Veterinary Medicine: SNOMED International. Northfield, Ill: College of American Pathologists, 1993.
- 41.Crist-Grundman D, Douglas K, Kern V, et al. Evaluating the impact of structured text and templates in ambulatory nursing. Proc 19th Annu Symp Comput Appl Med Care. 1995: 712-6. [PMC free article] [PubMed]
- 42.Douglas K, Nubie M. System design challenges: the integration of controlled vocabulary use into daily practice. In: Gerdin U, Tallberg M, Wainwright P (eds). Nursing Informatics: The Impact of Nursing Knowledge on Health Care Informatics. Stockholm, Sweden: IOS Press, 1997: 167-71. [PubMed]
- 43.American Diabetes Association Web site. Available at: http://www.diabetes.org/diabetescare/supplement197/s5.htm. Accessed November 5, 1997.
- 44.Cork RD, Morris JA, Kern VC, et al. Structured data entry: what menu choices do users select, and why? Paper presented at Medical Record Institute Meeting: Towards an Electronic Patient Record. 1997. Nashville, Tenn.
- 45.Liu JCS, Shiffman RN. Operationalization of clinical practice guidelines using fuzzy logic. Proc AMIA Annu Fall Symp. 1997: 283-7. [PMC free article] [PubMed]
- 46.Shiffman RN, Greenes RA. Improving guidelines with logic and decision table techniques. Medical Decis Making. 1994;14: 245-54. [DOI] [PubMed] [Google Scholar]
- 47.Fridsma DB, Gennari JH, Musen MA. Make generic guidelines site-specific. Proc AMIA Annu Fall Symp. 1996: 597-601. [PMC free article] [PubMed]
- 48.Ornstein SM, Garr DR, Jenkins RG, et al. Computer-generated physician and patient reminders: tools to improve population adherence to selected preventive services. J Fam Pract. 1991;32(1): 82-90. [PubMed] [Google Scholar]