Abstract
Background
Improving health in our nation requires strengthening four major domains of the health care system: personal health management, health care delivery, public health, and health-related research. Many avoidable shortcomings in the health sector that result in poor quality are due to inaccessible data, information, and knowledge. A national health information infrastructure (NHII) offers the connectivity and knowledge management essential to correct these shortcomings. Better health and a better health system are within our reach.
Discussion
A national health information infrastructure for the United States should address the needs of personal health management, health care delivery, public health, and research. It should also address relevant global dimensions (e.g., standards for sharing data and knowledge across national boundaries). The public and private sectors will need to collaborate to build a robust national health information infrastructure, essentially a 'paperless' health care system, for the United States. The federal government should assume leadership for assuring a national health information infrastructure as recommended by the National Committee on Vital and Health Statistics and the President's Information Technology Advisory Committee. Progress is needed in the areas of funding, incentives, standards, and continued refinement of a privacy (i.e., confidentiality and security) framework to facilitate personal identification for health purposes. Particular attention should be paid to NHII leadership and change management challenges.
Summary
A national health information infrastructure is a necessary step for improved health in the U.S. It will require a concerted, collaborative effort by both public and private sectors.
If you cannot measure it, you cannot improve it. Lord Kelvin
Background
Better health in the 21st century – it is what we seek for our families, our patients, our neighbors, our communities, our nation, and indeed people all over the world. It is a matter of quality of life for individuals, stability and happiness within communities, productivity for industry, security for nations, and professional satisfaction for heath workers. Maintaining and improving health is not an abstract notion. We already know much about where and how we fall short in assuring health. We also know the potential to improve health grows daily as result of the steady flow of research advances.
The health of individuals and the population depends on four major domains of our vast, complex, and disorganized health sector. To improve health, we must strengthen and integrate four cornerstones of our health system:
• Personal health management: citizens/patients equipped to manage their personal health
• Health care delivery: a health care delivery system that meets evidence-based expectations for safety, quality, cost, and access
• Public health: a public health system capable of monitoring, promoting, and protecting the health and safety of the total population and subpopulations, and
• Research: a biomedical, health services, and health policy research infrastructure robust enough to assure continued development of knowledge through primary discovery and in response to clinical and public health insights.
The health sector's most avoidable shortcomings can be linked to data, information, or knowledge that are inaccessible or demonstrate poor quality. Lost data, poor documentation, lack of access to available knowledge, and reliance on memory all impede the delivery of high quality health care services. [1,2] Public health agencies lack the ability to share critical information quickly and encounter substantial difficulties when attempting to pool existing data for analysis. [3,4] Advances in medical knowledge and treatment capabilities often take too many years to reach patients; many therapeutic interventions in use are not supported by evidence of effectiveness. [5,6] Practice patterns differ across institutions and regions, resulting in varying health outcomes and costs of care. [7] Patients trying to make informed health decisions often encounter conflicting information with varying degrees of quality. [8,9] And, care delivery is often extraordinarily wasteful of patients' time.
The health sector has begun to unleash the transformational power of information and communications technology (ICT). The field of health informatics focuses on using computers to improve health through management of the knowledge base and improved decision support. Bioinformatics (i.e., biology linked to computer science) enabled the success of the Human Genome Project and supports genomics, proteomics and correlations with phenotypic data, (e.g. personal disease and treatment data available from medical records). Information and communications technology is improving the ability of patients to manage their own health information and decisions and, on a limited but growing scale, improving communication between clinicians and patients. [10] Some health care delivery organizations have already succeeded in implementing an information infrastructure and have demonstrated that improved access to knowledge for clinicians improves the quality and/or safety and efficiency of health care services. [11-14]
Much more remains to be achieved and the benefits of ICT must be consistently applied across the health system through the development of a national health information infrastructure (NHII). A NHII is the means by which we can improve the quality of health data, information, and knowledge used to support decisions at all levels and in all domains of the health sector (i.e., personal health, health care delivery, public health, and research). A NHII will lead to essentially 'paperless' health care. While paper will still be used, the default system for communication and system memory will be computer-based and not paper-based. [15]
The connectivity and knowledge management capabilities provided by a NHII offer myriad ways of making progress toward better health. For example, a NHII can support sound decision making by clinicians and patients, flag health threats to localities, enable citizens to receive laboratory results promptly and reliably, allow clinicians to monitor disease and coach patients with chronic conditions between appointments, transform individual data elements into pools of anonymous data for research and public health needs, allow researchers from around the country to collaborate without leaving their labs, link a new medical advance to an individual patient, speed new useful knowledge to clinicians, and automate routine tasks so that chances of human error are greatly reduced. When successfully implemented, its impact on human health may rival or exceed that of sanitation, antibiotics, vaccines, and other major medical advances. Thus, a NHII is a necessary step for an improved health system and improved health of U.S. citizens in the 21st century. The NHII is not, however, a panacea and its development must be accompanied by parallel work to remedy the other impediments to optimal performance within the health sector such as needed redesign of care processes, care financing, reimbursement incentives that do not encourage undesired behaviors, and new skills needed by patients and the workforce. [2]
Using a framework based on the four major domains of personal health management, health care delivery, public health, and research, this article describes the ICT infrastructure that is needed to bring the health sector from a memory-based non-system to a computer-assisted integrated care system. It argues that investment in a NHII should be a high priority for both the public and private sectors and outlines first steps that will lead to creation of a robust NHII in the U.S.
Discussion
Defining the NHII
An information and communications infrastructure exists to connect users – to each other, to information, and to analytical tools – and to enable management and generation of knowledge. Connectivity is achieved through a combination of technology, standards for transmitting data, and agreed upon rules and processes. A NHII for the United States would connect the multitude of participants in the health sector who interact on a routine basis and provide the means for managing the massive volumes of health data, information, and knowledge that are increasing by the hour. When fully implemented, the NHII would also enable automation of routine tasks, simplification of complex tasks, democratization of functions, customization of services, management of the knowledge base, and greater collaboration across the domains of the health sector.
The concept of a NHII for the United States began to emerge about 15 years ago. The initial concept limited its scope to use by health professionals. A 1986 planning panel helping to develop a long-range plan for the National Library of Medicine (NLM) urged NLM to work towards "a national computer network for use by the entire biomedical community, both clinical and research professionals." [16] The 1991 Institute of Medicine (IOM) report on computer-based patient records included a vision of a "national health care information system" that would support data transfer for clinical purposes, reimbursement, and research, with appropriate confidentiality measures, and would bring knowledge resources to practitioners. Such an information system would "support the coordination and integration of health care services across settings and among providers of care."[17] During the past decade, individuals and groups within and outside government addressed the issue of a NHII for the United States. [4,18-26] These deliberations were informed by changes in the health sector, the dramatic advances in and widespread use of information technology, experiences gained by other industries and individual health organizations implementing information technology, and plans and development of health information infrastructures in other countries, especially Australia, Canada, and England. [27-30]
The National Committee on Vital and Health Statistics (NCVHS) articulated the current framework for the NHII in the U.S. in a report to the Secretary of Health and Human Services. [24] This framework is based on three interlocking dimensions that correspond to the domains of personal health, health care delivery, and public/population health. The NCVHS defined the NHII as
the set of technologies, standards, applications, systems, values, and laws that support all facets of individual health, health care, and public health. The broad goal of the NHII is to deliver information to individuals - consumers, patients, and professionals – when and where they need it, so they can use this information to make informed decisions about health and health care. [[24], p. 1]
Each component of the NHII provides analogous functions to its users – data capture, storage, communication, processing, and presentation of information – although not necessarily in the same way given their differing needs. Within the NHII, some data will be common to all three dimensions, some shared between two dimensions, and some unique to a particular dimension.
The personal health dimension includes a personal health record that is created and controlled by the individual or family, plus non-clinical information such as self-care trackers and directories of health care providers. The health care delivery dimension includes information such as provider notes, clinical orders, decision-support programs, digital prescribing programs, and practice guidelines. Both these dimensions contain information on individual patients such as health history, health insurance, and medication alerts. The personal health, health care provider, and public health dimensions can share tools to enable improved clinical management of populations of patients such as vital statistics, population health risks, and disease registries. All three dimensions will have access to community directories and survey data.
Discussion of the NHII in the U.S. has focused on the three domains of personal health, health care delivery, and public health. To achieve the full potential of the NHII, however, we must view it as a framework for supporting use of existing information and knowledge in decision-making and as a driver in the generation of new knowledge and improved system performance. The informatics and communications technology needs of all relevant stakeholders in the health sector – patients, citizens, practitioners, public health officials, managers, policy-makers, and researchers – must be addressed in the NHII. If not, we risk losing the benefits of collaboration among the domains and we are unlikely to be prepared for emerging practices (e.g., personal genomics or so-called individualized' medicine) that stand at the nexus of research and clinical practice. Just as the distinctions among some basic sciences and clinical specialties have begun to blur, we can expect that there will be ever increasing interdependence among the various sets of users. Thus, in addition to supporting personal health, health care delivery, and public health, the NHII must include a robust research component. The President's Information Technology Advisory Committee (PITAC) supports this perspective; its report strongly emphasizes meeting the information infrastructure needs of the medical research complex as well as the three other domains. [25]
Leveraging resources (particularly ideas, innovations, information, and knowledge) among these domains will enable stronger performance within each of them. Thus, there must be a technological architecture and a learning culture that promote easy communication among players, facile data sharing, and efficient knowledge diffusion. All the major components must be capable of functioning in such a way that they can inform one another to improve health care management and health status (e.g., system architecture and data standards must support communication and knowledge development across the domains). Support for education and a learning culture must be an integral part of the strategy. Efforts must be explicitly directed to support constructive change through improved communications and coordination.
In addition, NHII development in the U.S. has several global dimensions. First, there is significant potential to learn from those countries that are ahead of the U.S. in NHII planning and development. Second, data content and format standards must cross national boundaries. The international use of standards will facilitate global public health surveillance, enable clinical research across national boundaries, and allow comparisons of health system performance in different countries. Third, medical knowledge is one of the most important resources that the U.S. provides to other nations. Additional funding can assure that the already prodigious achievements of the NLM will guarantee global access to health-related literature in a variety of languages and cultural contexts. Progress towards establishing a globally shared vocabulary and reference architecture will be pivotal to this goal.
Why should the U.S. build a NHII?
Some observers are wary of the significant hurdles to and substantial costs associated with building the NHII and do not support a large-scale effort to create a robust NHII. They may be inclined to rely on the existing information resources and allow the marketplace to fill gaps over time. Or, they may be concerned that such an approach would be too rigid and thwart desired innovation. Or, they are concerned that if successful it will be too insensitive to privacy considerations. These views are relevant but ultimately are shortsighted. Just as buildings and other forms of physical capital were essential to operating in previous centuries, an information infrastructure is an essential tool for any enterprise in the 21st century, particularly an information intensive industry such as health care. To bring the health sector into the knowledge age so that the full value of data, information, and knowledge is used in real time to support care delivery and other health goals, an integrated infrastructure allowing interoperability and scalability is essential. Further, only an infrastructure that is well conceived, adequately funded, and effectively deployed and maintained can achieve the benefits while at the same time avoid more of the unavoidable limitations.
There are at least three reasons why the public and private sectors should collaborate to build a NHII in the U.S. First, a NHII is the only means by which we can possibly manage the exponential growth of health data, information, and knowledge. We are compiling more data about patients in order to care for their health needs as they live longer and experience more chronic illness, as diagnostic capabilities have improved and we conduct more tests, and as we broaden the kinds of data we collect (e.g., genomic, environmental exposures, alternative medications, and occupational experiences). Researchers similarly face larger and more complex data sets. For example, searching for new drugs through genomics and proteomics requires "the gathering and sifting of oceanic volumes of data about molecules and their reactions to one another." Vertex Pharmaceutical "simulates 47 billion reactions between drugs and proteins a day – nearly as many as the number of emails sent out in the world each week." [31]
Administrative information used in the health sector changes on a daily basis. Patients and clinicians must keep pace with an ever-changing set of reimbursement and other administrative rules. Health care organizations must track regulations and guidelines from a multitude of sources. Researchers face an evolving set of rules that regulate their research including their ability to access and sensitively deal with person-specific data.
The health care knowledge base is expanding and being refined continually, making it increasingly difficult for patients and health professionals in all domains to determine what constitutes the most reliable knowledge on which to base decisions. These decisions range from selecting treatments for individual patients, to determining clinical protocols for health delivery organizations, to choosing which procedures and pharmaceuticals will be covered by third-party payers, to planning clinical trials and to allocating public health resources. Access to current reliable knowledge is a key determinant in the behavior of patients and the performance of health care professionals, organizations, and the system as a whole.
The NHII will enable knowledge to be used more effectively within each of the four domains. For example, a recent study by the Dartmouth Center for Evaluative Clinical Sciences reinforces the need for stronger reliance on medical evidence in clinical practice and the financing of health care services. The study found that Medicare spending per person varies more than twofold among regions of the country. These variations are not due solely to differences in health and do not result in more effective care or better health outcomes. Use of treatments that are supported by clinical evidence varies extensively among hospital referral regions. The researchers concluded that the wide variation in effective care use is due to "the lack of infrastructure to ensure compliance with well-accepted (evidence-based) standards of practice". [7] Eliminating regional disparities would yield an estimated $40 billion that could be reallocated to provide needed benefits (e.g., prescription drugs) or to support the infrastructure needed to support evidence-based medicine.
This potential for improvement cannot be left to chance. For knowledge to have maximum impact on the health of patients, it must be organized, accessible, and integrated into the workflow of clinicians. Robust decision support systems are a pivotal tool for moving knowledge into routine practice and an important component of a NHII.
Second, the NHII will enable data, information, and knowledge to be shared across domains on a right-to-know and need-to-know basis. Health care delivery data already routinely supports health services research, but often requires cumbersome, inefficient, and costly data manipulation to do so. Reduced research costs and turnaround times will likely result from greater data standardization.
Data from personal records can answer public health questions. For instance, the Real-time Outbreak and Disease Surveillance System (RODS), developed at the University of Pittsburgh, is a functioning prototype of how real-time clinical data from emergency departments within a geographic region can be integrated to provide an instantaneous picture of symptom patterns and early detection of epidemic events. [32] Moreover, public health/population data are needed to improve clinical delivery systems. For example, one typically would consider evaluating the quality of patient care for diabetics by studying the records of those receiving care. However, it is equally and possibly more important to know the predicted prevalence of diabetes in a community with a given set of demographics since these data reveal how the number of diabetics being treated compares with those within the general population who are not receiving care.
Third, a basic technological platform is needed to assure that the benefits of ICT diffuse to all health care organizations. The current information infrastructure is incomplete, unevenly developed, and not designed to achieve synergies across the domains of the health sector. Within the clinical domain, the level of diffusion and effective use varies widely. Although an increasing number of institutions rely on an information infrastructure to practice evidence-based medicine and to refine continually the base of medical knowledge, other organizations do not yet use electronic mail routinely. In some cases, individual units within health care organizations are successfully using information technology, but the enterprise as a whole lags behind.
The research domain has specific needs for high performance computing. In time, high-performance computers almost certainly will analyze clinical care patterns within large clinical databases and determine the most crucial factors of organization of care for optimal disease management. Outcomes of such research should lead to the creation of highly robust navigation care maps for clinical decision-support embedded within computer-based patient records. [9]
Personal genomics – the analysis of the gene maps of individuals "to diagnose risks and states of illness and to plan therapy" – is already in limited use. [33] For such work to reach its zenith, researchers need large datasets (phenotypic data) that comprise histories of health and illness that can be linked to emerging knowledge about the molecular underpinnings of disease. To meet these and other research challenges, there is a need for intensive computing capability; tools to sort, organize, and display the vast amount of data in an integrated way; new algorithms to reconcile various vocabularies; and a robust communications infrastructure to allow colleagues to exchange information and to access the same databases (e.g., bandwidth requirements for biomedical research applications often exceed current availability). Further, research in the post-genomic era will depend on availability of scientists trained to "bridge the gap" between information technology, engineering, and molecular biology. [34]
Today, public health officials often have limited electronic communication capacity, data systems, and other informatics tools. [3] Recent bioterrorism threats have highlighted the need for a robust, well-conceived, state-of-the art technology, information and communications infrastructure to support bio-defense response and preparedness. [35] Yet even without these threats, public health professionals would benefit from ICT resources that enable them to integrate data, produce high quality community-level data, identify significant health trends in real time, display geographic information graphically, and support participation and collaboration at the community level in health promotion. [35-37]
Personal health management is the youngest of the four domains and is the least developed in terms of an information infrastructure. It will ultimately depend on patient and citizen access to ICT, health care professionals willing to interact with patients in new ways, education for both patients and professionals, and a supportive environment. Successful developers in this area have begun to demonstrate that while many patients will access non-interactive websites only infrequently ("clicks" alone), they are quite enthusiastic about securing well designed personal health management services that link them to their providers ("clicks and mortar"). [38,39] In addition to e-mailing their providers for the answers to questions, additional services include automated prescription refills, results reporting, appointment scheduling, and support relating to administrative questions. A range of relevant questions needs to be addressed before this field matures fully but one can predict that this domain will eventually emerge as a central dimension of a 'patient-centered' care system.
Who should build the NHII?
The NHII should be built through a combination of public and private effort and investment. The federal government must assure that the backbone of the NHII (i.e., standards, laws, and a communications network) is developed with appropriate participation by the private sector and that mechanisms and funding are in place to manage its content (i.e., data and knowledge) over time. Private sector organizations, localities, and states must provide input to NHII planning and assure that they have the information and communications systems (i.e., equipment and applications) and skills to use the NHII backbone to access public information and knowledge resources, manage their own data, and contribute data to public health and research databases. Individuals and organizations in both sectors must attend to the significant change management challenges that will arise as the NHII evolves and offers opportunities to transform health care delivery in the United States. Hence, leadership and change management must be an incorporated into all NHII planning and implementation.
As a first step, the federal government must assume explicit leadership and responsibility for NHII development. There are three reasons why the federal government must take the lead on NHII development. First, the NHII is a public good and its benefits will accrue to many parties and across generations rather than solely to those who invest in it. This is why other governments around the world are investing in a NHII for their nations. In the United States, public health and research have long received federal investment because of the public goods they produce. Similarly, the federal government played a pivotal role in the building both the interstate highway system and the Internet.
Second, the federal government has a substantial stake in all of the domains to be supported by the NHII. For example, the current investment of tax dollars in the Medicare program alone is a huge expenditure that will clearly be a major beneficiary of NHII development. The sooner a robust NHII is in place, the sooner a more sensible allocation of scarce resources can be realized and high administrative costs can be shrunk. Moreover, bioterrorism concerns have reinforced the need for a robust public health ICT infrastructure.
Third, experience teaches us that the private sector does not have sufficiently centralized power or the resources to lead an integrated NHII effort. Today, most health care organizations are struggling to meet basic budget requirements and have a limited capacity to invest in information technology. As a result their ITC investments tend to be highly focused on meeting internal demands and not system-wide connectivity. Despite considerable efforts by many talented and dedicated individuals, progress on cross cutting information technology issues in the health sector has been slow. Implementing the standards setting components of the Health Insurance Portability and Accountability Act (HIPPA) is an acknowledgement of the challenges associated with setting standards. Only the federal government has the resources to lead NHII development. It is well positioned to lead an integrated planning and implementation effort, allocate sufficient resources, create appropriate incentives, and address longstanding barriers. It can develop the nationally coordinated approach to mobilize the public and private sector to foster best practices for safety, efficiency, effectiveness, timeliness, equitable, and patient-centered care.
A series of publications have advocated the need for a serious national initiative. In November 2000, a leader of the medical informatics community called for "strategic, cross-agency leadership in IT" within the Department of Health and Human Services (DHHS). [40] The National Research Council reached a similar conclusion in Networking Health when it called on DHHS to "more aggressively address the broad set of policy issues that influence the development, deployment, and adoption of Internet-based applications in the health sector." [[23], p. 264] Two reports on the NHII – Information for Health by the National Committee on Vital and Health Statistics (NCVHS) and Transforming Health Care through Information Technology by the President's Information Technology Advisory Committee (PITAC) Panel on Health – specifically call for the federal government to lead a national effort to build the NHII. [24,25] Most recently, an IOM report on building blocks for health system change noted that "federal leadership will be needed to move forward" in many areas of NHII development. [[15], p. 60]
The NCVHS called for "a new senior position" within DHHS with the "resources and mandate to coordinate all efforts for the NHII, internally and externally and in both public and private sectors." [[26], pp. 3–4] The vision is much like the High Performance Computer Center, a virtual agency of the mid-1990s. The goal is not to run the development but have sufficient budget and authority to assure that key pieces are built and interact appropriately. Recently, one well-qualified person has been reassigned to this agenda in the office of Secretary of DHHS. The relatively new Consolidated Health Informatics Initiative (CHI) is striving to utilize common standards across health-related agencies such as DHHS, the Veterans Administration, and the Food and Drug Administration. While these are steps in the right direction, substantial staff support and a dedicated budget are needed to demonstrate meaningful commitment from DHHS and assure needed progress.
Much of the experience of and knowledge on implementing ICT in health organizations resides in the private sector. There must be active participation from throughout the private sector at all stages of NHII planning and implementation. The NCVHS report on NHII development calls for action by a wide range of stakeholders, including state and local governments, health care provider organizations, health care provider membership and trade organizations, health care plans and purchasers, standards development organizations, the information technology industry, consumer and patient advocacy groups, community organizations, and academic and research organizations. [24] This participation may take the form of responsibility for providing input to or developing key sections of the NHII plan or building certain NHII components on behalf of the government.
Several recent initiatives demonstrate the private sector's growing interest in the NHII. In May 2001, the e-Health Initiative (eHI), a consortium of 100 health care organizations, was launched. [41] Its goals include laying the foundation for "an interconnected, electronic health information infrastructure by promoting the adoption of clinical data standards and interoperability." [41] One of eHI's current efforts focuses on accelerating adoption of electronic prescribing in the ambulatory care environment.
In June 2002, the Markle Foundation, a private not-for-profit philanthropy that seeks to advance ICT, launched a public-private collaborative, "Connecting for Health," to "catalyze specific actions on a national basis that will rapidly clear the way for an interconnected, electronic national health information infrastructure."[42] The collaborative focuses on three key areas: accelerating adoption of national clinical data standards, identifying practical strategies that will ensure secure transmission of medical information, and working to understand consumer needs and expectations for an interconnected health information system. Also in June 2002, the American Hospital Association along with 29 other organizations announced the creation of the National Alliance for Health Information Technology (NAHIT) to "improve quality and performance through standards-based information systems." [43] The NAHIT will create and implement distinct projects that will contribute to the development of a viable health information infrastructure. It has begun its work by focusing on standardized bar codes on products used by health care organizations.
In October 2002, the Healthcare Information and Management Systems Society (HIMSS) announced creation of a NHII Task Force. [44] This task force also seeks to help the health care industry build a NHII and will begin its work with an inventory of existing technologies and practices in health care. These groups should help assure private sector partnership among key players in order to relate constructively with the government. Given the magnitude of effort and resource required to implement a NHII, all of these initiatives will be beneficial if they can coordinate their activities to give sufficient attention to relevant dimensions while minimizing duplication of effort.
At the organizational level, individual health organizations and professionals must implement robust institutional information systems that connect to the NHII. This will require leadership, time, and financial investment as organizations grapple with the cultural, behavioral, organizational, and technical challenges of implementing or enhancing information technology within their institutions. The change management challenge for organizations and individual professionals should not be underestimated. Organizations that involve their staff in ICT planning, fully consider operational implications of ICT developments, acknowledge and address outdated organizational habits and structures that impeded ICT implementation, and provide adequate support during transitional periods will be more likely to reap the benefits of ICT systems. Health care organizations will be well served if they collaborate with vendors to design and implement information and knowledge management systems that meet the needs of patients, clinicians, public health professionals, and researchers. Vendors face the particular challenge of improving the interoperability and performance of health information systems. [45,46]
Key NHII enablers
The U.S. health care sector is an information intensive industry that is under-invested in ICT. [47,48] Although there are signs that investment is increasing, wider diffusion of ICT in the health sector must be assured as part of NHII development. [26] Lack of ICT investment by health care organizations is due to a diverse set of technological, financial, legal, cultural, personnel, and leadership issues. [17,25,26,45,46] Work on four areas in particular – incentives, standards, a privacy (i.e., confidentiality and security) framework, and funding – will help to convert these impediments into enablers of the NHII.
Incentives
In the clinical domain, limiting investment in ICT has been a rational choice for health care organizations. Reimbursement mechanisms and consumer demand have not created a strong enough incentive for health organizations to bear the financial costs and confront the non-financial obstacles to ITC implementation. Under fee for service, reimbursement typically rewards production volume rather than quality or outcomes. In some instances, the benefits of ICT or other quality enhancing practices contribute to lower costs of care, better patient outcomes, and higher patient satisfaction but cause health care organizations to lose disproportionate revenue because they reduce office visits. Patient demand is often inhibited by choices available through their insurance. Even when patients are aware of organizations that offer high quality care, they cannot always choose that health care provider, thereby eliminating a potential way to reward institutions for investing in and using ICT as a means of improving their performance. Under capitation, ICT is a potential source of cost reductions, but health care delivery organizations must grapple with the substantial capital requirements associated with ICT systems and determine if the potential savings outweigh known costs.
It is essential that that the health sector resolve the issue of disincentives created by current reimbursement mechanisms. Private sector employers, third-party payers, consumer organizations, and the government must create an environment for the delivery of health care services that is driven by quality and based on evidence. Health care professionals and organizations need to be compensated for managing the health of their patients or achieving desired outcomes rather than simply delivering services. For example, the costs of capturing and managing patient data at the individual level for personal health management, institutional level for patient care, and at the regional and national levels for public health and research must be recognized as part of the cost of health care. [17,46,49] These data are essential for improved quality within the health system. Further, the potential savings associated with implementing cost-effective best practices is significant and could be used to offset the costs of pursuing best information practices by health care organizations. [7]
Issues surrounding who will pay for the systems to support self-care must be addressed. These systems may well help to lower the costs of managing chronic illness, but patients may not be willing or able to pay to acquire needed equipment. Long-standing questions about the reimbursement of telemedicine services appear to be slowly resolving themselves along with the questions around electronic consultations. Will we reward effective and efficient service by paying physicians for answering electronic mail that eliminates the need for an office visit? We should. In the meantime, efforts such as those of the Leapfrog Group, which urges businesses to base the purchase of health care on principles that encourage patient safety measures (e.g., computer physician order entry), should be expanded. [50]. In short, if health care providers are rewarded for safety and quality results, they are more likely to invest in ICT.
Standards
Standards are the backbone of technological progress. They enable parts to fit together, assure consistency over time, and facilitate communication. Standards are "prerequisites to the joining of information from many separate computer sources to produce a whole that is greater than the sum of its parts."[51] They are the means by which we can be assured in consistency of meaning across settings and over time. [52] They are means by which we are able to combine data from different sources and thereby generate value from raw data through analysis. They are also means by which other industries have fostered innovation and efficiency (e.g., VISA/MasterCard, ATM networks, cell phones, bar codes on consumer packaged goods). [43] Until the health sector develops and implements a comprehensive set of standards there will not be a NHII.
The many years of health care standards setting activities are yielding noticeable benefits, particularly in the area of messaging standards. [51] Health Level 7 (HL7) has become the accepted standard for communicating clinical data; it is supported by every major medical informatics system vendor in the U.S. and is used for communication among clinical systems in large hospitals and group practices across the U.S. and elsewhere. The American National Standards Institute Accredited Standards Committee X12N standard has been selected as the standard for electronic exchange of administrative and financial health care transactions (excluding retail pharmacy transactions) in compliance with the Health Insurance Portability and Accountability Act of 1996. [53]
Despite progress on this front, much work remains to be done. Sentara Health Care in Norfolk, Virginia calculated that it spends over $600,000 annually in resources to integrate 40 systems from various vendors. [54] An estimated 60 percent of these resources are dedicated to massaging data when moving from one system to another. These costs arise in part because health care data standards are not as precise as those in other industries and allow flexibility among health care organizations implementing the "standards."
Further, standards for clinical vocabulary do not show as much progress as messaging standards. Although standards development activities are primarily in the private domain, the federal government can adopt standards and support the development of new standards. For example, the federal government can speed the use of standard clinical vocabulary in electronic health data in the U.S. by
• identifying the specific sets of non-overlapping clinical vocabularies that federal agencies will use and then announce that they will use them
• funding ongoing maintenance and improvement of these vocabularies
• supporting no-cost dissemination of these vocabularies to all U.S. institutions and organizations that create and exchange health data and to all nations willing to work on the exchange of health data for public health purposes
• developing robust mappings between these clinical vocabularies and the HIPAA standard administrative code sets, and
• helping identify additional domains in which standards are needed.
Many federal agencies and private entities will participate in the ongoing improvement of clinical vocabularies and mappings, but a single federal agency should be given the authority and an adequate budget to support the ongoing maintenance of these vocabularies and related research, development, and demonstration projects. This will ensure the coordination needed to prevent duplication and overlap among the set of standard clinical vocabularies. The value of this strategy has already been demonstrated with the development of the LOINC (Logical Observations: Identifiers, Names, Codes) coding system for electronic exchange of laboratory test results and other observations. LOINC development involved a public-private partnership comprised of the several federal agencies, academia, and the vendor community. This model can be applied to other standards setting domains.
The National Library of Medicine (NLM) has the perspective and experience to effectively fill this coordination role. By virtue of its mission, NLM routinely works across the disciplines and domains of health care. NLM successfully developed the Unified Medical Language System (UMLS) to enhance literature retrieval and led the partnership that developed LOINC. Further, NLM with others is working to bring coding systems into the public domain (e.g., SNOMED-CT, Systematized Nomenclature of Medicine – Clinical Terms).
Creating good standards is both a science and an art form. It requires an admixture of thought and practice to refine standards to the stage where they need to be generally adopted. Since practice and technology remain moving targets, the work is never done. Having said this, health care faces a backlog of standard development work and the issue we should face in five years should be quite different from what we face now if we can build on HL7, the Markle Foundation's Connecting for Health initiative, the work of the NAHIT, e-Health Initiative, the HIMSS NHII Task Force, and others to generate greater national and international collaboration among governments and vendors. The early work of the Markle Communications for Health group has revealed a need for standards in the following areas: reference information models (RIM), data types, terminology, clinical documents, clinical templates, data interchange, implementation manuals, rules, tool sets, terminology services, security, identifiers, and guidelines. Further, SNOMED-CT, a standard whose content is essential to the NHII, either needs to be brought into the system or a separate clinical vocabulary must be developed independently and without delay.
There are nearly 100 standards setting activities underway in health care today. [43] The addition of new collaborative organizations and greater involvement by the federal government should provide the needed leverage to make dramatic progress on standards in the next two years. How to assure that the alliances, budgets, and government come together to serve the greatest good is the challenge. In the meantime, vendors must incorporate existing standards into their systems and health care organizations should only purchase systems that use current standards. This will require that health care organizations track and participate in these important developments.
Privacy (confidentiality and security) framework
The debate over how to establish an appropriate privacy framework for this country has been long, heated, and inadequately focused. [55] Unfortunately, the issue has been framed overwhelmingly as a human rights issue rather than one of both human health and rights. While millions of Europeans now use a 'smart card' for personal authentication for health care and have unique personal identifiers as a matter of national policy, America is at risk of being unable to take this essential step. Further, much of the public policy created over the past decade has been informed by limited research on how individuals relate to informed consent for use of their data, past tissue samples, etc. Much of the data that has influenced policy has been generated through general public polls and surveys generated primarily by narrowly focused privacy advocates rather than through studies of actual circumstances relating to care delivery and research itself. Growing experience with e-health and computer-based personal records around the nation and other nations shows that the great majority of patients are more interested in engaging with their clinicians, getting laboratory results, and scheduling clinic visits over the Internet today than foregoing care or such communications for the sake of potential risks to personal privacy. Patients who are extremely fearful about the security of their personal data will most likely remain a particular challenge since going forward absolute security cannot be guaranteed and is far from guaranteed in current care settings.
The Health Insurance Portability and Accountability Act of 1996 (HIPAA) raised the bar for protection of individually identifiable health data and rightly so. This is a major achievement and it should help a great deal. Nonetheless, the recently released final modifications to the Standards for Privacy of Individually Identifiable Health Information (i.e., the Privacy Rule mandated by HIPAA) have not ended the debate on how to achieve an appropriate balance between privacy protection and legitimate use of health data to strengthen either personal care or the health system as a whole. [56-59] Thus, implementation of the current regulations is certain to be accompanied by continued debate between the human rights and human health perspectives of privacy.
A well-balanced research program is needed to examine carefully all dimensions of protection and use of person specific data. The evaluation of this complex topic should include demonstration projects to explore all aspects of privacy protection and legitimate access to data, particularly at the front line of patient care in office, clinic, and hospital settings. For example, research is needed that shows how crucial access to person-specific data is for progress in human health in an era of genomics and bioterrorism.
Finally, consideration should be given to establishing a voluntary organization, akin to the National Association for Biomedical Research (NABR), to advocate for continued responsible access to personal health data. NABR has been essential to assure that animals used in research are treated humanely while also advocating successfully for their use for legitimate biomedical research. Similar responsible advocacy for access to human health data may well be needed if data are to remain available for bioinformatics, health services and health policy purposes. This advocacy in the cause of human health will almost certainly be needed to counterbalance the continued erosion in public trust that has come from the drive for greater personal autonomy. [60]
Funding, planning, and implementation
The NHII deserves a long-term commitment of funding parallel to other important federal investments over the years, (e.g., federal research funding for the Internet, military investments in aircraft design and deployment that have benefited the airline industry, and the interstate highway system. Former Assistant Secretary for Health Philip R. Lee, M.D. recommended a ten-year, $14 billion investment by the federal government for NHII development. [36] This level of investment is commensurate with that of other countries that have already committed to developing and implementing a NHII. This federal investment would be used to fund NHII coordination, the NHII planning process, standards development and maintenance, research on a variety of issues, and fast track demonstration projects.
In a recent report, Fostering Rapid Advances in Health Care: Learning from System Demonstrations, the IOM recommends that DHHS implement eight to ten ICT demonstrations (along with demonstrations in chronic care, primary care, state health insurance coverage, and state liability) to serve as building blocks for health system change. [15] The goals of the ICT infrastructure demonstrations match NHII goals – ready access of relevant information for clinicians, support tools for patients including educational materials, electronic communications between patients and clinicians and among clinicians, data capture and decision support, management, performance measurement for ongoing assessment of safety and quality, and accountability. The IOM envisions the first set of ICT demonstration projects as initial nodes on a NHII. One time federal support would be used to establish the public-private partnership and infrastructure; health care organizations would allocate resources to maintain and enhance their ICT infrastructure and to redesign care processes to take full advantage of the NHII. The IOM report emphasizes the need for learning collaboratives to support the demonstration projects through information exchange and joint problem solving and for specific strategies to disseminate knowledge gained from the demonstrations.
NHII funding can be structured to foster collaboration within the private sector and between the public and private sectors as National Institute for Standards and Technology (NIST) does with its Advanced Technology Program (ATP). ATP is a cost-sharing program that partners the federal government with the private sector and encourages collaboration among private sector participants to develop and disseminate high-risk technologies that "offer the potential for significant, broad-based economic benefits to the nation."[61] NIST launched the Information Infrastructure for Healthcare (IIH) program in 1994. As of December 2001, this program has funded 32 projects aimed at developing technologies that will support a NHII. For example, IDX Systems Corporation is collaborating with Apelon, Inc., Intermountain Health Care (IHC) Health Services, Inc., Mayo Clinic Rochester, Stanford University, and the University of Nebraska Medical Center to "develop advanced software tools and technologies to enable the widespread creation, distribution, and application of electronic clinical practice guidelines."[62] Another active grant is supporting the development of an extensive, homogeneous infrastructure for distributed bioinformatics resources.[63] While the ATP program may have some detractors, it does represent a working collaborative model.
Beyond research and demonstration projects, serious consideration should also be given to providing grants to help health care organizations pay for implementing information systems. Revenue-neutral strategies can also be adopted that pay a premium for computer-based billings or offer incentives for ITC capital investments.
Conclusion
The developed economies of the world are beginning to understand some crucial elements needed for success for large-scale health care systems. A robust information and communications infrastructure is widely acknowledged as one such element. James Madison noted that a society that wishes to remain free must avail itself of the information and knowledge it needs to do so. While a health information infrastructure capable of managing the knowledge base for a free, complex society is no small undertaking, it is essential for personal health, safe communities, effective and efficient high quality care services, and timely patient-centered care. The NHII is crucial to our future health and as such it is a noble as well as essential undertaking. Now is the time for the United States to establish the national coordination structure that will build and maintain a standards framework, support the requisite research and evaluation culture for education and training, and create sufficient incentives and regulations to assure uptake and use by clinicians and other key stakeholders.
Summary
The health sector's most avoidable shortcomings can be linked to poor quality or lack of access to data, information, and knowledge. A national health information infrastructure is a necessary step for an improved health system and improved health of U.S. citizens. The NHII should be built through a combination of public and private investment and effort. The federal government should, however, assume a leadership role in this critical endeavor. Progress in four areas is pivotal to NHII development: incentives, standards, a privacy (confidentiality and security) framework, and funding.
Pre-publication history
The pre-publication history for this paper can be accessed here:
Acknowledgments
Acknowledgements
A previous version of this paper was presented to the National Quality Forum/Institute of Medicine Information Technology Summit held on March 5–7, 2002 in Washington, DC. A number of colleagues offered helpful advice to it. The author gratefully acknowledges the support of the Markle Foundation in the preparation of the manuscript. The views expressed are the author's and do not necessarily reflect the views of the Markle Foundation. The editorial support of Elaine Steen is most gratefully acknowledged.
References
- Institute of Medicine To Err Is Human: Building a Safer Health System. Washington, DC, National Academy Press. 2000.
- Institute of Medicine Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC, National Academy Press. 2001. [PubMed]
- Koo D, O'Carroll P, LaVenture M. Public health 101 for informaticians. J Am Med Inform Assoc. 2001;8:585–597. doi: 10.1136/jamia.2001.0080585. [DOI] [PMC free article] [PubMed] [Google Scholar]
- United States Public Health Service Making a Powerful Connection: the Health of the Public and the National Information Infrastructure. 1995. http://nnlm.gov/fed/phs/powerful.html Accessed 01/02.
- Balas EA, Boren SA. Managing clinical knowledge for health care improvement. Yearbook of Medical Informatics Bethesda, Md., National Library of Medicine. 2000. pp. 65–70. [PubMed]
- Nordin-Johansson A, Asplund K. Randomized controlled trials and consensus as a basis for interventions in internal medicine. J Intern Med. 2000;247:94–104. doi: 10.1046/j.1365-2796.2000.00583.x. [DOI] [PubMed] [Google Scholar]
- Wennberg JE, Fisher ES, Skinner JS. Geography and the debate over Medicare reform. 2002. http://www.healthaffairs.org//WebExclusives/Wennberg_Web_Excel_021302.htm Accessed 02/02. [DOI] [PubMed]
- Benton Foundation Networking for Better Care: Health Care in the Information Age. 1999. http://www.benton.org/library/health Accessed 03/02.
- Sim I, Gorman P, Greenes RA, Haynes RB, Kaplan B, Lehmann H, Tang PC. Clinical decision support systems for the practice of evidence-based medicine. J Am Med Inform Assoc. 2001;8:527–534. doi: 10.1136/jamia.2001.0080527. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Borowitz SM, Wyatt JC. The origin, content, and workload of e-mail consultations. JAMA. 2000;280:1321–1324. doi: 10.1001/jama.280.15.1321. [DOI] [PubMed] [Google Scholar]
- Bates DW, Cohen M, Leape LL, Overhage JM, Shabot MM, Sheridan T. Reducing the Frequency of Errors in Medicine Using Information Technology. J Am Med Inform Assoc. 2001;8:299–308. doi: 10.1136/jamia.2001.0080299. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Dexter PR, Perkins S, Overhage JM, Maharry K, Kohler RB, McDonald CJ. A computerized reminder system to increase the use of preventive care for hospitalized patients. NEJM. 2001;13:965–70. doi: 10.1056/NEJMsa010181. [DOI] [PubMed] [Google Scholar]
- Feachem GA, Sekhri NK, White KL. Getting more for their dollar: a comparison of the NHS with California's Kaiser Permanente. BMJ. 2002;324:135–143. doi: 10.1136/bmj.324.7330.135. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Pestotnik SL, Classen DC, Evans RS, Burke JP. Implementing antibiotic practice guidelines through computer-assisted decision support: clinical and financial outcomes. Ann Intern Med. 1996;124:884–890. doi: 10.7326/0003-4819-124-10-199605150-00004. [DOI] [PubMed] [Google Scholar]
- Institute of Medicine Fostering Rapid Advances in Health Care: Learning from System Demonstrations. Washington, D C, National Academy Press. 2002. [PubMed]
- National Library of Medicine Planning Panel Number 4 Long Range Plan on Medical Informatics. Bethesda, Md. 1986.
- Institute of Medicine . The Computer-based Patient Record: An Essential Technology for Health Care. In: Dick RS, Steen EB, editor. (revised edition edited by Dick RS, Steen EB, Detmer DE) Washington, DC. 1. National Academy Press; 1991 and 1997. [PubMed] [Google Scholar]
- National Institute of Standards and Technology Putting the Information Infrastructure to Work: Report of the Information Infrastructure Task Force Committee on Applications and Technology. Gaithersburg, Md, US Department of Commerce. 1994.
- Office of Technology Assessment Bringing Health Care Online: The Role of Information Technologies. US Government Printing Office, Washington, DC. 1995.
- Council on Competitiveness Highway to Health: Transforming U.S. Health Care in the Information Age. Washington, DC. 1996.
- Detmer DE. The future of IAIMS in a managed care environment: a call for private action and public investment. J Am Med Inform Assoc. 1997;4:S65–S72. [PMC free article] [PubMed] [Google Scholar]
- National Committee for Vital and Health Statistics Assuring a Health Dimension for the National Information Infrastructure. 1998. http://ncvhs.hhs.gov/hii-nii.htm Accessed 03/02.
- National Research Council Networking Health: Prescriptions for the Internet. Washington, DC, National Academy Press. 2000. [PubMed]
- National Committee on Vital and Health Statistics Toward a National Health Information Infrastructure: Interim Report. 2000. http://www.ncvhs.hhs.gov/NHII2kReport.htm Accessed 10/02.
- President's Information Technology Advisory Committee Transforming Health Care through Information Technology. 2001. http://www.ccic.gov/pubs Accessed 03/02.
- National Committee for Vital and Health Statistics Information for Health: A Strategy for Building the National Health Information Infrastructure. 2001. http://ncvhs.hhs.gov/nhiilayo.pdf Accessed 03/02.
- National Health Service Information for Health. 1998. http://www.doh.gov.uk/ipu/strategy/index/htm Accessed 03/02.
- National Health Information Management Advisory Council (NHIMAC) Health Online: A Health Information Action Plan for Australia. Commonwealth of Australia Canberra. 1999.
- National Electronic Health Records Taskforce A Health Information Network for Australia. 2000. http://www.health.gov.au/healthonline Accessed 01/02.
- Office of the Health and Information Highway Tactical Plan for a Pan-Canadian Health Infostructure: 2001 Update. 2001. http://www.hc-sc.gc.ca/ohih-bsi/available/plan200111/plan-e.html Accessed 01/02.
- Kher U. Crunching digits for drugs. Time Archive. http://time.qpass.com/time/magazine December 24 2001. Accessed 01/02.
- RODS Laboratory RODS Demonstration Systems in Public Health Surveillance – Completed, Ongoing, and Planned. 2002. http://www.health.pitt.edu/rods Accessed 02/02.
- Gorman C. Rethinking breast cancer. Time. February 18 2002. [PubMed]
- Friedman CP, Ozbolt JG, Masys DR. Toward a new culture for biomedical informatics: report of the 2001 ACMI symposium. J Am Med Inform Assoc. 2001;8:519–526. doi: 10.1136/jamia.2001.0080519. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Science and Technology Policy Institute RAND A Framework for the Information Technology Infrastructure: Results of the 1st Summit, Draft Report. 2001. http://www.rand.org/scitech/stpi/infrastructure/summit1/summary.pdf Accessed 03/02.
- Lee PB, Abramovice BG, Lee PR. Written Supplement to the Testimony of Dr. Philip R. Lee at the Joint Hearings on the National Health Information Infrastructure and Health Statistics for the 21st Century. Washington, DC, National Committee on Vital and Health Statistics. October 2001.
- Yasnoff WA, Overhage JM, Humphreys BL, LaVenture M. A national agenda for public health informatics: summarized recommendations from the 2001 AMIA Spring Congress. J Am Med Inform Assoc. 2001;7:1–21. doi: 10.1136/jamia.2001.0080535. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Caregroup Healthcare System PatientSite: Healthcare you can connect with. 2002. https://patientsite.caregroup.org/whatis.asp Accessed 12/02.
- Personal Path Systems Inc Welcome to Personal Path Systems. 2002. http://www.personalpathsystems.com Accessed 12/02.
- Shortliffe EH. Networking health: learning from others, taking the lead. Health Aff. 2000;19:9–22. doi: 10.1377/hlthaff.19.6.9. [DOI] [PubMed] [Google Scholar]
- The e-Health Initiative What we do. 2002. http://www.ehealthinitiative.org/ Accessed 11/02.
- Markle Foundation Markle Foundation launches initiative to promote adoption of key clinical health information standards. 2002. http://www.markle.org/news/newspressrelease_062102.stm Accessed 10/02.
- The National Alliance for Health Information Technology Prospectus. 2002. http://www.nahit.org/change.htm Accessed 10/02.
- Healthcare Information and Management Systems Society HIMMS News. 2002. http://www.himss.org/ASP/ContentRedirector.asp?Contentid=22401 Accessed 12/02.
- Clayton PD. The state of clinical information systems after four decades of effort. Yearbook of Medicine Informatics 2001 Stuttgart, Germany, Schattauer. 2001. pp. 333–337. [PubMed]
- Raymond B, Dold C. Clinical Information Systems: Achieving the Vision. Oakland, CA, Kaiser Permanente Institute for Health Policy. 2002.
- Henry D, Cooke S, Montes S. The Emerging Digital Economy. 1998. http://www.ecommerce.gov/emerging.htm Accessed 03/02.
- Henry D, Cooke S, Buckley P, Dumangan J, Gill G, Pastore D, LaPorte S. The Emerging Digital Economy II. 1999. http://www.ecommerce.gov/ede/report.html Accessed 03/02.
- Schneider EC, Riehl V, Courte-Wienecke S, Eddy D, Sennett C. Enhancing performance measurement: NCQA's road map for a health information infrastructure. JAMA. 1999;282:1184–1190. doi: 10.1001/jama.282.12.1184. [DOI] [PubMed] [Google Scholar]
- Leapfrog Group Fact sheet: the Leapfrog Group for patient safety. 2002. http://www.leapfroggroup.org Accessed 03/02.
- McDonald CJ, Shadow G, Suico J, Overhage JM. Data standards in health care. Ann Emerg Med. 2001;38:303–311. doi: 10.1067/mem.2001.117947. [DOI] [PubMed] [Google Scholar]
- Landro L. Electronic medical records call for common language. The Wall Street Journal Online. http://webreprints.djreprints.com August 15 2002. Accessed 10/02.
- Department of Health and Human Services Frequently asked questions about electronic transaction standards adopted under HIPAA. 2000. http://aspe.hhs.gov/admnsimp/faqtx.htm Accessed 02/02.
- The National Alliance for Health Information Technology Statement of the National Alliance for Health Information Technology in support of the National Committee on Vital and Health Statistics, Subcommittee on Standards and Security. 2002. http://nahit.org/content/press020829ncvhs.doc Accessed 10/02.
- Detmer DE. Your privacy or your health: will health medical privacy legislation stop quality health care? Intrntl J Qual Health Care. 2000;12:1. doi: 10.1093/intqhc/12.1.1. [DOI] [PubMed] [Google Scholar]
- Annas GJ. Medical privacy and medical research – judging the new federal regulations. NEJM. 2002;346:216–220. doi: 10.1056/NEJM200201173460320. [DOI] [PubMed] [Google Scholar]
- Kulynch J, Korn D. The effect of the new federal medical-privacy rule on research. NEJM. 2002;346:201–204. doi: 10.1056/NEJM200201173460312. [DOI] [PubMed] [Google Scholar]
- Goldstein A. Medical privacy revised: Bush administration moves to protect confidentiality. p. 1A. August 10, 2002.
- U.S. Department of Health and Human Services Modifications to the standards for privacy of individually identifiable health information – final rule. 2002. http://www.hhs.gov.news/press/2002pres/20020809.html Accessed 08/04. [PubMed]
- O'Neill O. Autonomy and Trust in Bioethics. Cambridge, Cambridge University Press. 2002.
- Lide B, Spivack RN. Advanced Technology Program Infrastructure for Healthcare Focused Program: A Brief History. 2000. http://www.atp.nist.gov Accessed 02/02. [DOI] [PMC free article] [PubMed]
- Advanced Technology Program Project brief: standards-based interoperable guideline systems. 2001. http://www.atp.nist.gov/awards/00004606.htm Accessed 02/02.
- Advanced Technology Program Project brief: development of an extensible, homogeneous infrastructure for distributed bioinformatics resources. 2001. http://jazz.nist.gov/atpcf/prjbriefs/prjbrief.cfm?ProjectNumber=00-00-4407