Abstract
OBJECTIVE:
Create an interoperable set of nursing flowsheet assessment measures within military treatment facility electronic health records using the 3M Healthcare Data Dictionary (HDD).
DESIGN:
The project comprised three phases: 1) discovery included an in-depth analysis of the Essentris data to be mapped in the HDD; 2) mapping encompassed the creation of standard operating procedures, mapping heuristics, and the development of mapping tools; and 3) quality assurance incorporated validation of mappings using inter-rater agreement.
RESULTS:
Of 569,073 flowsheet concepts, 92% were mapped to the HDD. Of these, 31.5% represented LOINC concepts, 15% represented SNOMED CT and 1% represented both. 52.5% were mapped to HDD concepts with no standardized terminology representations.
CONCLUSIONS:
Nursing flowsheet data can be mapped to standard terminologies but there is not the breadth of coverage necessary to represent nursing assessments. Future work is necessary to develop a standard information model for the nursing process.
Introduction
As nurses provide care, they assess every patient. These assessments are documented electronically on nursing flowsheets within the electronic health record (EHR). Cross-organizational EHR communication is essential not only for physicians but also for nurses. Nurses should have access to pertinent patient care data regardless of the setting where the data was collected or created. In addition, the data entered by nursing, such as vital signs, are also used by other disciplines. The Department of Defense (DoD) has a commitment to enable interoperable patient data documented by nursing within their military treatment facilities (MTFs). As a result, they initiated a project for consistently encoding nursing flowsheet data across MTFs.
This project addressed the variability in the representation of nursing assessment measures across MTFs resulting in diminished interoperability. Nursing assessment measures were mapped to the 3M Healthcare Data Dictionary (HDD) that includes terminologies such as the Systemized Nomenclature of Medicine - Clinical Terms® (SNOMED CT®) and Logical Observations Identifiers Names and Codes® (LOINC®). The selection of SNOMED CT and LOINC as the primary source for encoding the nursing assessments was based on two precedents. First, both terminologies have been endorsed by the National Committee on Vital and Health Statistics (NCVHS) as terminologies to facilitate interoperability for the documentation of assessment measures1. Second, the American Nurses Association (ANA) has recognized the same terminologies for use by nursing within the EHR2.
Background
It is crucial that nursing documentation be standardized in order to share comparable data with other healthcare organizations or settings. Exchanging data without translation is the definition of interoperability3. Interoperability within and among EHRs cannot be achieved without the implementation of standardized clinical terminologies4. The ANA strongly supports interoperability, as evidenced by the fact that they recognize terminologies appropriate for use by nursing5.
The nursing process forms the foundation for nurse decision making6. It is composed of six integral parts: assessment, diagnosis, outcomes identification, planning, implementation, and evaluation. The first step in delivering care is the assessment. Data documented during the assessment is used to determine other components of the nursing process. Assessment measures are the defining characteristics for nursing problems. Assessment measures are the indicators used to determine progression toward a goal or outcome. Additional interventions and care is also determined by values obtained from an assessment. Assessment data is the type of data found on flowsheets, which are structured electronically as clinical observations7. A clinical observation consists of a question and answer pair (also known as name/value pair). The two nursing terminologies that can be used on nursing flowsheets for point-of-care nursing assessment questions recognized by the ANA are SNOMED CT and LOINC8, 9.
SNOMED CT is an internationally recognized clinical terminology comprised of Concept IDs, fully specified terms, preferred terms, synonyms, and relationships used to represent clinical concepts across the scope of healthcare10. SNOMED CT consists of 19 top-level hierarchies. The top-level hierarchy used for the assessment question is “Observable Entity” and the top-level hierarchy for the answer (or value) is “Clinical Finding”.
LOINC is also an internationally recognized healthcare terminology that provides a common language for observations. Clinical LOINC includes assessment items applicable to nursing8. This includes clinical assessment scales, such as the Morse Fall Scale, the Braden Scale, and Apgar Score. It also includes assessment panels, such as vital signs assessment, skin assessment, and wound assessment. LOINC provides a code system for the observation identifier field (OBX-3) of the Health Level Seven (HL7) observation-reporting message11.
The DoD currently uses the 3M Healthcare Data Dictionary (HDD) to encode data; therefore, the motivation for this project was to map the nursing flowsheets to the same data dictionary to facilitate interoperability. The HDD is a controlled medical vocabulary server that supports mapping from multiple institutions to one central concept, allowing multiple legacy information systems to effectively share information. Terminologies within the HDD are mapped to HDD concepts and the corresponding names and IDs are structured as representations of the concepts. Each concept is assigned a unique HDD Numeric Concept IDentifier (NCID). The mappings facilitate communication between applications, and reference terminologies via standard services, thereby facilitating translation and integration of healthcare data.
Project Context
The Essentris® hospital information system from CliniComp International (CCI) is the inpatient EHR deployed at 59 DoD Military Treatment Facilities (MTFs). The system is an inpatient point-of-care solution used for charting at each inpatient facility. Each facility can customize the front-end of their charting system for their needs and requirements. It stores information from patient charts for analytics and reporting purposes. The patient chart includes nursing flowsheets, treatment notes, order entry, medication administration records, and care plans.
There are different types of flowsheets within the Essentris system; the main flowsheets are fluid balance, vital signs, and medications. Other flowsheets are custom for the nursing unit they represent and the assessments needed for that particular patient population. The system has the capabilities to interface with physiological monitors that can auto populate fields within particular flowsheets. It also can be configured to support the needs of numerous inpatient nursing environments including critical care, medical/surgical, labor and delivery, emergency room, pediatric units, psychiatric units and other pertinent inpatient environments. End-users are able to add rows and choice lists for a given patient on a particular flowsheet.
CCI provides technical support and metadata for each facility as the system is deployed at the site. When sites wish to add concepts to the flowsheets they send a request to CCI who provides an internal name, description, and internal number for the item. These data elements (metadata) are called the database items (DBIs) and are used to encode patient data.
Although the Essentris system has a master database defining DBIs, each DBI, when used at a site, can take on a site-specific name and meaning independent of, and uncoordinated with, the master database and any other sites. This may result in different DBI meaning across sites, while sharing the same identifier. The lack of data standardization in Essentris prevents consistent clinical application, data collection, or research across sites. Most importantly, it prevents the sharing of patient information between MTFs. Therefore, the goal of this project was to facilitate interoperable nursing documentation data across 59 MTFs. The DoD tasked 3M to map the DBIs into the HDD.
The project objectives were to 1) analyze terms currently used; 2) determine why the terms are different site-by-site; 3) evaluate the meaning of the terms; 4) map the terms into the HDD for normalization/standardization within the DoD; 5) evaluate whether the concepts exist in LOINC, SNOMED CT or any other standards; and 6) provide recommendations for future standardization of terminology content used within the nursing flowsheets. Using the HDD for mapping accomplishes multiple objectives efficiently including normalization, standardization, cross walking to standards as appropriate, and forward/backward compatibility with legacy systems and standard terminologies.
This project of flowsheet mapping was the first of several project phases that encompass other portions of the EHR (such as notes, orderables, etc.). Participation for this field study consisted of 59 inpatient DoD MTFs. Five subject matter experts (SME) and two programmers were added to the HDD group to help in the mapping and analysis of the data. The period of performance for this particular project phase occurred from October 2010 through September 2011.
Methods
The methodology for this project included three phases, with the objectives of evaluating the terminology model, the information model, and the conceptual meaning of the terms. These phases included a discovery phase, a mapping phase, and a quality assurance phase. A final report outlining recommendations to the DoD was delivered at the end of the period of performance.
The discovery phase consisted of analysis to determine the data model. The goal of this phase was to understand what a DBI means and understand how the DBI is used at each site. This required extraction of the site database data, including all flowsheet tables per nursing unit, DBIs, and their choice lists from each MTF. The data extracted for mapping, consisted of each site’s, internal name, internal number, description, full name, display name, units of measure, data type, data size, and minimum/maximum values.
The main goal of the mapping phase was to map flowsheet concepts to the HDD to provide the DoD with a granular concept of their flowsheet DBIs. The first step was the development of a standard operating procedure (SOP) that contains rules to ensure consistency and accuracy of mapping across sites.
The next step in mapping was to separate a site’s assessment data into “questions” or “answers”. SMEs then mapped the DBI to HDD concepts. The heuristic developed for mapping the “questions” was to first search concepts with a LOINC representation, and if no concept was found, concepts with a SNOMED CT representation were searched. Finally, if there was not an HDD concept with a LOINC or SNOMED CT code represented other concepts in the HDD were searched; if not found, the DBIs were flagged for creation of a new concept in the HDD. For the answers, mapping started with concepts having a SNOMED CT ID represented in the “Clinical Finding” domain. Concepts with no match followed the same path outlined above.
3M modified existing in-house mapping tools and principles to accommodate the Essentris data. The flowsheet data was imported into the tool and linked to the HDD, hence facilitating automated matching. The tool highlighted the differences in the DBI name, flowsheet, flowsheet section, choice lists, environment, and site. This process was designed to help the SMEs determine the semantic meaning of each concept.
The Quality Assurance (QA) phase of the project included reviewing each DBI mapped to an NCID. The SMEs analyzed the data across sites to determine the variability between facilities. This ensured accurate and consistent mapping decisions across sites.
Results
In the discovery phase, data analysis provided vast evidence of inconsistent DBIs across sites. Within the Essentris system there is a master database that contains DBIs. Each DBI was considered a unique item within the system. These DBIs are represented as clinical concepts on flowsheets. They have an internal number and an intended meaning, represented by an internal name and a description. Every DBI is hard coded and cannot be changed by the MTF. When the system was deployed at an MTF, DBIs were added to the site database from the master database. After installation the flowsheet DBIs can be reconfigured at the site on a per environment-flowsheet-section basis.
The internal name and description were hidden from the end-users but they had the functionality to create their own display name and full name. Therefore, because the actual meaning is not presented to the end-user, when the display was reconfigured, they had no idea that the DBI meaning had been changed.
With this variability between DBIs, a unique identifier called the Basic Mapping Unit (BMU) was created. The BMU concept includes the DBI name, internal number, site, environment, flowsheet, and flowsheet section for a given item. With the creation of the BMU, 3M assigned an NCID to a flowsheet clinical concept, and has the capability to evaluate changes within the system for any new extract. Table 1 illustrates the reason why BMUs were created. In the first two rows, the internal number is the same, but the flowsheet and DBI names have been reconfigured resulting in two different concepts. In the last two rows, the flowsheet remained the same but an additional qualifier was added to the DBI, therefore adding additional meaning.
Table 1.
Internal number Different BMU Example
| Internal number | Site | Env | Flowsheet | DBI Name | Meaning |
|---|---|---|---|---|---|
| 0131 | Site 1 | ICU | Blood Gas | vBE | Base Excess Venous Blood |
| 0131 | Site 1 | PACU | Lab results | BE.mvbg | Base Excess Mixed Venous Blood |
| 4478 | Site 3 | ICU | Vital Signs | Pulse | Heart Rate |
| 4478 | Site 4 | ED | Vital Signs | Left Pulse | Heart Rate on the left side |
The internal name and number could not serve as the unique identifier for the DBI because the end-user can reconfigure the display name or full name on the flowsheet. As a result, 3M requested that extracts from each MTF include the representation displayed to the end-user.
Another conclusion reached during the discovery phase of the project was that no formal information model existed. Instead, the graphical user interface (GUI) provided the structure and contextual meaning for the clinical data stored in the system. Although the SMEs were able to work around the lack of an information model structure by using BMUs to complete the initial mapping, this issue has serious implications for sustainment and future mapping. There is a risk that mapping between the HDD and flowsheet data will be unsynchronized due to the lack of control and modeling, thus disabling interoperability.
The BMU was used for mapping because it provided contextual meaning of the concept; as a result, the SMEs were able to determine what the end-user would chart for a particular DBI. A total of 569,073 BMUs were generated from the data extractions and reviewed for mapping from the 59 MTFs. This number of BMUs resulted from the variations of 4,964 distinct DBIs as they were used in flowsheets across the sites.
Of the 569,073 BMUs reviewed, 45,834 BMUs (8%) were considered “unmappable”, because their meaning was unknown, ambiguous, or inter-rater agreement could not be achieved. They were generated from 1221 distinct DBIs. These BMUs could not be included in the HDD and were sent to the sites for clarification. The remaining 523,239 BMUs (92%) were mapped and loaded into the HDD. A concept required 100% inter-rater agreement before it was mapped and loaded. Table 2 illustrates these results.
Table 2.
New Name Assignment Changing Meaning
| Review Results | Number of BMUs | Percent of Total | Number of DBIs |
|---|---|---|---|
| Unmappable | 45,834 | 8% | 1221 |
| Mapped and Loaded into the HDD | 523,239 | 92% | 4,493 |
| Total | 569,073 | 100% | 4,964 |
3M evaluated which DBIs were commonly used in flowsheets across the MTFs. Of the 4,964 distinct DBIs 39% were used by only one MTF, while 61% were used by multiple MTFs. When a DBI was used by multiple MTFs, this indicated that the MTF started by selecting the same DBI from the master database, but assigned different meanings across sites, or even within the same site.
The variability of DBIs between sites was determined by linking the internal number for each BMU to its “originating” DBI. 1.2% of the DBIs used in flowsheets remain unchanged when deployed.
The percent mapped to each standard terminology was evaluated (Table 3). This was guided by the mapping heuristics. 31.5% represented LOINC concepts, 15% represented SNOMED CT, and 1% represented both. 52.5% were mapped to HDD concepts with no standardized terminology representations.
Table 3.
Mapping to NCIDs in the HDD
| HDD | Number of NCIDs | Percent of Total | Number of BMUs |
|---|---|---|---|
| NCIDs with LOINC (but not SNOMED CT) | 1,291 | 31.5% | 266,098 |
| NCIDs with SNOMED CT (but not LOINC) | 619 | 15% | 62,335 |
| NCIDs with LOINC and SNOMED CT | 32 | 1% | 6,972 |
| NCIDs without LOINC or SNOMED CT | 2,147 | 52.5% | 187,834 |
| Total | 4,089 | 100% | 523,239 |
The results for this project were provided to the DoD. This included recommendations regarding general terminology management, information modeling, and process improvement.
The initial analysis and mappings normalized the data and provided an understanding of the variability about the use of the DBIs at the sites. However, in the current system, the DBI changes are ongoing, unpredictable, and voluminous. For interoperability to be maintained in the system, the uncontrolled variability in the DBIs must be eliminated. Information modeling and control over the variability of the DBIs would allow the HDD to synchronize with Essentris for mapping updates, creating a way for sites to share information in a meaningful way.
Understanding terminology requirements is the foundation for developing enterprise policies and system constraints to avoid semantic shift or drift in the metadata used to encode patient information. The project findings are explained using the Cimino’s desiderata as a framework12.
The second recommendation is to incorporate information models into the Essentris system. Some guidelines are: 1) use the existing GUI as the basis for generating an explicit model to support backward compatibility; 2) constrain Essentris to the information model to the extent possible, either through policy or software modification, preferably both; 3) reflect allowable changes to the structure of the data collected through the GUI in the model; and 4) bind data elements in the information model to concepts, reference set/value sets in the HDD. The information model must be tightly coupled to the reference terminology, which is provided by the HDD.
This pragmatic approach will achieve a formal, explicit, stable information model that can be mapped to other standard models. An academic approach that starts with some existing standard model and tries to impose its constraints on the Essentris system has a lower likelihood of successful implementation because of greater initial complexity (having to reconcile different purposes, granularity, compositional structures, required vs. optional elements, etc.) and failure to take the DoD requirements into account. The goal of information modeling is not a theoretical result, but a functional model that can be of practical use in sustainment mapping.
The HDD is an independent terminology server that works with any external information model such as the Health Level Seven (HL7), Reference Information Model (RIM), any vendor’s, or homegrown information models. Hence, it is recommended that Essentris develop a formal and meaningful information model. Whereas it may not be realistic to retrofit the software completely to follow the information model, it can serve to inform the data governance policy and facilitate data extraction for mapping.
Discussion
Nursing entities need the ability to adapt systems as needed. For interoperability to occur, the BMUs were mapped into the HDD, which provides the system with some amount of interoperability. However, sites need the flexibility to add additional representations (such as display) to a concept without changing its meaning. In addition, some form of control process needs to be enforced so that display changes do not change the meaning of the master concepts.
Encoded assessment measures are required for all portions of the nursing process. These coding measures need to be consistent and semantically structured across settings. The first requirement for a good terminology is content coverage12. It was determined that the Essentris database had good coverage for nursing assessments, but it was unexpected to discover that over 50% of the concepts did not exist in either SNOMED CT or LOINC. This means that less than half of the concepts are interoperable with institutions outside of the DoD MTFs. The low percentage of DBIs that mapped to HDD concepts with a SNOMED CT or LOINC representation indicates that neither terminology has the breadth and depth of expression needed to capture nursing assessments.
Another requirement for mapping is clear meaning with no ambiguity. The 8% of DBIs that could not be mapped will remain inoperable because their meaning was unknown or ambiguous. These could be mapped and interoperable if clear heuristics were developed for defining concepts across settings.
NANDA International has not coded their nursing indicators; therefore, they are not available for use electronically13. Similarly, Nursing Intervention Classification (NIC) has not coded the nursing activities of which assessment measures are a part. Therefore, nursing needs to identify the terminology content required for point-of-care measures pertinent for the nursing process to accurately represent the nursing process and care plan within the electronic health record. This will require coordination and harmonization among nursing terminologies.
The findings of this project period of performance also revealed that there are not any standard information models defined to represent nursing practice beyond the HL7 observation name/value pair structure. An information model defines the rules for storing data in the context of a medical event. It also defines how the concepts in the coded vocabulary should be used to instantiate the information model so that the event is accurately described. The information model for clinical data is analogous to the grammar and usage rules for the English language that dictates how to combine words in a proper way to convey meaningful information. The information model depends on the coded vocabulary for concepts to be used to encode patient data. Without the information model, the coded concepts cannot be used correctly as a group to describe the clinical event completely. Without a reference terminology to instantiate the information model, the data cannot be semantically interoperable and computable. In this sense, the vocabulary and the information model are inseparable (“terminology binding” for an information model).
The information model serves as the glue that binds various user interface artifacts to different terminology concepts and free text entries. The information model also serves as the glue that holds together the captured clinical information, its storage, and its retrieval in a meaningful way. Most importantly, having an information model with terminology binding frees the data from being dependent on the user interface for structure and meaning. The encoded data is persistent and consistent in its meaning, and secondary use of the data is enabled.
Work has begun at HL7 and Integrating the Healthcare Enterprise (IHE) to create an information model for the nursing care plan. There has also been work in the industry on detailed clinical models14, 15. A Detailed Clinical Model specifies clinical knowledge about small items of information, usually single observations or actions, or small clusters of observations that belong together. An example of a detailed clinical model would be a model for vital signs. The detailed clinical models provide the standard structure and terminology bindings needed for clinical decision support and automated data analysis. These detailed clinical models need to be created for nursing to support semantic interoperability. Therefore, there needs to be nursing involvement in the development of the detailed information models required to support nursing care.
One limitation of this project is that only one vended system was evaluated. However, to our knowledge, this is the largest mapping of nursing data to standard terminologies ever undertaken. Because the initial project phase encompassed 59 MTFs across the globe and was an evaluation of a vended system, we believe that the results can be generalizable to other organizations.
Another limitation is that the SMEs at 3M had access to only the terminology metadata but were not able to obtain a report on how the data was stored in patient records. Because of this, all assessment measures were treated equal. It would have been helpful to know the most common assessment measures stored on patients and correlate those with the mappings to the HDD concepts represented by standard terminologies.
The biggest unanswered question is where and how should nursing come together to standardize nursing terminology and information models. There are numerous disjointed nursing terminology projects underway. A path needs to be blazed that joins these projects together for the common good. One possible solution to these separate and sometimes overlapping ontologies would be for the ANA and Office of the National Coordinator to lead the effort to standardize and synchronize nursing terminologies and information models. This effort would include governmental entities, such as the DoD, VA, CMS, and industry nursing informatics leaders.
Conclusion
Nurses should have access to all pertinent patient care data regardless of the setting where the data was collected or created. A pragmatic, viable option for initial and ongoing normalization and standardization of nursing data is achievable, as proven by the successful mapping of the flowsheet metadata. The results of the data analysis and initial mapping can be used by the DoD to formulate the data governance policy required to implement data standardization. There is much work yet to be done on standard terminologies and information models to standardize the nursing process within an EHR. The actual mappings done for this project can be leveraged for content submission to the standard terminology bodies. Interoperability for nursing charting of patient data is essential and achievable through the use of standardized terminologies and information models.
Table 4.
Terminology Recommendations Based on Desiderata
| Desiderata | Description | Project Findings |
|---|---|---|
| Content | Rich content including breadth and depth of expression. | The Essentris system contains all content required for nursing assessments. |
| Concept orientation | Concepts and synonyms must correspond to exactly one clear definition (meaning). | DBIs do not function as unique concepts, which is why the clinical data encoded with the internal number at the sites is not semantically interoperable. |
| Non-semantic concept identifier | Concept identifiers should not have meaning and each concept must have a unique identifier. | One internal number can have multiple different meanings, and multiple internal numbers can have the same meaning within the Essentris System. |
| Concept permanence | Once a concept is defined, it should not be deleted nor should its code (identifier) be reused. | Users can change the full name and display name for the internal number, which amounts to code reuse and concept meaning change. |
| Polyhierarchy | Concepts should be able to participate in more than one hierarchy. | No explicit hierarchy or relationships for the DBIs exist, which would provide definition and meaning. |
| Formal definition | Formal concept definitions are computable through explicit relationships. | There is no means to define DBIs formally at the site; users deploy the DBIs as if they are placeholders and change the meaning arbitrarily resulting in multiple meanings/usages of a DBI. |
| Context representation | A terminology should take into account the context in which its concepts will be used. | No DBI constraints are in place. Contextual information could be provided by the site but was transient and inconsistent. |
| Graceful evolution | Terminology should change in a consistent way to ensure backward and forward compatibility. | When a site requires a new DBI, one is created in the master database so the MTF can have it in their site database, or they are assigned a current DBI for arbitrary use, resulting in a change in meaning of the DBI. |
| Recognition of redundancy | A terminology should avoid having redundant concepts. | Redundancy is present for DBIs and DBI names, due to the lack of control over site deployment usage. |
References
- [1].National Committee on Vital and Health Statistics . Recommendations for PMRI Terminology Standards. Washington D.C.: 2003. Nov. 2003 Contract No.: Document Number|. [Google Scholar]
- [2].American Nurses Association . ANA Recognized Terminologies that Support Nursing Practice. Washington, D.C.: ANA; 2010. [cited 2012 March 10]; Available from: http://www.nursingworld.org/Terminologies. [Google Scholar]
- [3].Institute of Electronics Engineers . IEEE Standard Computer Dictionary: A Compilation of IEEE Standard Computer Glossaries. New York: IEEE; 1990. [Google Scholar]
- [4].Foley M, Garrett G. The code ahead: key issues shaping clinical terminology and classification. J Ahima. 2006 Jul-Aug;77(7):24–30. [PubMed] [Google Scholar]
- [5].Warren JJ, Bakken S. Update on standardized nursing data sets and terminologies. J Ahima. 2002 Jul-Aug;73(7):78–83. quiz 5–6. [PubMed] [Google Scholar]
- [6].American Nurses Association . The Nursing Process. American Nurses Association; 2012. [updated 2012; cited 2012 7 March 2012]; Available from: http://nursingworld.org/EspeciallyForYou/What-is-Nursing/Tools-You-Need/Thenursingprocess.html. [Google Scholar]
- [7].Bakken S, Cimino JJ, Haskell R, Kukafka R, Matsumoto C, Chan GK, et al. Evaluation of the Clinical LOINC (Logical Observation Identifiers, Names, and Codes) Semantic Structure as a Terminology Model for Standardized Assessment Measures. J Am Med Inform Assoc. 2000 2000 Nov 1;7(6):529–38. doi: 10.1136/jamia.2000.0070529. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [8].Matney S, Bakken S, Huff SM. Representing nursing assessments in clinical information systems using the logical observation identifiers, names, and codes database. J Biomed Inform. 2003 Oct;36(4–5):287–93. doi: 10.1016/j.jbi.2003.09.008. [DOI] [PubMed] [Google Scholar]
- [9].Scichilone RA. The benefits of using SNOMED CT and LOINC in assessment instruments. J Ahima. 2008 Jul;79(7):56–7. [PubMed] [Google Scholar]
- [10].IHTSDO . SNOMED Clinical Terms® User Guide January 2010 International Release. Copenhagen, Denmark: IHTSDO; 2010. [cited 2012 Feb 17]; Available from: http://www.ihtsdo.org/ [Google Scholar]
- [11].McDonald CJ, Huff SM, Suico JG, Hill G, Leavelle D, Aller R, et al. LOINC, a universal standard for identifying laboratory observations: a 5-year update. Clin Chem. 2003 Apr;49(4):624–33. doi: 10.1373/49.4.624. [DOI] [PubMed] [Google Scholar]
- [12].Cimino JJ. Desiderata For Controlled Medical Vocabularies in the Twenty-First Century. Meth Inform Med. 1998;37(4–5):394–403. [PMC free article] [PubMed] [Google Scholar]
- [13].Lundberg C, Warren J, Brokel JM, Bulechek G, Butcher H, McCloskey Dochterman J, et al. Selecting a Standardized Terminology for the Electronic Health Record that Reveals the Impact of Nursing on Patient Care. Online Journal of Nursing informatics. 2008;12(2) [Google Scholar]
- [14].Goossen W, Goossen-Baremans A, van der Zel M. Detailed clinical models: a review. Healthcare informatics research. 2010 Dec;16(4):201–14. doi: 10.4258/hir.2010.16.4.201. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [15].Goossen WT, Goossen-Baremans A. Bridging the HL7 template - 13606 archetype gap with detailed clinical models. Stud Health Technol Inform. 2010;160(Pt 2):932–6. [PubMed] [Google Scholar]