Abstract
Objective To describe the perspectives of Regenstrief LOINC Mapping Assistant (RELMA) users before and after the deployment of Community Mapping features, characterize the usage of these new features, and analyze the quality of mappings submitted to the community mapping repository.
Methods We evaluated Logical Observation Identifiers Names and Codes (LOINC) community members’ perceptions about new “wisdom of the crowd” information and how they used the new RELMA features. We conducted a pre-launch survey to capture users’ perceptions of the proposed functionality of these new features; monitored how the new features and data available via those features were accessed; conducted a follow-up survey about the use of RELMA with the Community Mapping features; and analyzed community mappings using automated methods to detect potential errors.
Results Despite general satisfaction with RELMA, nearly 80% of 155 respondents to our pre-launch survey indicated that having information on how often other users had mapped to a particular LOINC term would be helpful. During the study period, 200 participants logged into the RELMA Community Mapping features an average of 610 times per month and viewed the mapping detail pages a total of 6686 times. Fifty respondents (25%) completed our post-launch survey, and those who accessed the Community Mapping features unanimously indicated that they were useful. Overall, 95.3% of the submitted mappings passed our automated validation checks.
Conclusion When information about other institutions’ mappings was made available, study participants who accessed it agreed that it was useful and informed their mapping choices. Our findings suggest that a crowd-sourced repository of mappings is valuable to users who are mapping local terms to LOINC terms.
Keywords: LOINC, Clinical laboratory information systems, medical record systems, vocabulary, controlled
BACKGROUND AND SIGNIFICANCE
The adoption of health information technology is spreading worldwide.1–4 The United States federal government passed the Health Information Technology for Economic and Clinical Health (HITECH) Act in 2009 to accelerate health information technology's adoption in this country. Through the Meaningful Use program, HITECH provides financial incentives to physicians and hospitals for implementing electronic health records (EHRs) and using them in ways that are anticipated to improve the quality and efficiency of care. From 2009–2013, the percentage of hospitals and physicians in the United States who had (at least) a basic EHR system increased from 19-59% and 22-48%, respectively.5
Although patients readily seek care across many settings and institutions,6 the purview of EHRs usually stops at organizational boundaries. Electronic data sharing across institutions is hampered by the heterogeneous names and formats used to store clinical data.7,8 Only by adopting syntactic and semantic data standards can EHRs become interoperable and capable of integrating vast stores of variously formatted clinical data. Aiming to make clinical data available when and where it is needed, the certification criteria for EHRs under HITECH requires the use of common messaging and vocabulary standards.
Logical Observation Identifiers Names and Codes (LOINC®) is a universal code system for identifying laboratory and clinical observations.9 LOINC is designed for use within messaging standards such as Health Level Seven (HL7). When LOINC and HL7 are used together, independent systems can electronically exchange test results with one another in an understandable way. This data exchange formula has been used successfully around the world. At present, LOINC is used in more than 164 countries by many kinds of organizations, including large reference laboratories, healthcare organizations, insurance companies, regional health information networks, and national standards.10,11 Over 25 countries, including the United States, have adopted LOINC as a national standard. The Meaningful Use program requires that LOINC be used in messages that report laboratory test results, exchange medical summaries, and send data to cancer registries and public health agencies.
Before healthcare organizations can leverage the value of unified data, they must first map their local test codes to codes in LOINC. Unfortunately, this process is complicated and resource-intensive.12–14 The detailed work of mapping requires expert domain knowledge of both the local tests performed and of LOINC. Often, the task is made more difficult because the information available from laboratory test master files lacks the specificity needed for accurate mapping. For example, if a test definition is missing specific information such as specimen type, a unit of measurement, or an example result value, it is impossible to accurately identify the correct LOINC term for that test. Reducing the effort required to accurately map local terms to LOINC would accelerate interoperable health information exchange and would be especially helpful for resource-challenged institutions.7,12,15,16
Prior studies have evaluated different informatics tools and techniques to assist mapping local laboratory tests to LOINC.17–22 Despite their promise, even the best of these approaches still require expert review to validate the computer-generated candidate mappings.
Fidahussein and Vreeman22 demonstrated that a large corpus of existing mappings could be used to help match local terms from novel institutions with LOINC terms. Encouraged by these results, we sought to create a large shared repository of local terms mapped to LOINC terms. Vreeman et al.23 described how a relatively small number of tests account for the vast majority of test result data volume. The popular, no-cost Regenstrief LOINC Mapping Assistant (RELMA®) program for mapping local terms to LOINC now carries a test frequency rank that can be used to narrow the list of candidate LOINC terms returned by the program.24 We therefore hypothesized that the number of organizations and local test terms that were mapped to a particular LOINC term would provide another frequency-based statistic that could inform the mapping process. Thus, we sought to build tools to help mappers from other institutions apply this “wisdom of the crowd.”
We previously described25 how we gathered existing sets of mappings from diverse institutions into a shared community repository and enhanced the RELMA mapping program to make use of this information. We programmed RELMA to display counts of local terms that had previously been mapped to a given LOINC term and also enabled users to view the details of those mappings, including local term names and the organizations that create the mappings. If information about existing mappings from other institutions helps people perform new mappings, such tools could improve the effectiveness and efficiency of mapping local vocabularies to standard ones.
In this study, we evaluate the perceptions of LOINC community members about this newly available “wisdom of the crowd” and how they used the new RELMA Community Mapping features. Specifically, the purposes of this study are to describe the perspectives of RELMA users before and after we deployed the Community Mapping features, characterize the usage of these new features, and analyze the quality of the mappings submitted to the Community Mapping repository.
METHODS
Overview of RELMA Community Mapping Features
From 2012–2013 we developed, then implemented a set of enhancements to the RELMA mapping program. We previously described25 our development methodology in detail. Briefly, we issued a call to the LOINC community in which we asked for voluntary contributions of existing local terms mapped to LOINC terms. Twenty-two organizations from five countries responded to our call and provided 102 484 total local mappings. These mappings were used to seed a shared online repository that we call the LOINC Community Repository. In parallel, we programmed RELMA to show counts of local terms and institutions mapped to a particular LOINC term (Figure 1) and the actual local test names and units of measure (Figure 2).
Figure 1:
Screenshot of the Regenstrief LOINC Mapping Assistant (RELMA) software showing the new functionality that displays the number of local terms and organizations previously mapped to a given Logical Observation Identifiers Names and Codes (LOINC) term. Fields other than the primary LOINC code and name have the following meanings: ExUCUMunits, Example units of measure from the Unified Code for Units of Measure (UCUM); ComMaps, Number of local test codes mapped to a given candidate LOINC term; ComInst, Number of institutions that have mapped to a given candidate LOINC term; Rank, Ranking within the LOINC Top 2000+ Lab Observations, a subset of LOINC codes that represents about 98% of the test volume from three large organizations that mapped all of their laboratory tests to LOINC codes.
Figure 2:
Screenshot of the Regenstrief LOINC Mapping Assistant (RELMA) software showing new functionality that displays detailed information on the local terms and organizations that previously mapped their local terms to a given Logical Observation Identifiers Names and Codes (LOINC) term.
The new information from the community mappings was made available to study participants as they used RELMA. We also built functions into RELMA that make it easy for users to contribute mappings they had completed back to the LOINC Community Repository. The new functionality was first released to the public with RELMA version 6.0 in December 2012, and we then began promoting it at meetings and presentations and on the website.26 Since that time, the Community Mapping features have continued to be available in subsequent releases of the software (released every 6 months, in June and December).
Study Design
In the present study, we performed a four-part evaluation of LOINC community members’ perceptions about the information provided by the LOINC Community Repository and how they used the new RELMA features. First, we conducted a pre-launch survey to capture perceptions of the LOINC community about the proposed functionality of those new features. Next, over the 1-year study period (all of 2013), we monitored how the new RELMA features and data provided by those features were accessed. Then, we conducted a follow-up survey of users who had volunteered to try the software’s new features. Lastly, we analyzed the community mappings using automated methods to detect potential errors in the contributed mappings. This study was approved by the Institutional Review Board at Indiana University (Protocol #1206008989).
Pre-Launch Survey
Just prior to launching the new RELMA features, we conducted a convenience survey of LOINC community members, asking them for their perceptions of the proposed functionality of these new features. We invited all 3554 users on the LOINC mailing list to participate in the survey with an email that contained a link to our online questionnaire. Data were collected over a 1-month period between November 26 and December 31, 2012.
The study team designed the prelaunch survey and revised it based on pilot testing with volunteers from the LOINC Committee, a group of experts who advise Regenstrief on LOINC development. Survey respondents were asked about their role in mapping local test terms to LOINC terms, which software tools they used for mapping, their current perceptions of the RELMA program, and their opinions on the potential value of information about other institutions’ mappings. The survey respondents’ perceptions and opinions were assessed using a five-point Likert scale ranging from “Strongly Disagree” to “Strongly Agree.”
We received survey responses from 182 members of the LOINC community. Survey respondents were eligible for our analysis if they were directly involved in mapping local terms to LOINC terms. Of the total respondents, 155 (85.2%) indicated they were directly involved in mapping efforts.
Usage Monitoring
After the launch of the new RELMA features, we monitored study participants’ adoption of these features and their use of the new functionality for the entire study period. We used server log files and website statistics from Google Analytics to track how often RELMA was downloaded and how often study participants accessed RELMA’s Community Mapping features. During the study period, new community mappings submitted to the LOINC team were collected, reviewed for conformance to our data format, and then added to the LOINC Community Repository.
Post-Launch Survey
At the end of the study period, we conducted a follow-up convenience survey of the study participants. We invited all 200 study participants to complete the survey via an email that contained a link to an online questionnaire. Fifty participants (25%) submitted responses to the questionnaire. Data were collected over a 3-month period (from October to December 2013).
The study team designed the post-launch survey and revised it based on pilot testing with volunteers from the LOINC Committee. Respondents were asked about their role in mapping local test codes to LOINC codes, which software tools they used for mapping, and their current perceptions of the RELMA program with the new features that allow users to view detailed information about other institutions’ mappings. The survey respondents’ perceptions and opinions were assessed using a five-point Likert scale ranging from “Strongly Disagree” to “Strongly Agree.”
Review of Community-Contributed Mappings
In addition to surveying community members and monitoring the use of the new RELMA features, we also analyzed the local test-to-LOINC mappings submitted to the LOINC Community Repository. Lin et al.14 demonstrated that about 4% of mappings are not mapped to a correct LOINC term, even when the mappings are performed by expert mappers. Yet, because of the well-documented ambiguity in local test naming conventions,12–14,27 it was not possible for us to determine with certainty whether a given mapping was correct or not. For example, in the study by Lin et al.,14 64% of the local test names did not contain information about the specimen type. Thus, a thorough review of the mappings submitted to the Community Repository would require dialogue with the local laboratory personnel in order to validate missing information. Such a resource-intensive process was out of the scope of the current study. Therefore, our approach to validating LOINC mappings relied on automated tools.
As described in the software documentation, the RELMA program contains several safeguards to protect against assigning erroneous mappings.28 Before adding any mapping contribution to the shared repository, files are checked for basic format consistency and to ensure that all LOINC identifiers are valid terms. Regenstrief has also developed a set of executable mapping validation rules. These rules are employed in production system environments at Regenstrief to flag semantic relationships that need further expert human review. These validation rules are not absolute, because there can be rare exceptions. For example, rules that check the local unit of measurement cannot account for all of the string variations that laboratories use. Although these validation rules are imperfect, we selected a subset of rules to use when analyzing the submitted community LOINC mappings for known quality issues. Table 1 lists the validation rules we selected for our analysis.
Table 1:
Automated Validation Rules Used to Assess the Accuracy of LOINC Mappings
| Rule Name | Rule Description |
|---|---|
| Unknown local units | Unrecognized local units of measure |
| Deprecated LOINC | Local terms should not be mapped to a LOINC term that has a status of “deprecated” |
| Discouraged LOINC | Local terms should not be mapped to a LOINC term that has a status of “discouraged” |
| Property/units mismatch | The local units of measurement should be consistent with the property of the LOINC term it is mapped to. For example, a test with units of “mcg/24 h” should be mapped to a term with a property of mass rate (MRat). |
| Scale/units mismatch | The local units of measurement should be consistent with the scale of the LOINC term it is mapped to. For example, a test with units should be mapped to a quantitative LOINC term, not an ordinal term. |
| Concentration missing denominator | If the LOINC term in a mapping represents a concentration, the units of measurement for the local term should contain a denominator. |
Some of the local test records submitted to the LOINC Community Repository contained more than one unit of measurement. For the purposes of our evaluation, we treated each test code and unit of measurement pair as a record. Thus, the total number of mappings evaluated was 102 579, which is slightly higher than the number of distinct local codes.
RESULTS
Pre-Launch Survey
Many of the pre-launch survey respondents were relatively new LOINC mappers. Of the 155 survey respondents, almost half had been involved in mapping for <1 year (48%, n = 74), a third had been mapping for 1–5 years (33%, n = 51), and relatively few had been mapping for > 5 years (19%, n = 30). About half of respondents indicated that their highest level of education was bachelor’s degree (47%, n = 73), with fewer having a master’s degree (25%, n = 39) or a doctorate degree (14%, n = 21). The most prevalent professional credentials held by the survey respondents were medical technologist or biomedical scientist (41%, n = 64) and the American Society for Clinical Pathology certification (27%, n = 41).
Table 2 summarizes the responses to questions on the pre-launch survey regarding the existing RELMA software and the potential usefulness of information about other institutions’ mappings. All survey respondents (n = 155) answered the question regarding their current use of RELMA. Of those respondents that were eligible to answer the questions about RELMA and the proposed functionality of the software’s new features (n = 125), 116 (92.8%) answered questions about the existing RELMA software and 98 (78.4%) completed questions about the proposed functionality of the new features.
Table 2:
Pre-Launch Survey Responses About RELMA and Proposed New Features
| Respondent Characteristic | n (%) |
|---|---|
| n = 155 | |
| Currently uses RELMA program | |
| Yes | 125 (80.7) |
| No | 30 (19.3) |
| n = 116 | |
| I am able to access the information I need to complete a mapping between my organization’s local concept and a LOINC code using RELMA. | |
| Positive | 91 (78.4) |
| Neutral | 13 (11.2) |
| Negative | 12 (10.3) |
| The information presented in the RELMA software tool is easy to understand. | |
| Positive | 72 (62.1) |
| Neutral | 21 (18.1) |
| Negative | 21 (18.1) |
| n = 98 | |
| Information on the number of organizations that have previously mapped a particular LOINC to a local data dictionary concept would be useful. | |
| Positive | 76 (77.6) |
| Neutral | 17 (17.3) |
| Negative | 5 (5.1) |
| Detailed information on other organizations’ local concept mappings to LOINC codes would be useful. | |
| Positive | 68 (70.8) |
| Neutral | 24 (25.0) |
| Negative | 4 (4.2) |
Overall, the survey respondents gave a positive assessment of the existing RELMA software, with more than three quarters of the respondents (n = 91) indicating that it provided the information they needed to map local terms to LOINC terms. Despite the survey respondents’ general satisfaction with RELMA, nearly 80% (n = 76) also indicated that having information on how often other users had mapped to a particular LOINC term would be helpful. A slightly lower percentage (71%, n = 68) of the survey respondents indicated that detailed information (eg, local test name, units of measurement) from other organizations’ mappings would be helpful.
Usage Monitoring
During the 12 months of our study period that RELMA was available with the Community Mapping features, the software was downloaded 10 617 times. At the close of the study period, 200 study participants from 15 countries had consented and were eligible to access the Community Mapping features in RELMA. Each time participants started the RELMA program, they were asked to log in, and, once logged in, their session continued until they exited the program. During the study period, the 200 study participants logged into the Community Mapping features of RELMA an average of 610 times per month (a total of 7235 times, for all users). The average number of times a user logged into the Community Mapping features during the study period was 40 (range = 1–475).
While using RELMA, study participants viewed the mapping detail pages (which showed the local test terms mapped to a particular LOINC term) a combined total of 6686 times. When looking at these detail pages, users spent an average of 3.1 min on each page. This is slightly longer than the average time of 2.5 min per page that these users spent reviewing the main details pages for LOINC terms (which showed the LOINC term display names, definition, etc., but not the mappings of local terms).
Since launching the initial LOINC Community Repository, we have received eight additional submissions of local tests mapped to LOINC. There are now 35 local institution mapping files in the repository, containing 102 484 local term-to-LOINC term mappings. The mapping files range in size from 105 to over 25 700 local terms mapped to LOINC terms.
Post-Launch Survey
Similar to our pre-launch survey, respondents to our post-launch survey were relatively novice LOINC mappers. Of the 50 post-launch survey respondents, 44% (n = 22) had been involved in mapping for < 1 year, 29% (n = 16) had been mapping for 1–5 years, and 20% (n = 10) had been mapping for > 5 years. Post-launch survey respondents were largely educated at the bachelor’s degree (40%, n = 19) or master’s degree (38%, n = 18) levels. A lower percentage of respondents had a doctorate degree (15%, n = 7) or an associate’s degree (6%, n = 3). The most common professional credential held by respondents was medical technologist or biomedical scientist (60%, n = 29).
Of those post-launch survey respondents who used the RELMA program for mapping and viewed the Community Mapping features (n = 18), all indicated that these features were useful. Similarly, those respondents who reviewed the detailed information about each local term mapped to a LOINC term unanimously (n = 18) agreed that this information was useful. Likewise, almost all of this group of survey respondents (n = 17) indicated that the information about others institutions’ mappings influenced their own mapping choices. One respondent indicated that they were unsure whether the Community Mapping features were useful.
Review of Community-Contributed Mappings
Overall, 95.3% of the mappings submitted to the LOINC Community Repository passed our automated validation checks. Across those institutions that provide data to the repository, the percentage of mappings that passed all of our automated quality checks ranged from 67%–100%. Table 3 presents the number of mappings with issues that were detected by our automated rules. The most frequent potential problem we detected was a mismatch between the property of the LOINC term chosen and the units of measurement associated with the local term.
Table 3:
Automated Validation Results for Detected Issues in Local Mappings
| Rule Name | n (%) |
|---|---|
| Unknown local units | 1453 (1.4) |
| Deprecated LOINC | 318 (0.3) |
| Discouraged LOINC | 633 (0.6) |
| Property/units mismatch | 2326 (2.3) |
| Scale/units mismatch | 266 (0.3) |
| Concentration missing denominator | 1688 (1.6) |
For example, a local term for measuring a plasma lactate level was mapped to the LOINC term Lactate [Mass/volume] in Serum or Plasma (14118-4). Because the units of measurement associated with this local term were millimoles per liter, this mapping was flagged as having a mismatch between the property of the LOINC term and the local units. If the reporting units are correct (which we suspect they are), then this local term should have been mapped to the LOINC term Lactate [Moles/volume] in Serum or Plasma (2524-7).
DISCUSSION
We developed, launched, and evaluated enhancements to the RELMA mapping program that enable access to information about mappings from other institutions. A modest, but adequate and representative, sample of the LOINC community provided early input on the need for and usefulness of the information. Overall, these community members felt that having access to information about other institutions’ mappings would aid their own mapping efforts. The majority of the community members we heard from indicated that knowing how often other users had mapped to a particular LOINC term and detailed information such as local test name and units of measurements from other organizations’ mappings would be useful. After such information was made available, the smaller set of users who accessed it and responded to our post-launch survey unanimously agreed that it was useful. Furthermore, nearly all of these users indicated that the information on others’ mappings informed their own mapping choices. Our findings suggest that a crowd-sourced repository of mappings is valuable to users who are mapping local terms to LOINC terms.
National health information technology policies in the United States are exerting pressure to get mappings from local laboratory terms to LOINC terms in place quickly.5 In 2012 and 2013, while we were preparing and conducting this study, the number of registered LOINC users grew by more than 14 000. We anticipate that this pressure will continue under the activities outlined in the 2015–2020 Federal Health IT Strategic Plan29 and Meaningful Use Stage 3. However, mapping is a complex, time-consuming, and often expensive process. Our results indicate that study participants found the new RELMA features and crowd-sourced repository of mappings helpful and used them regularly. Moreover, nearly half of respondents to each of the surveys we conducted were new to mapping (<1 year), suggesting that the new RELMA features address the information needs of those entering the LOINC community in response to health information technology policies.
Based on previous work with expert LOINC mappers,14 we expected a higher error rate in the mappings collected from the broader LOINC community. However, we observed that just 5% of local term mappings might have problems, as detected by our automated checks. This error rate is very close to the 4% error rate among mappings performed by experts. We did not conduct a manual review of the community-supplied mappings, so it is probable that the true error rate of these mappings is higher. Nevertheless, our findings suggest that the Community Repository is of sufficient quality to inform new LOINC mappings.
While our study was focused on crowd-sourced mappings to LOINC, we hypothesize that our findings would apply to the use of other standard vocabularies, such as Systematized Nomenclature of Medicine - Clinical Terms (SNOMED CT). Tools and techniques that reduce the burden of mapping may help move the nation more quickly towards the goal of interoperable health data exchange.
Our study has some limitations. We did not assess mapping quality or efficiency with a controlled trial of the Community Mapping features. Our observations are instead based on usage data and survey responses, which limits our ability to draw definitive conclusions about whether these features will result in better or faster mappings. Rigorous methods are preferable, yet it is not always possible to conduct controlled studies. Because LOINC and RELMA are funded and released publicly, it is difficult to withhold functionality from some users, as would be necessary in such a trial. Furthermore, we were unable to explore negative survey responses. Although there were only a few negative responses (4–5 respondents), negative views can yield valuable insights. The surveys we conducted did ask open-ended questions, which only a few respondents utilized. For example, in the post-launch survey, when asked what enhancements would better support mapping local terms to LOINC, one respondent answered “Being able to clone ourselves.” While illustrative of the inherent challenges of mapping, more constructive criticism from dissenting respondents would have been valuable.
The value of the LOINC Community Repository depends on its size and the representativeness of its contents. We optimistically expected to receive contributions from more institutions than we actually did during the study. Several members of the LOINC community who initially indicated that their organization could likely contribute mappings were ultimately unable to do so, despite several prompts from the LOINC team. Common reasons given by institutions for their lack of participation included: 1) they lacked the resources to extract the mappings, 2) they could not get organizational approval to release the mappings publicly, and 3) they did not consider their mappings “finalized.” Hesitation to contribute mappings to the repository remains an ongoing challenge that could limit the long-term sustainability of the repository. We can appreciate the social-, political-, and business-related barriers to sharing mappings. Despite these barriers, we would advocate that such mappings should be viewed not as proprietary knowledge, but rather as open data that could help the clinical field reach the goal of ubiquitous and interoperable health information exchange more readily.
Because of this study’s early findings, the Regenstrief LOINC team decided to continue making the new RELMA features available to the LOINC community. The current version of RELMA contains the Community Mapping features and is available at no cost from the LOINC website (http://loinc.org). In addition, Regenstrief continues to welcome and support additional contributions of mappings to the Community Mapping repository.
CONCLUSION
Reducing the effort required to accurately map local terms to LOINC might accelerate interoperable health information exchange. We evaluated LOINC community members’ perceptions about new features in RELMA that made available the “wisdom of the crowd” in regards to mapping local terms to LOINC terms and analyzed how members used the these new features. When information about other institutions’ mappings was made available, study participants who accessed it considered it to be useful and reported that it informed their own mapping choices. Our findings suggest that a crowd-sourced repository of mappings is valuable to users who are mapping local terms to LOINC terms.
FUNDING
This work was made possible, in part, by support from the Indiana Clinical and Translational Sciences Institute which is funded, in part, by Grant Number TR000006 from the National Institutes of Health, National Center for Advancing Translational Sciences, Clinical and Translational Sciences Award (B.E.D. and D.J.V.), a contract (HHSN2762008000006C) from the National Library of Medicine (D.J.V.), and by the Department of Veterans Affairs, Veterans Health Administration, Health Services Research and Development Service CIN 13-416 (B.E.D.). Dr Dixon is a Health Research Scientist at the Richard L. Roudebush Veterans Affairs Medical Center in Indianapolis, Indiana. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Clinical and Translational Sciences Institute (CTSI), the National Library of Medicine (NLM), the Department of Veterans Affairs, or the federal Government.
COMPETING INTERESTS
All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf and declare: DJV, JH, and BED reports support from National Institutes of Health, National Center for Advancing Translational Science for the submitted work; DJV and JH reports grants from National Institutes of Health, National Library of Medicine during the conduct of the study. BED reports grants from Department of Veterans Affairs, Veterans Health Administration, Health Services Research and Development Service, during the conduct of the study; DJV received research support from Siemens Medical Solutions and bioMérieux outside the submitted work in the past three years; no other relationships or activities that could appear to have influenced the submitted work.
CONTRIBUTORS
DJV and BED guided the design of the study. J Hook designed the software enhancements to RELMA. DJV drafted the article. BED provided critical revision of the article. DJV finalized the article for publication. All authors reviewed and approved the final version.
Acknowledgments
The authors would like to thank the Regenstrief LOINC team for their dedicated work and support of this project. We further thank Masoud Hosseini, MS, a doctoral student the Indiana University School of Informatics and Computing, for his assistance in summarizing the survey data to aid in our analysis.
REFERENCES
- 1.Stroetmann KA, Artmann J, Stroetmann V. Developing national eHealth infrastructures–results and lessons from Europe. AMIA Annu Symp Proc. 2011;2011:1347–1354. [PMC free article] [PubMed] [Google Scholar]
- 2.Stanimirović D, Vintar M. Development of eHealth at a national level - comparative aspects and mapping of general success factors. Inform Health Soc Care. 2014;39:140–160. [DOI] [PubMed] [Google Scholar]
- 3.Currie WL, Seddon JJM. A cross-national analysis of eHealth in the European Union: Some policy and research directions. Inf Manage. 2014;51:783–797. [Google Scholar]
- 4.Adler-Milstein J, Sarma N, Woskie LR, et al. A comparison of how four countries use health IT to support care for people with chronic conditions. Health Aff. 2014;33:1559–1566. [DOI] [PubMed] [Google Scholar]
- 5.The Office of the National Coordinator for Health Information Technology (ONC), The Office of the Secretary, United States Department of Health and Human Services. Update on the adoption of health information technology and related efforts to facilitate the electronic use and exchange of health information. U.S. Department of Health and Human Services, 2014. Available at: http://www.healthit.gov/sites/default/files/rtc_adoption_and_exchange9302014.pdf.
- 6.Finnell JT, Overhage JM, Grannis S. All health care is not local: an evaluation of the distribution of Emergency Department care delivered in Indiana. AMIA Annu Symp Proc. 2011;2011:409-416. [PMC free article] [PubMed] [Google Scholar]
- 7.Dixon BE, Vreeman DJ, Grannis SJ. The long road to semantic interoperability in support of public health: experiences from two states. J Biomed Inform. 2014;49:3–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Dixon BE, Colvard C, Tierney WM. Identifying health facilities outside the enterprise: challenges and strategies for supporting health reform and meaningful use. Inform Health Soc Care. 2014;1–15. doi: 10.3109/17538157.2014.924949. [DOI] [PubMed] [Google Scholar]
- 9.McDonald CJ, Huff SM, Suico JG, et al. LOINC, a universal standard for identifying laboratory observations: a 5-year update. Clin Chem. 2003;49:624–633. [DOI] [PubMed] [Google Scholar]
- 10.Vreeman DJ. LOINC Introduction and Overview. http://loinc.org/slideshows. Accessed November 1, 2014. [Google Scholar]
- 11.Vreeman DJ, Chiaravalloti MT, Hook J, et al. Enabling international adoption of LOINC through translation. J Biomed Inform. 2012;45:667–673. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Baorto DM, Cimino JJ, Parvin CA, et al. Combining laboratory data sets from multiple institutions using the logical observation identifier names and codes (LOINC). Int J Med Inf. 1998;51:29–37. [DOI] [PubMed] [Google Scholar]
- 13.Lin M-C, Vreeman DJ, Huff SM. Investigating the semantic interoperability of laboratory data exchanged using LOINC codes in three large institutions. AMIA Annu Symp Proc. 2011;2011:805-814. [PMC free article] [PubMed] [Google Scholar]
- 14.Lin M-C, Vreeman DJ, McDonald CJ, et al. Correctness of voluntary LOINC mapping for laboratory tests in three large institutions. AMIA Annu Symp Proc. 2010;2010:447-451. [PMC free article] [PubMed] [Google Scholar]
- 15.Williams C, Mostashari F, Mertz K, et al. From the Office of the National Coordinator: the strategy for advancing the exchange of health information. Health Aff. 2012;31:527-536. [DOI] [PubMed] [Google Scholar]
- 16.Were MC, Mamlin BW, Tierney WM, et al. Concept dictionary creation and maintenance under resource constraints: lessons from the AMPATH Medical Record System. AMIA Annu Symp Proc. 2007;791–795. [PMC free article] [PubMed] [Google Scholar]
- 17.Vreeman DJ, McDonald CJ. A comparison of Intelligent Mapper and document similarity scores for mapping local radiology terms to LOINC. AMIA Annu Symp Proc. 2006;809–813. [PMC free article] [PubMed] [Google Scholar]
- 18.Vreeman DJ, McDonald CJ. Automated mapping of local radiology terms to LOINC. AMIA Annu Symp Proc. 2005;769–773. [PMC free article] [PubMed] [Google Scholar]
- 19.Khan AN, Griffith SP, Moore C, et al. Standardizing laboratory data by mapping to LOINC. JAMIA. 2006;13:353–355. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Lau LM, Johnson K, Monson K, et al. A method for the automated mapping of laboratory results to LOINC. AMIA Annu Symp Proc. 2000;472–476. [PMC free article] [PubMed] [Google Scholar]
- 21.Zunner C, Bürkle T, Prokosch H-U, Ganslandt T. Mapping local laboratory interface terms to LOINC at a German university hospital using RELMA V.5: a semi-automated approach. JAMIA. 2013;20(2):293–297. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Fidahussein M, Vreeman DJ. A corpus-based approach for automated LOINC mapping. JAMIA. 2014;21(1):64–72. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Vreeman DJ, Finnell JT, Overhage JM. A rationale for parsimonious laboratory term mapping by frequency. AMIA Annu Symp Proc. 2007;771–775. [PMC free article] [PubMed] [Google Scholar]
- 24.Regenstrief Institute, Inc. Common LOINC Laboratory Observation Codes. http://loinc.org/usage/obs. Accessed January 2, 2015.
- 25.Dixon BE, Hook J, Vreeman DJ. Learning from the crowd in terminology mapping: the LOINC experience. Lab Med. 2015;46(2):168–174. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Regenstrief Institute, Inc. LOINC Version 2.42 and RELMA Version 6.0. 2012. http://loinc.org/news/loinc-version-2-42-and-relma-version-6-0-available.html. Accessed November 1, 2014.
- 27.Kim H, El-Kareh R, Goel A, et al. An approach to improve LOINC mapping through augmentation of local test names. J Biomed Inform. 2012;45:651–657. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Regenstrief Institute, Inc. RELMA Users’ Manual. Version 6.0. Regenstrief Institute, Inc.; 2012. Available at LOINC: http://loinc.org/downloads/relma/.
- 29.The Office of the National Coordinator for Health Information Technology (ONC), The Office of the Secretary, United States Department of Health and Human Services. Federal Health IT Strategic Plan 2015-2010. 2015. http://www.healthit.gov/sites/default/files/federal-healthIT-strategic-plan-2014.pdf. Accessed February 20, 2015.