Abstract
Postmarketing drug surveillance is critical to assessing adverse events associated with medications, because prelaunch clinical trials frequently miss negative drug effects. The Informatics for Integrating Biology and the Bedside platform (i2b2) has been used effectively for this. However, previous work suffers from incomplete medical data present in electronic health record (EHR) systems. Here, we develop a system to integrate non-traditional data sources with EHR data: pharmacy dispensing information and patient-reported data. We implement and validate a toolset to gather medication data from a Pharmacy Benefit Manager network, import it into an i2b2 EHR repository using a standard data format, merge it with the EHR data, and present it to for annotation with results returned to i2b2. This toolkit is enabling studies on medication list data quality, adherence, and adverse event detection.
Introduction
Postmarketing drug surveillance is critical to assessing adverse events associated with medications, because prelaunch clinical trials frequently miss negative drug effects. Such clinical trials have limitations in population size, diversity, disease comorbidities, and concurrent medication use. [1] Incident Reporting Systems, such as FDA MedWatch, are used to manually report postmarketing adverse events. However, their utility is limited by underreporting and poor compliance by physicians. [2] The Informatics for Integrating Biology and the Bedside platform (i2b2) has been used effectively for postmarketing drug surveillance. [3,4] i2b2, the output of an NIH National Center for Biomedical Computing grant, is an open source medical data analytics platform which is in use at over 100 sites nationwide that house data on over 90 million patients.
However, incomplete medical data is rampant, and because i2b2 relies on the “ambient” data generated by the electronic health record systems, it suffers as well. Medication histories are frequently poorly maintained, which is not aided by the significant fragmentation of care across sites (increasing the likelihood of incomplete data at any given site). [5–7]
We hypothesize that it is possible to improve the medication list by enlisting nontraditional data sources. We have begun to explore two sources thus far.
First, medication dispensing information, recorded by pharmacies when patients pick up medications, is fueled by the pharmacy supply chain. The pharmacy supply chain already has rather advanced data networks, traditionally used for payments, claim adjudications, and inventory management. [8] These data can be used as a source for postmarket drug surveillance. [9]
Second, the importance of patient reported data is increasingly becoming recognized. [10,11] Such data have been found to be timely and reliable, and patients are frequently willing to share and participate in their healthcare. [12,13] Along these lines, the Patient Centered Outcomes Research Institute recognizes and emphasizes patient centered research. They are leading “a national movement that meaningfully involves patients and incorporates their voices in producing trusted, evidence-based information, promoting better decisions and ultimately better health for all.” [14]
We previously developed the i2b2 Standards-based Extract, Transform, and Load (SETL) tool, which integrates i2b2 data with data in a standardized format, the Consolidated Clinical Document Architecture (C-CDA). [15] This work was timely because an embodiment of C-CDA is required by Meaningful Use and is therefore becoming widely available. [16,17] Our previous work supports import of medication prescribing events in C-CDA, which originate from the EHR. However, C-CDA can also represent many types of medication events with high granularity, including prescribing, billing, and dispensing events.
In this manuscript, we leverage our previous work on the SETL tool to develop a patient-centered medication annotation platform for i2b2 that can import granular medication information from a C-CDA source, combine it with existing EHR data stored in i2b2, and enable multi-source patient medication annotation via a mobile app. We validate this platform by capturing pharmacy-benefit-manager-derived (PBM) data from a commercial PBM that is a major source of pharmacy dispensing data into i2b2 on an on-demand basis. We transfer PBM data into the i2b2 clinical instance at Boston Children’s Hospital alongside EHR data, and validate the app’s ability to interpret these data for patient action. This platform will enable forthcoming research in medication data quality, adherence, and adverse event detection. Presently, researchers are preparing to use this platform to study medication adherence by assessing the discordance between the clinical medication list and a patient-generated list.
Methods
Design
C-CDA
The C-CDA is a rich but complex XML document language for healthcare derived from the Health Level Seven (HL7) Reference Information Model. [16] It is elegant but so expressive that differences of opinion arise on the best way to represent healthcare events. A great deal of work has gone into defining Meaningful Use templates for the Continuity of Care Documents required by that program. However, no such templates exist for the differentiation of medication prescribing, dispensing, billing, and patient-provided medication data. We augmented the Meaningful Use templates with existing C-CDA templates to support the complete scope of medical data required for this study.
HL7v2 and PBM
PBM data is conveyed in record-based HL7 version 2 format (HL7v2), which is a declining but still predominant format for the exchange of healthcare data. Although used as foundation for most older interoperable healthcare systems, its record-based format is recognized as limited and many data sources that use HL7v2 add custom extensions to the format in order to achieve expressiveness; this can be a hindrance to interoperability. We developed a mapping from the PBM elements of HL7v2 to C-CDA.
i2b2 and Patient Data Objects
The i2b2 software is compartmentalized into functional units known as cells. These cells, which collectively make up the i2b2 hive, interact through web service calls. Clinical data is exchanged with these web services in the i2b2 Patient Data Object format, which is a denormalized data structure that reflects the underlying star schema design of the i2b2 database. [18] The atom of a PDO is the observation. Fact observations record clinical information associated with the patient, such as a medication he/she is taking. Modifier observations are linked to fact observations and provide additional information, such as medication route, dose, and information source (e.g., medication list, dispensing event, or patient report). The set of possible observations are defined as trees of facts and modifiers through i2b2 ontology services. Although some standard ontology trees are emerging as i2b2 is increasingly used for interoperable networks (for which a standardized ontological information model must be defined) [19], i2b2’s general position has been to leave development of the ontology trees to individual use cases, thus maximizing flexibility. We expanded our previous C-CDA to PDO mapping to include the more complex C-CDA documents used here.
System description
The architecture of the platform is outlined in Figure 1. This necessitated development of three new software components:
-
A new i2b2 cell, the i2me2 cell. It includes two main functions, which are similar conceptually to previous work, but with important differences:
- ○ The first function is similar to the SETL cell previously described: transform data in a standard format (C-CDA) into i2b2 PDO format. [15] Like the SETL cell, the i2me2 C-CDA transformation functionality relies on the Model Driven Message Interoperability (MDMI) toolkit, and it loads data into the i2b2 EHR repository using the i2b2 Clinical Research Chart (CRC) Loader web service. Unlike the SETL cell, the i2me2 cell supports the more complex medication C-CDAs developed here.
- ○ Second, the i2me2 cell manages authentication, authorization, and communication with the medication annotation app. Authentication and authorization are accomplished separately from i2b2 in order to accommodate the enhanced granularity of access control and auditing required. This additional layer of authorization augments i2b2’s access controls, which offer access levels (such as full access or aggregate counts only) only across the entire repository, due to its cohort study focus. The i2me2 authorization schema enables subjects full access to only their own data, augmented by personally identifying information not available to i2b2. This is accomplished through an Identity and Demographics Module (IDM) within i2me2. The cell transmits a subject-specific token to the annotation app that is first used to request limited demographic data (e.g. name) to confirm the identity of the subject. This token is then passed back to i2me2 with a request for the subject’s medication data. Upon receipt of this token, the i2me2 cell obtains medication data from i2b2 and transmits it to the app, possibly requesting medication information from RxConnect if PBM data has not already been imported into the i2b2 EHR repository. Finally, following use of the app, the updated medication list is transmitted from the app to i2me2 in PDO format. This is added to the i2b2 repository by i2me2 using the CRC Loader service. This follows a similar approach to the previously described i2b2-SMART cell, which uses the SMART RDF format to provide medical apps with subject-specific i2b2 data. [20] Because the i2b2-SMART cell presently supports neither bi-directional communication nor differentiation of medication event type (e.g., prescribing or dispensing), we currently use this more direct PDO approach.
A software component, RxConnect, provides a synchronous interface to the PBM web service RxConnect receives patient demographic information from the IDM module via i2me2, which is used to identify a subject in the PBM patient index. The PBM then asynchronously returns medication dispense and claims information for the respective subject. RxConnect receives the HL7v2 response message from the PBM, parsing this using the HAPI open-source toolkit (http://hl7api.sourceforge.net). The returned message is processed, filtered of personally identifying information, and then converted to C-CDA via MDMI.
A patient-facing application enabling patient input into the i2b2 medication data model. The app consumes an i2b2 PDO for a single subject that can reflect medication data from multiple sources. A list of deduplicated medications is generated for presentation, one medication at a time, to the subject performing medication self-validation. Using a two-step process, the subject indicates if and how often a medication is taken. Missing medications may also be added. The patient-annotated medication list is then added back to the i2b2 repository to as patient reported data, which exists alongside (and not in replacement of) other sources of medication data in the repository. This multi-sourced data is then available to researchers for analyses.
Figure 1:
The “i2me2” architecture, outfitted to combine i2b2 EHR medication data with PBM dispensing and claims data, as well as patient-reported medication information. Data flows from the PBM into i2b2 (steps 1–3), then to the subject for annotating corrections, and finally back to the i2b2 data repository. (Steps 4–5) The data are transformed from HL7v2 to the standard C-CDA to i2b2 Patient Data Objects.
The data conversions from the record-based HL7v2 format to the document based C-CDA format to the denormalized PDO format were developed with the MDMI toolkit, which is an open source project in the Open Health Tools (OHT) consortium that seeks to make interoperability a commodity. [21] MDMI provides a graphical data-mapping tool that supports rapid design and iterative development and can move data in any arbitrary data format to and from the MDMI model, known as the Referent Index. This Referent Index is an evolving standard that attempts to represent all possible healthcare data using a common semantic structure based elements identified in the C-CDA specification. By divorcing semantics from data structure, MDMI provides a unique approach to move data between different types of formats (in this case, record based, document-based, and denormalized).
These software components enable nontraditional data sources (pharmacy dispensing data and patient annotation information) to be added to an i2b2 EHR repository of medication data in five steps (shown in Figure 1).
Subject information is requested from the PBM network, which asynchronously provides dispensing and insurance claims information for a particular subject in HLv2 format. This may be scheduled or on-demand.
RxConnect processes the message, converts it to C-CDA, and sends it to the i2me2 cell.
The i2me2 cell converts the incoming message from C-CDA to i2b2 PDO format as needed, and loads this data into the i2b2 repository alongside the pre-existing EHR data.
When a subject is ready to annotate their medication information, the i2me2 cell requests a combined PDO containing both dispensing data (from the PBM) and prescribing data (from the i2b2 EHR repository). The PDO is then forwarded securely to the app.
Subjects use the app to construct a current medication list that is added back to the i2b2 EHR repository as patient reported data to exist alongside (and not in replacement of) other sources of medication data in the repository.
The output of the five steps is a rich database of medication information containing prescribing, dispensing, and patient-reported medication data, as well as other i2b2 data sources including EHR and disease registry medication data. This will enable more accurate adverse drug event research. All data in the i2b2 repository can be studied via the i2b2 web client query tool, which supports dimensional data queries using the ontology hierarchies.
This patient annotation platform is not intended for clinical use at this time – the required quality testing would be a costly and time-consuming endeavor that is unnecessary for research use. The platform is research-focused and its usage will vary depending on the study question. Patients must be recruited for a particular IRB-approved study question before the five steps above can be performed.
Results
Implementation
In this proof-of-principle, we implemented the architecture in Figure 1 at Boston Children’s Hospital and obtained 11 messages from the PBM corresponding to 11 synthetic patients intended for testing. We used these cases to develop our HL7v2 to C-CDA to PDO mappings, using the graphical MDMI Map Editor.
Our approach to constructing valid C-CDA and PDO messages involved a number of considerations due to complexities in data transformation.
First, the meaning of the HL7v2 medication order segment (i.e. dispensing vs. payment event) is determined by a custom element defined by the PBM. Although the MDMI specification is designed to support such overloaded fields through “linked data” functionality, the current MDMI transformation engine did not provide this functionality, as MDMI’s focus is declarative transforms. Declarative transforms encourage map developers to adopt straightforward designs that only rely on the semantic meaning of each individual element in hierarchical data and not the content of sibling elements. In this case, however, the sibling directly determines the semantic meaning. We worked with the MDMI group to implement this “linked data” functionality.
Second, MDMI was designed to expect both source and target messages to follow roughly the same hierarchy (e.g., all information on a medication is a child of that medication). C-CDA employs hierarchical information to embed observation details (e.g., medication dose and frequency), whereas PDO is a flat structure that uses a sequence of linked observations (a fact and its modifiers). In order to support modifiers, we worked with the MDMI group to add a feature to ‘flatten’ hierarchical information. Presently this feature is only supported uni-directionally, so it is not possible to move from PDO to other more compact formats using MDMI at present.
Third, the complexity of tracking and testing transformations between three different formats mandated improvements in existing debugging tools and techniques. Using MDMI’s new end-to-end traceability application, we produced traceability reports for these maps. This application takes maps and generates a report on how each data element in the various formats can be traced to the other data elements in the other formats. This tool should be valuable in future MDMI i2b2 projects.
A simplified example of a single medication dispense event in the three formats is shown in Figure 2. The example includes data elements needed to create a valid i2b2 observation as well as information on dispense quantity and instructions. The HL7v2 format is structured around “segments”: the PID segment provides patient information, the RDS segment provides pharmacy dispensing data, the ORC sub-segment provides metadata about the order, and the RXD subsegment provides data on the drug being dispensed. The C-CDA is a deeply nested XML structure that puts all information under a “substance administration supply” event (except the patient ID, which is not shown for compactness). Notice that some pieces of information require even more deeply nested structures under the supply event (e.g., patient instructions are four levels deeper). In contrast, the i2b2 format is a nearly flat structure organized into a sequence of observations. This example shows two linked modifier observations: dispense quantity and instructions. The core components of the observation are repeated for each modifier.
Figure 2:
An example of a medication dispensing event in the three data formats: HL7v2, C-CDA, and i2b2 PDO. HL7v2 is presented in a condensed XML format for readability – the actual format is a very terse pipe-delimited structure that is not human-readable. Notice the record-based format for HL7v2, the deeply-nested document-based format for C-CDA, and the denormalized PDO format (wherein a single logical observation requires multiple i2b2 “observations”).
The C-CDA and PDO mappings will be released open source in Open Health Tools with the enhanced MDMI toolkit, which includes the transformation engine and the graphical map editor (http://sourceforge.net/projects/mdmi). [21] The HL7v2 mappings reference proprietary PBM data structures and cannot be released.
Finally, we developed i2b2 ontologies necessary for querying the prescribing, dispensing, and payment data. Because these ontologies are based on the C-CDA, we believe these are the beginning of standards-based medication ontology for i2b2. The medication tree was based on our previously developed RxNorm tree organized by Veteran’s Administration (VA) drug class. [15] Because the dispensing data is in National Drug Code (NDC) format, we used the linkage information present in the Unified Medical Language System (UMLS) to augment our ontology. Specifically, NDC codes for specific drugs are placed as children of the RxNorm drug formulation they are associated with. The ontology for modifiers is a list directly derived from the modifier codes in the PDO. Moreover, the PDO modifier codes are themselves derived from the MDMI Referent Index names. Once the Referent Index is an approved standard, this provides one path toward canonical naming of i2b2 modifiers. Because such a vast amount of medication metadata is included in the dispensing data, for this initial version we prioritized the modifiers most important in obtaining accurate patient medication information: detailed information on the drug, what was dispensed, and refills remaining. Not included are many of the details on credentials, such as NCPDP number (national pharmacy identifier code). More detail on the modifier codes in the ontology is shown in Table 2.
Validation
We deployed i2b2 1.7.04 with our i2me2 cell and the RxConnect tools at Boston Children’s Hospital. The patient annotation app was deployed as a web application that can interact with the i2me2 cell. To validate the patient annotation process shown in Figure 1, we loaded synthetic test data in HL7v2 format into i2b2.
We manually augmented the loaded data with prescribing information provided by the i2b2 demo data and used the patient annotation app to view a patient’s combined medications. We also verified our ability to run patient count queries on the imported data in the i2b2 Query Tool.
Discussion
This represents a successful validation of an approach to integrate important non-traditional data sources to enhance medication lists for post-market drug surveillance. In this work, we described the successful end-to-end tests of a toolset that combines PBM data with i2b2 data, which is presented to patients for corrections via tablet-based patient annotation. This study also describes how complex medication information can be represented in three very different data formats.
The project team is presently recruiting patients for studies that examine the data quality of the medication list using this toolset. The current goal is to identify discordance between the clinical medication list and the patient-generated list (the output of the annotation app). Then, techniques will be developed to assess adherence from the discordance.
This is a generalizable approach to importing a variety of medication data into i2b2 via C-CDA messages. Our inclusion of PBM data enables a novel data source for patient-performed medication list annotation. Also, the enhanced MDMI toolkit and the maps developed for this project offer a starting point for data interoperability between i2b2 and a variety of standardized or custom data formats. The MDMI toolkit and maps for i2b2 and C-CDA will be released open-source shortly on the Open Health Tools website, and non-PBM-specific components of the i2me2 codebase will be released open source by the completion of the project.
Limitations and Future Directions
The patient-facing app functionality for loading reconciled medication data into i2b2 remains under active development. A standard ontology for patient reported medication information is being developed. The initial version of the app app uses i2b2 PDO format, but the final version will use the Fast Healthcare Interoperability Resources (FHIR) format, which is considerably more compact as well as more efficient to transmit and parse. Upon completion of full end-to-end testing using synthetic data, we will proceed to test our approaches using real-time data from consented subjects.
Perhaps the greatest technical limitation of this toolset at present is the reliance on PDO format for client-facing transactions. While PDO format is flexible, the inherent denormalization creates highly verbose output. Therefore it is not an efficient transmission format between the app and i2b2. In the future, we will be leveraging previous work in implementing SMART on i2b2, including addition of FHIR-based SMART messaging.
Conclusion
We have demonstrated the ability to incorporate a novel data source (pharmacy dispensing data) into the medication list of the i2b2 EHR data warehouse. The C-CDA approach is generalizable and uses standard data transmission format. This combined list is then presented to patients for annotation, adding a second novel data source (the patient). These tools will be released open source and they are currently being used at Boston Children’s Hospital to power studies on the concordance of medication lists across data sources. This work enables important research to improve medication list quality for drug surveillance and to improve patient drug adherence.
Table 1:
Mapped elements that translate to i2b2 modifiers and their HL7v2 location and i2b2 modifier code. The actual values are stored in appropriate columns in the i2b2 observation fact table. Shown here are codes for a dispensing event.
Name | HL7v2 | PDO modifier code |
---|---|---|
# of days filled | ORC/Duration | MED:DISPENSEDURATION |
Pharmacy Name | RXD/DispenseProviderFamilyName | MED:DISPENSEORG |
# of refills remaining | RXD/NumberOfRefillsRemaining | MED:REFILLSREM |
The type of pharmacy | RXD/DispenseProviderIdentifierTypeCode | MED:DISPENSETYPE |
Actual # of pills dispensed | RXD/ActualDispenseAmount | MED:QUANTITY |
Patient Instructions provided on label (the “sig”) | RXD/DispenseNotes | MED:INSTRUCTIONS |
Acknowledgments
Thanks go to: Lori Phillips on the i2b2 team for contributing her expertise in ontology design and development; to Florence Bourgeois in the Children’s Hospital Informatics Program for her leadership of study design and research applications of the i2me2 platform; and, to the MDMI team (Ken Lord, Sean Muir, Sally Conway, and Gabriel Oancea) who worked closely with us to make sure all needed functionality was available to us. This work was supported by NIGMS/NIH grant R01 GM104303.
References
- 1.Strom BL. How the US drug safety system should be changed. Jama. 2006;295:2072–5. doi: 10.1001/jama.295.17.2072. [DOI] [PubMed] [Google Scholar]
- 2.Pham JC, Girard T, Pronovost PJ. What to do With Healthcare Incident Reporting Systems. J Public Health Res. 2013;2 doi: 10.4081/jphr.2013.e27. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Brownstein JS, Sordo M, Kohane IS, Mandl KD. The tell-tale heart: population-based surveillance reveals an association of rofecoxib and celecoxib with myocardial infarction. PloS one. 2007;2:e840. doi: 10.1371/journal.pone.0000840. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Brownstein JS, Murphy SN, Goldfine AB, Grant RW, Sordo M, Gainer V, Colecchi JA, Dubey A, Nathan DM, Glaser JP, Kohane IS. Rapid Identification of Myocardial Infarction Risk Associated With Diabetes Medications Using Electronic Medical Records. Dia Care. 2010;33:526–31. doi: 10.2337/dc09-1506. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Bourgeois FC, Olson KL, Mandl KD. Patients treated at multiple acute health care facilities: quantifying information fragmentation. Archives of internal medicine. 2010;170:1989–95. doi: 10.1001/archinternmed.2010.439. [DOI] [PubMed] [Google Scholar]
- 6.Johnson KB, Unertl KM, Chen Q, Lorenzi NM, Nian H, Bailey J, Frisse M. Health information exchange usage in emergency departments and clinics: the who, what, and why. Journal of the American Medical Informatics Association. 2011;18:690–7. doi: 10.1136/amiajnl-2011-000308. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Finnell JT, Overhage JM, Grannis S. All Health Care is Not Local: An Evaluation of the Distribution of Emergency Department Care Delivered in Indiana. AMIA Annu Symp Proc. 2011;2011:409–16. [PMC free article] [PubMed] [Google Scholar]
- 8.Kaiser Family Foundation Follow The Pill: Understanding the U.S. Commercial Pharmaceutical Supply Chain. 2005. [accessed 4 Mar 2015]. http://kff.org/other/report/follow-the-pill-understanding-the-u-s/
- 9.Iglehart JK. Prioritizing comparative-effectiveness research—IOM recommendations. New England Journal of Medicine. 2009;361:325–8. doi: 10.1056/NEJMp0904133. [DOI] [PubMed] [Google Scholar]
- 10.Basch E. The Missing Voice of Patients in Drug-Safety Reporting. New England Journal of Medicine. 2010;362:865–9. doi: 10.1056/NEJMp0911494. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Kohane IS, Altman RB. Health-information altruists–a potentially critical resource. N Engl J Med. 2005;353:2074–7. doi: 10.1056/NEJMsb051220. [DOI] [PubMed] [Google Scholar]
- 12.Weingart SN, Gandhi TK, Seger AC, Seger DL, Borus J, Burdick E, Leape LL, Bates DW. Patient-reported medication symptoms in primary care. Archives of Internal Medicine. 2005;165:234–40. doi: 10.1001/archinte.165.2.234. [DOI] [PubMed] [Google Scholar]
- 13.Zulman DM, Nazi KM, Turvey CL, Wagner TH, Woods SS, An LC. Patient Interest in Sharing Personal Health Record InformationA Web-Based Survey. Ann Intern Med. 2011;155:805–10. doi: 10.7326/0003-4819-155-12-201112200-00002. [DOI] [PubMed] [Google Scholar]
- 14.Washington AE, Lipstein SH. The Patient-Centered Outcomes Research Institute — Promoting Better Information, Decisions, and Health. New England Journal of Medicine. 2011;365:e31. doi: 10.1056/NEJMp1109407. [DOI] [PubMed] [Google Scholar]
- 15.Klann JG, Mendis M, Phillips LC, Goodson AP, Rocha BH, Goldberg HS, Wattanasin N, Murphy SN. Taking advantage of continuity of care documents to populate a research repository. J Am Med Inform Assoc. 2014 doi: 10.1136/amiajnl-2014-003040. amiajnl – 2014–003040. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Health Level Seven (HL7) Implementation Guide for CDA® Release 2: IHE Health Story Consolidation, DSTU Release 1.1. Ann Arbor, MI: 2012 Jul. Report No.: CDAR2_IG_IHE_CONSOL_DSTU_R1.1_2012JUL. [Google Scholar]
- 17.Meaningful Use Stage 2 Rules Finalized – InformationWeek. [accessed 23 Aug2012];Informationweek. http://www.informationweek.com/healthcare/policy/meaningful-use-stage-2-rules-finalized/240006128. [Google Scholar]
- 18.Giovinazzo WA. Object-oriented Data Warehouse Design: Building a Star Schema. Upper Saddle River, NJ, USA: Prentice Hall PTR; 2000. [Google Scholar]
- 19.Klann JG. SCILHS PCORnet Common Data Model Ontology. i2b2 Wiki. [accessed 27 Feb2015]. https://community.i2b2.org/wiki/display/SCILHS/
- 20.Wattanasin N, Porter A, Ubaha S, Mendis M, Phillips L, Mandel J, Ramoni R, Mandl K, Kohane I, Murphy Shawn N. Apps to display patient data, making SMART available in the i2b2 platform; Proceedings of the AMIA Symposium; 2012. [PMC free article] [PubMed] [Google Scholar]
- 21.OHT/MDMI Project. SemantX, Inc. 2013. [accessed 20 Feb2014]. http://www.semantxinc.com/ohtmdmi-project.html.