Federated Aggregate Cohort Estimator (FACE): An easy to deploy, vendor neutral, multi-institutional cohort query architecture

Abstract

Cross-institutional data sharing for cohort discovery is critical to enabling future research. While particularly useful in rare diseases, the ability to target enrollment and to determine if an institution has a sufficient number of patients is valuable in all research, particularly in the initiation of projects and collaborations. An optimal technology solution would work with any source database with minimal resource investment for deployment and would meet all necessary security and confidentiality requirements of participating organizations. We describe a platform-neutral reference implementation to meet these requirements: the Federated Aggregate Cohort Estimator (FACE). FACE was developed and implemented through a collaboration of The University of Alabama at Birmingham (UAB), The Ohio State University (OSU), the University of Massachusetts Medical School (UMMS), and the Denver Health and Hospital Authority (DHHA) a clinical affiliate of the Colorado Clinical and Translational Sciences Institute. The reference implementation of FACE federated diverse SQL data sources and an i2b2 instance to estimate combined research subject availability from three institutions. It used easily-deployed virtual machines and addressed privacy and security concerns for data sharing.

1.0 Introduction

Cross-institutional data sharing is a fundamental component in the development and implementation of large-scale clinical studies. Whether within existing research consortia or for proposed collaboration, the identification across sites of patient populations suitable for participation in clinical and translational research is critical to decision-making. While there are many valuable applications of data sharing, cohort discovery (identifying research subject sample populations) is an important first step for many clinical and translational research initiatives. Identification of clinical study participants beyond a single institution is required both for rare disease research, where even the largest individual institutions are not always able to find a sufficient patient population, and for many studies where very large or diverse populations are needed.

Despite strong and ongoing efforts to develop data sharing mechanisms within research networks such as the CTSA consortium[1] by using a common technological architecture, e.g., i2b2 [2], the ability to include all sites in such consortia remains an unmet challenge. While some purpose-specific research networks rely on use of a common repository such as i2b2[3], have a common data model like the VDW[4], used by the HMO Research Network[5], or have a common vendor-supported data warehouse, these approaches fail to address an inherent limitation: no specific data warehousing technology infrastructure is common to all sites.

Data sharing strategies that rely on all sites having the same internal technical architecture will not succeed in involving all potential sites, especially sites outside the large academic medical centers. In addition, current data repository approaches typically require substantial infrastructure investment by each site, even with open-source software such as i2b2 or, more so, through commercial data warehousing solutions. In addition, some sites with fewer resources still do not have an enterprise data warehouse or have limited access to clinical repositories for research collaborations. As clinical and translational research expands to involve more multi-center collaborations and community healthcare sites, the issue of personnel and other resource limitations is a growing concern.

Despite efforts to address broad scale data sharing and interoperability, there are still significant challenges: (1) labor intensive deployment models, (2) sites using a variety of source data models, (3) varying data exchange security models and the ability to navigate multiple IRB and institutional security requirements, and (4) semantic interoperability requirements assuring that data from each of the sites are consistent and understood.

We addressed these problems by developing an innovative clinical research informatics approach to create a simple, platform-neutral cohort discovery tool that can be implemented by institutions with minimal technical expertise and resources. The approach, known as the Federated Aggregate Cohort Estimator (FACE)provides an easy way to facilitate cross-institutional data sharing and supports cohort discovery across the translational continuum.

2.0 Methods and Materials

2.1 Partnering Institutions

The FACE project involved collaboration among The University of Alabama at Birmingham (UAB), The Ohio State University (OSU), the Denver Health and Hospital Authority (DHHA), which is a clinical affiliate of the Colorado Clinical Translational Sciences Institute (CCTSI), and the University of Massachusetts Medical School (UMMS). UAB was the lead institution and was also responsible for developing the user interface and query controller. The grid infrastructure security was provided by OSU’s TRIAD project (authentication, authorization, etc.).[6], and UMMS and DHHA were the test sites. All sites contributed to the design of the user interface and the overall design of the system.

2.2 Architectural Model

The project implements an architectural model leveraging an existing grid-based infrastructure that supports federated queries to multiple data sources. It includes a central user interface and query controller, a Virtual Machine (VM) appliance deployment model, a common query expression structure, and a series of query translators.

2.2.1. Grid infrastructure

We utilized several elements from the TRIAD project (www.triadcommunity.org), which has implemented and extended a grid infrastructure service for translational research[6]. This service includes components to address security and data interoperability issues, data service representation and discovery, and communication across multiple institutions. OSU provided the TRIAD grid services infrastructure, and the central authentication service (Dorian Authentication and Grid Grouper Authorization components of TRIAD).

2.2.2. Central User Interface and Query Controller

A single user interface (UI) and supporting application was developed for use across all participating sites. This UI provides a query construction interface and access to aggregate results from that query from each participating institution. There is a secure, centrally-managed query controller which implements query distribution and management based on federated end-user authentication.

2.2.3 Virtual Machine (VM) Appliance Deployment Model

Each site received a standard virtual machine (also called virtual server or VM) which included all necessary components to accept requests for data, apply security rules as configured by local system administrators, map requests for data to the local data source, query the local data source, and return results. The VMs could easily be deployed within institutional firewalls on existing hardware, reducing the resource burden on participating sites.

A key component of the VMs is the common query processor, which allows an identical query to be sent to each site based on the central query construction interface. This query processor is the same for any deployment regardless of the underlying data source, and enables consistent communication between the central infrastructure and the various sites.

The VM queries for count data only, and only count information is sent to the secure central query controller and represented by the user interface. Therefore, protected health information (PHI) and other patient-level data in source systems are protected. While only aggregate data leave each site, obfuscation is still necessary in order to reduce the risk of re-identification.[7] FACE implements obfuscation both at the central query controller, and, optionally, local obfuscation settings can be supported at each institution.

Because of the federated nature of the architecture, participating sites maintain control of their own data. End user credentials are passed through to the local data services and local administrators retain full ability to control which users and/or user groups are able to query data from their local data source.

2.3.4 Common Query Expression Structure

The VDW data model and content standards were used for the common federated queries. The HMO Research Network VDW data structure specification defines content, format and valid values in 13 domains including demographics, encounter, diagnosis, and lab results. Valid value definitions adhere to standard code sets such as ICD-9, and NIH designations of race, ethnicity and gender. Additionally the VDW specification includes a table structure with fields, field types, and structures. While this data structure was used in the project as one of the target source data systems, it was also used as the content and structure for the construction of the common query definition. This choice was because VDW is a reasonably simple data model, with effectively a single table for each of the most commonly used elements for a cohort discovery solution (diagnosis, demographics, laboratory test results, vital signs, procedures), and a content specification that is oriented towards frequently used standards such as ICD-9, . As such it accommodates the types of content and queries expected for cohort discovery, and is already well specified and documented.[4] The VDW specification contained the required data and was simple to implement. While our current use is limited to diagnosis and demographics, extension to laboratory test result, medications, procedures, etc. are accommodated by the specification.

If cohort discovery query parameter requirements were to expand beyond what is represented in the VDW specification, then either the specification would need to be expanded, or a supplemental specification could be implemented for specific subdomains. The intent of this project is to consider a broadly applicable common set of data commonly used for cohort discovery. Any self-service cohort discovery method should be kept at a simple enough level to be easily navigable. These two factors have guided what content to prioritize within the highly successful collaborative approach of the HMO Research network and VDW enhancement over time.

The query construction interface provides the opportunity to use drop down lists as well as Boolean operators to select the query parameters. Users are able to constrain the query based on current age, gender, race and ethnicity. See. figure 1. Diagnoses can be searched, with auto complete based on either diagnosis code or description. Diagnosis constraints can be defined as either “all of the following” or “at least one of the following”. Results are returned in aggregate at the institution level, in order to gain a quick indication of organizations of interest. See figure 2. Organization’s results can then be either excluded or included in the representation of the stratified aggregate results. Users can dynamically drill down by applying a stratification based on age, race, gender, or any combination thereof.

Figure 1.

FACE Query construction

Figure 2.

FACE Results

The query translators hosted on the VMs translate from the VDW-specified model to the model and terminology of the local data store.

2.3.5 Query Translators

For our reference implementation, we developed query translators for the i2b2 Hive and the VDW SQL schema. For i2b2, we executed queries via i2b2’s RESTful web service API provided by the CRC Cell, and, as such, the FACE queries are visible within the standard i2b2 interface and appear like i2b2 queries. The queries are executed with i2b2 application credentials rather than direct database credentials. For VDW, which includes no such API, queries are executed directly against the source database. The VDW translator maps the query structure to the native VDW schema using hibernate over a JDBC connection. Non-VDW schemas are supported using database views to map the required attributes to the VDW specification (the approach implemented at UAB).

The query content, such as the diagnosis, is expected to meet the content standards of the specification. In VDW, the physical structure is known and the query translator is aware of the logical data model to which the query structure is translated. For i2b2, while the content standard is expected, and the REST web service limits the need to define physical structure, there is an additional configuration step. As an example, while the ICD-9 diagnosis may be used across multiple i2b2 instances, it is possible that the way in which the concepts are navigated might be different. Each i2b2 implementation may have a different hierarchy, or navigation path to their ICD-9 codes. These navigation paths are defined and manifested as a set of concept paths in an ontology within i2b2. To make the FACE query translator aware of the specific concept paths within an institution’s i2b2 instance there are two choices. If the core hierarchy is the same, the standard configuration file can be updated to indicate the root nodes that contain the concept. If a custom hierarchy or ontology is used to house the ICD-9 concepts, a script is run to consume those concept paths. The script extracts the local i2b2 ontology and makes it available to the data service. The query translator uses the resulting configuration file to insert the appropriate concept path for a given ICD-9 code as it constructs the messages to send to i2b2. Construction of a new query translator, to a new underlying data structure would be a moderate development effort, but would then be usable by any data source of the same type. Additionally not all variations in an underlying data source would result in the construction of a new query translator. Rather, creation of a database view to represent the data as expected by an existing query translator would sometimes be a less resource-intensive approach. This approach was used very effectively for a second instance of UAB’s data as part of the proof of concept and is a much simpler approach than constructing a new translator. This approach proved to require only limited resources to construct the view based on the VDW specification, which is purposefully simplified.

2.3.6 Research Approval

The cohort discovery tool project was designated by the participating Institutional Review Boards as “Not engaged in human subjects research”. As appropriate, the architecture was also reviewed by healthcare institutions’ data security process. Attributes of the architecture that are beneficial for mitigating security concerns are described in the discussion.

The project components were deployed and instantiated at each site, but were only accessible to the project team as a proof-of-concept implementation. Though the infrastructure was put into place, with a live user interface and queries to institutional data sources, the system was not deployed as a production system and was not made accessible to those outside the project team.

3.0 Results

Figure 1 below illustrates how FACE was deployed at the participating sites.

The FACE VMs were deployed at three sites (1) UMMS, which had a de-identified i2b2 data repository, (2) DHHA which had a de-identified warehouse using the VDW data model, and (3) UAB who operationalized two separate data sources: one with an identified instance of i2b2 (not shown in Figure 1), and one using internal data sources separate from either i2b2 or our enterprise data warehouse. The purpose of UAB’s implementation was to approximate use at an institution that does not have a data warehouse.

The FACE architecture developed is a secure infrastructure that includes two central components as well as lightweight components at each participating site. The central infrastructure includes a collaboratively-developed query construction UI and a query controller application. The user identifies what patient characteristics are of interest and the user interface submits the query to the central query controller application. The controller issues a query to each of the participating sites, retrieves results, and presents them to the user, while handling authentication, obfuscation, and other administrative functions. The FACE data service VM for site deployment contains all of the grid components to instantiate a secure grid service and communicate with the central architecture, including GUARDS/Globus authentication, and the Introduce toolkit.

Implementation of the VM at each site was accomplished by downloading the self-contained VM image file, loading that file into a virtual machine host, and entering local settings into configuration files on the VM. VM file deployment includes a virtual server within the existing VM infrastructure. The files are created such that they provide the physical server configuration parameters necessary to deploy easily. When “physically” deployed, the VM is assigned an IP address and firewall and port security settings are configured to allow connectivity of that VM to the underlying institutional data source. The VM typically requires this step and thus it is a standard deployment process. With file download, import into the virtualization environment (VMWare) and deployment and port configuration were completed at UAB, normally within a few hours of request. The VMs were deployed without error by the server or virtualization staff within each institution. Once the VM was deployed, the institution configured the FACE software. The image contained in one place all of the necessary software components and configuration files. The configuration parameters include the connection information for the source database, such as address, port, username/password. The VM is largely pre-configured to connect to the federated query processor and authentication services and requires very little time to add the local data source specifics. For the two sites with direct database implementations (UAB and DHHA), the VM included components for query translation to the SQL implementation of the VDW schema. For the sites in which the target data source was i2b2 (UMMS and one of UAB’s repositories), the VM included the query translator that performed RESTful web service calls to the i2b2 CRC Cell and a results aggregator that received the standard web service responses from i2b2.

The data included in the reference implementation were demographics (Age, Gender, Ethnicity, Race) and Diagnosis as represented by ICD-9 codes[8]. An example query within the reference implementation was to identify the number of patients with a diagnosis of Spinal Muscular Atrophy (ICD-9 code 335.10). As shown in Figure 2, results were returned from the four data sources . [UMMS (i2b2) – 110 patients, DHHA (VDW) −2 patients; UAB (i2b2) 101 patient, UAB (custom db) 106 patients]. The difference in the two UAB data sources is due the use of an obfuscation algorithm. Results from each data source were dynamically included or excluded from the displayed results, and results were also dynamically stratified by any combination of gender, age and race. The patient sets queried included 1.3 million patients at UAB, 1000 patients at DHHA and 3.4 million patients at UMMS.

4.0 Discussion

We developed a federated cohort capability, FACE, focused on demographics and coded data in the form of ICD-9 diagnoses [8]. FACE includes a well-documented technical architecture, appropriate separation of components, component characteristics, and communication protocols, which allows for efficient review by IRBs and institutional security teams with little custom configuration. A key architectural component advantageous during security review is that FACE uses a VM constructed to interact with a data source that remains in its original location rather than copying information to the VM. This is advantageous because the VM can be placed in a DMZ or otherwise separated from the protected health information (PHI) inside the institution’s firewall. PHI is not at rest in any place other than its original source system, introducing less risk. Only aggregate data are sent outside the institution, meaning that data are de-identified, and no patient level data are exposed. This generally considered low risk during security and compliance review. The TRIAD technology is a well-known and standard authentication and authorization model that was readily accepted by data security and compliance officers. This approach also provides an advantageous architecture for further functionality. FACE was developed with an emphasis on using commonly deployed data models and promoting reuse of components to enable ease of expansion and extension beyond the reference implementation. This approach makes it relatively easy for new sites to participate.

The platform allows the use of any underlying data source repository, and can allow collaboration across institutions regardless of their existing data warehousing infrastructure. The “platform neutral” architecture works with a variety of architectures because each site represents the mapping of its existing data via query translators, rather than all parties needing to store their data in an identical way. Implementing the standard deployment components with a specific, lightweight, query translator reduces the resource burden on participating sites, as compared to a full implementation of a common data warehouse.

The query controller and application can easily be configured to recognize and represent new sites that implement a FACE VM. This reduces the burden of implementation for a collaboration or consortium since there is little or no central work associated with each additional site. This architecture also means that addition of sites requires configuration changes only at the central application, and not at all existing participating sites.

Allowing sites to leverage the existing TRIAD infrastructure[6] is a major difference in our approach from others used for cohort discovery and helps to achieve the goal of easy deployment. The grid approach used in FACE comes with the added benefit of having inherent security capabilities that preclude the need for ad-hoc or site-specific security configurations and point-to-point firewall exceptions. TRIAD’s Grid Authentication and Authorization with Reliably Distributed Services (GAARDS) Framework provides services and tools for the administration and enforcement of security policy in an enterprise grid. GAARDS was developed using the Globus Toolkit security framework. GAARDS and Globus are well-tested platforms with well-structured authentication and authorization systems that also support federated user credentials from home institutions.[9] The grid data service artifacts automatically incorporate the security layer, representing the content of the service and how to use it. In addition to technical security, the overall architecture eases security approval as well.

The query translation layer is reusable with other i2b2 or VDW instances. Additional i2b2 or VDW sites can reuse the existing FACE VM image, modifying only the configuration and concept mapping settings. Both the i2b2 and VDW data models were easy to map to existing architectures at all three sites. Because any individual implementation of i2b2 can result in varying ontology hierarchies, a mapping process is required. Querying i2b2 using the standard data service and standard query structure only required loading a concept path properties file that added an additional ontology. Scripts were created to automate most of the work involved in creating the needed local mapping file for a given i2b2 instance. This did not require change to any existing ontologies, merely inclusion of a consistent ontology to represent the existing diagnosis concepts. Because the VDW model strictly prescribed the ICD-9 content and schema structure, no mapping was necessary for VDW implementations. While the reference implementation includes query translators for only two underlying sources, additional data sources types and schemas (RDF, other data warehouse structures, etc.) can be incorporated into the network by adding additional query translators.

Other data in addition to demographics and diagnoses can be accommodated to the extent that they can be represented in a standard way (such as ICD-9 Procedures[8], CPT procedures[10], LOINC[11], NDC[12], RxNorm[13], ICD-10[8]). With the federal initiatives related to standards for meaningful use of electronic health records and information exchange[14] and the President’s Council of Advisors on Science and Technology (PCAST) report[15] recommending a universal exchange language, it is likely in the future that data types other than diagnoses will be available for mapping to the existing infrastructure and these could be incorporated into FACE. This first version, supporting only diagnosis and limited demographics, is useful in and of itself for identifying cohorts for rare diseases that have specific ICD-9 codes because it makes it feasible for multiple sites to participate. The output of FACE allows potential investigators to determine feasibility as well as to target specific collaboration opportunities. While the rare disease use case is important for a simple cohort estimator, the tool has value as well with larger populations or more prevalent diseases, especially when specific inclusion and/or exclusion criteria exist.

Our decision to use the existing VDW published specification as the framework for query interactions not only contributed to ease of deployment, but also is a benefit for reuse of the components for future development. While our reference implementation communicated only with a single central query tool, the model supports additional functionality. For instance, a single VM/data service can join multiple networks and serve as an institution’s delivery model for multiple federated query projects. This means that a new user interface and query controller could be developed for a specific project, and because the query and content structure is known, the local data service can be configured to serve both networks without the site deploying a new VM or any duplicate implementation of additional data sources.

Using an API instead of direct database queries, can preserve the security, logging, and performance enhancement components of the source infrastructure. The use of the i2b2 RESTful web service preserved these and other functions native to i2b2. Unlike FACE’s use of the i2b2 web services, querying the database directly would bypass the intended security and compliance functions within i2b2.

However, utilizing the i2b2 RESTful web interfaces to execute expressive queries and retrieve data required addressing several challenges. The i2b2 RESTful interfaces were designed and intended for use only within the i2b2 system itself, so they were not clearly defined or documented. Specific message sequences were also not clearly defined with an eye towards use by an external program.

To develop the interface, we identified, extracted, enhanced and packaged a small subset of the i2b2 code base that encapsulated querying the CRC cell via its RESTful interface. If web services are available for other source systems, they can be utilized in a similar way. Otherwise, more direct database connections can be used as was done with our VDW data sources.

4.1 Comparison with Other Federated Query Tools

PopMedNet is a federated query tool that is neutral with regard to data model and, like FACE, currently has plug-ins to connect to both the VDW data model and i2b2. PMN architecture features the ability for data stewards at each site participating in a federated query to manually inspect each incoming query and block both the query execution and sharing of results. This feature has proven critical to garnering participation in some data sharing networks such as HMORN, but it is not ideal for automated cohort discovery because it introduces a delay in receiving results. Based on our experience with i2b2 and other cohort discovery interfaces, cohort discovery requires a series of increasingly refined queries executed in quick succession to hone in on the group of interest, a use case better addressed by FACE than by PopMedNet Use of PMN with i2b2 requires the installation of two additional i2b2 cells into the hive of each i2b2 instance wanting to use PopMedNet. FACE uses only the existing restful web service thereby eliminating modification of each institutions instance of i2b2. With PopMedNet the i2b2 web client is required to be used and with SHRINE a modified version of the i2b2 Web Client is needed to construct the query. FACE provides the ability to query with an alternate user interface, and other query construction interfaces could be developed, just as additional data sources could be added. Ease of deployment is a key consideration as well. It is likely that the resources required to implement an instance of i2b2 would be more than developing a query translator, and certainly more than implementing the FACE VM and use of an existing translator either natively with a data source, or via a database view on that data source. Because SHRINE uses i2b2 and many CTSA sites already have i2b2 SHRINE could potentially be used in all those sites, which would provide a large, although not comprehensive, set of institutions for collaboration. PopMedNet is able to address the issue of local, site-specific concerns about dissemination of information. While SHRINE and PopMedNet address some considerations in multi-institutional cohort discovery, the FACE architecture introduces some flexibility and lessened requirement on specific infrastructure.

4.1.2 Limitations and Challenges

The current reference implementation has some limitations. The deployment of a VM assumes that the organization has a virtualization environment of some sort. The use of virtualization is extremely common in healthcare delivery organizations and academic research centers. If no such environment is in place there are a number of free or low cost virtualization solutions that can be used either with existing or new hardware. While the FACE architecture is designed to accommodate multiple patient characteristics, we have so far only included demographics and diagnoses and only three institutions have so far participated. Also, we only used a limited number of data sources. Though we have used these particular systems/configurations for the reference implementation, the design can accommodate other underlying sources. The majority of the components of a site implementation can remain the same, with additional query translators built for other types of data sources. For example, if a site has an underlying data source in RDF, or in a commercial EHR vendor’s data warehouse, a query translator can be built to translate the common queries to the appropriate query structure. The query translator is sufficiently self-contained that it can be replaced within the deployment model without any negative impact on the rest of the infrastructure. Subsequent implementation of the same underlying structure or commercial product can then reuse that particular query translator, just as any i2b2 site can reuse the one that has already been built.

An additional limitation is that FACE is specifically targeted towards providing controlled, but real-time, access to data across institutions. For this reason, per-query manual review prior to execution at each individual site is currently unavailable. The expectation is that once a data service is configured to provide data, either at a network, institution, or user level, the service will provide that data. If a per-query verification or approval is necessary, additional functionality would need to be incorporated, but as we discussed above, such functionality would likely add delay to obtaining results.

In addition to the challenge of using the i2b2 RESTful web interfaces, we faced other challenges that are not specific to the FACE architecture, but are common to all multi-institutional collaborations. FACE does not strictly address differences in coding schemes between institutions. Though this has been a long-standing issue, the standard vocabularies are being used with more prevalence in clinical care as regulatory, and certification requirements emerge. Meaningful use,[14] EHR certification[16], and a general push for consistency in documentation are moving the industry towards the use of standards such as ICD-9[8], SNOMED[17], LOINC[11], RXNorm[13], etc. These standards represent the bulk of the content for the cohort discovery use case, and as institutions move towards their use, the ability to incorporate their data into a federated query model will become more straightforward.

5.0 Conclusion

FACE is a reference implementation that demonstrates cohort query across multiple institutions using different data warehousing platforms. It is a scalable, easy to deploy solution that incorporates architectural components to meet data security and IRB requirements for privacy and security. Our approach reduces burdens of implementation and concerns about security, increasing the likelihood of successful multi-institutional collaborations. Deployment of the VMs at the partner institutions was quite simple. With the reference implementation in place we can now easily extend our work to handle a greater number of clinical data types (procedures, labs, etc.), add new institutions, and add support for authentication with home institution credentials.

Figure 3.

FACE Architecture

Highlights.

• FACE is a federated, multi-platform approach for clinical study cohort estimation

• Data is aggregated from both i2b2 and Virtual Data Warehouse (VDW) data models

• The architecture helps meet IRB and security requirements easily with local control

• RESTful web services preserve security, logging, and features for i2b2 sources

• The virtual machine delivery model reduces the burden at each data sharing site