Establishing a research informatics program in a public healthcare system: a case report with model documents

Abstract

While much is known about governance models for research informatics programs in academic medical centers and similarly situated cancer centers, community and public health systems have been less well-characterized. As part of implementing an enterprise research governance framework, leaders in the Los Angeles County Department of Health Services established a research informatics program, including research data warehousing. The strategy is focused on high-priority, patient-centered research that leverages the investment in health IT and an efficient, sustained contribution from 2 affiliated Clinical Translational Sciences Institutes. This case study describes the foundational governance framework and policies that were developed. We share the results of several years of planning, implementation, and operations of an academically funded research informatics service core embedded in a large, multicenter county health system. We include herein a Supplementary Appendix of governance documents that may serve as pragmatic models for similar initiatives.

INTRODUCTION

Research operations in cancer centers and academic medical centers (AMCs), including research informatics services, are typically governed and resourced by AMC research missions.^1,2 In contrast, in public health systems and other community hospital settings, research may be perceived as conflicting with patient interests, institutional goals, and the need to align resources with patient care.^3,4 While many non-University health systems have affiliated nonprofit foundations devoted to research administration and community benefit tax exemptions, affiliates are often based on trial-focused models that predate contemporary standards for centralized health and research IT and attendant data-driven research practices.⁵ The path to mature research data warehousing varies across systems, and the required expertise and priorities may differ between AMCs and non-University settings.

Setting

Los Angeles County Department of Health Services (LACDHS), the second-largest municipal health system in the United States, provides care to over 600 000 unique patients per year including 380 000 empaneled individuals. The system includes 4 teaching hospitals and a network of 27 health centers, with a mixture of employed and contracted physician staff, most with academic appointments in 1 of 3 local Medical Schools, 2 of which have Clinical Translational Sciences Institutes (CTSIs). The California Medicaid Waiver Program introduced multiple requirements for reporting and quality improvements, motivating investment in IT and staffing. Each of the 4 medical centers has independent research foundations and local Institutional Review Boards (IRBs); research in the LACDHS Ambulatory Care Network (ACN) is governed by a single IRB also responsible for research in the LAC Department of Public Health. The mission of LACDHS is to advance the health of our patients and our communities by providing extraordinary care.⁶

Prior to 2015, research operations were not tightly overseen by the LACDHS enterprise. Often research was conducted on an ad hoc basis governed by each of the local IRBs, with unclear authority and procedures for resourcing space, personnel, and other assets. Research data provisioning was served locally by both research and IT staff, often operating on an opaque, influence-based model rather than a transparent, policy-based model. Thus, when the LACDHS data warehousing and business intelligence programs were implemented, enterprise-wide research informatics functions were sparse. Nonetheless, LACDHS has historically been the setting for multiple research projects (cf.^7–11).

With the implementation of an enterprise electronic health record (EHR) and Data Warehousing program, demand for data for clinical and research purposes increased substantially. Before the EHR implementation, data from lightweight practice management systems were used to meet reporting and documentation requirements. Procedure-based billing codes applied to fewer than 10% of patients, distinguishing LACDHS from systems that primarily serve fee-for-service payers. Enterprise-wide data analyses were provided by the Office of Planning and Data Analytics (OPA). The county also invested in enterprise Data Warehousing and Business Intelligence Programs, as well as a Population Health platform. Currently, researchers and staff have access to a vendor-supported cohort discovery tool (i2b2) and may request access to hundreds of existing IT reports curated in a metadata library. Unlike many AMCs, records for the LACDHS managed care population includes longitudinal records from outside health systems that can also be linked with other public services,¹² making LACDHS uniquely well-suited to safety-net outcomes and effectiveness research.

CTSI partnerships

The Los Angeles area has 2 CTSIs affiliated with USC and UCLA. The LACDHS Chief Deputy Director for Clinical Affairs (DDCA) along with UCLA and USC CTSA Principal Investigators established 2 LACDHS-embedded cores, one for Informatics and Analytics (led by USC), and the other for Innovation and Implementation (led by UCLA).¹³ While most CTSI funding comes from the NCATS Clinical Translational Sciences Awards, CTSI Informatics Programs are often also the functional home of University-funded informatics services, as is the case at USC.¹ This collaborative strategy included resource commitment from USC CTSI for embedded LACDHS informatics staff. Figure 1 shows how the 2 CTSIs contribute to the LACDHS Informatics Core. The Los Angeles CTSIs have increasingly supported “T4” research devoted to population and community health.¹⁴ This commitment culminated not only in sustaining the research data warehousing program but also a pilot grant for safety-net delivery system science, and limited pro-bono services for USC CTSI researchers.^15–17

Figure 1.

Organization and Resourcing for the LACDHS Informatics Program and Core Facility. LACDHS Research Informatics Program operations and services are overseen and supported by the ROB and LACDHS leadership and staff (bolded outlines) and academic partners (thin outlines). The ROB, chaired by the DDCA, consists of an IRB representative from each of the directly affiliated medical centers, CMOs or delegates responsible for rating alignment of research projects with LACDHS priorities, and a representative for LACDHS Enterprise Data Governance. The faculty director for the Informatics Core is a member of the ROB. Staffing includes a service navigator and 3 “seats” for academically funded analysts embedded in the larger LACDHS analysis workforce that serve all enterprise research data requests. Staff are funded primarily by institutional contributions from the USC and UCLA CTSIs (red and yellow fills, respectively). Currently, commitments from academically funded staff range from 10% to 75%. Fee-for-service contracts supplement CTSI funding on a project-by-project basis (red stripes). Clients include researchers from the academic centers directly affiliated with LACDHS and their collaborators from external organizations. This structure allows for flexibility and diversity of expertise to meet the dynamic demands of researchers. CTSI: Clinical Translational Sciences Institute; CMO: Chief Medical Officer; LACDHS: Los Angeles County Department of Health Services; ROB: Research Oversight Board.

The Research Oversight Board

In 2015–2016, enterprise-wide research planning began with the establishment of a Research Oversight Board (ROB) composed of Chief Medical Officer (CMO) delegates from the ACN and each of the medical centers, the OPA Director, and CTSI representatives (see Supplementary Appendix for rosters). Members of the ROB are mostly clinicians with research experience and histories of NIH funding. The ROB determined the types of research LACDHS would prioritize for the use of LACDHS resources. To manage the plan’s development and execution, an LACDHS role was established for a Research Program Director responsible for developing, synthesizing, disseminating, and operationalizing the new policies.

The resulting foundational document guiding LACDHS research is the ROB Framework (Supplementary Appendix S1). LACDHS leadership developed guiding principles summarized in Table 1, notably including the goal to enhance data for research within LACDHS. On a case-by-case basis, the facility CMOs and/or delegates rank each research project along a Five-Tier system indicating level of LACDHS resources and support for research projects (Table 2). The new system created a more egalitarian process, including rare occasions where raters assess projects sufficiently aligned with LACDHS objectives to be eligible for use of LACDHS resources, such as space or staff, as in Categories 1 and 2 (see Figure 2). More frequently, the projects are rated in a way that requires research teams to fully support all aspects of the project, including data requirements, with academic funds. Formation of the Informatics Core Facility funded by academic partners was a practical solution for projects with informatics needs that were not rated for use of LACDHS resources (like IT staff). However, this transition was not without friction, as a transparent process shifted decision-making roles. Given the centrality of data access in all phases of research, the new Research Informatics Program was, in practice, where the earliest impact of the new policies was experienced and tested.

Table 1.

Guiding Principles of the LACDHS Research Oversight Board (complete document Supplementary Appendix S1)

Support meaningful, proportionate, and impactful healthcare research that aims to achieve the LACDHS mission	Streamline and facilitate research approvals	Enhance accuracy, availability, and management of data for research	Set out a framework to guide oversight and the decision-making process	Promote institutional capacity and external collaboration for research
That is, support effectiveness research, which directly impacts the LACDHS patient population That is, support efficiencies that align project review, approval, and oversight processes	Streamline the IRB process with other regulatory and LACDHS oversight approvals Harmonize IRB and compliance processes across the network of Los Angeles County institutions	Subsume the rules for access and use of research data under the LACDHS Business Intelligence governance model	A guidance for resource and research planning (refer to Table 2) Discussions the framework will generate are understood to be complex and the Research Oversight Board is empowered to establish precedence	Facilitate changes to LACDHS operational practices from within the office of the Chief Medical Officer and from the Research Oversight Board Facilitate a meaningful and effective mentorship model for opportunity creation Facilitate partnerships with regional and national networks

IRB: Institutional Review Board.

Table 2.

Framework for research project categorization and LACDHS resource support

	LACDHS consumables and supplies	LACDHS staff time and effort, including administrative, IT back-end system access and clinical staff	Fixed cost LACDHS resources, for example, space, IT front-end systems, utilities, etc.	Access to Patients for study recruitment (study performed at another institution)
Category 1: Research performed at LACDHS sites and well aligned with LACDHS business needs and strategic priorities. It is anticipated that the results of the research will have a significant direct impact on clinical operations for a significant number of LACDHS patients. Direct LACDHS resources can be used to fully support projects in this category; however, researchers should make every effort to obtain extramural funding.	LACDHS supports fully or partially	LACDHS supports fully or partially	LACDHS supports fully or partially	LACDHS supports fully or partially
Category 2: Research performed at LACDHS sites and well aligned with LACDHS business needs and strategic priorities. The research results are expected to affect care of the population served by LACDHS. Some external funding support is required to accomplish the research objectives	LACDHS supports partially or none	LACDHS supports partially or none	LACDHS supports partially or none	LACDHS supports partially or none for group recruitment. Individual patient referral to an appropriate study is always allowed
Category 3: Research performed at LACDHS sites with potential benefit to LACDHS patients in the future. LACDHS will allow access to resources, but the study must cover the cost of these resources.	LACDHS supports Study must fully fund at least marginal cost	LACDHS supports Study must fully fund at least marginal cost	LACDHS supports Study must fully fund at least marginal cost	LACDHS allows access, but study must cover cost of any negative operational impact of recruitment Individual patient referral to an appropriate study is always allowed
Category 4: Research intended to support the development of new drugs, medical devices, or procedures that, although having the potential of improving the care of patients in the future, does not address a key component of LACDHS’s service priorities or needs that disproportionately affect LACDHS’s patient population. These projects have potential positive impact for the LACDHS population (eg, drug discovery)	Full compensation of at least marginal cost	Full compensation of at least marginal cost	Full compensation of at least marginal cost	LACDHS allows access, but study must cover cost of any negative operational impact of recruitment Individual patient referral to an appropriate study is always allowed
Category 5: Research that has no benefit to LACDHS or its partners and should not be conducted on a LACDHS campus	No	No	No	No

Figure 2.

ROB ranked projects from Highly Aligned (1) to Not Aligned (5). The ranking policy was fully adopted in 2016. Few projects were evaluated as Fully Aligned or Not Aligned. The median score shifted from 4 to 3, particularly after an inter-rater reliability exercise and additional guidance to raters were conducted in 2018 to ensure raters were using similar criteria. Restrictions were placed on projects not deemed to be essential to the COVID-19 Pandemic in Q2 of 2020, a policy that has recently been relaxed. ROB: Research Oversight Board.

FINDINGS AND LESSONS LEARNED

Leadership

Demonstrably, the most significant factor contributing to the success of the Informatics Program has been consistent and assertive policymaking from DDCA and the multistakeholder ROB representing all 4 medical centers and the health center network (Supplementary Appendix S5). The ROB is chaired by the DDCA f and managed by the Research Program Director. As a byproduct of CMO oversight, new research policies and strategic directions are developed and championed by operational leaders with direct feedback to researchers. Cohesive operations would not be feasible without this collaboration, particularly while transitioning to a standardized, transparent research process.

An embedded informatics team

Executive leadership required a straightforward organizational structure for the Research Informatics Program to implement an Informatics Core Facility: a team of CTSI-funded staff to embed within the LACDHS Planning and Data Analytics unit. This includes virtual and physical seats on premises at LACDHS. All research data requests are routed by the helpdesk to the LACDHS Research Program Director before assignment to analysts to ensure ROB resourcing Categorization and IRB reviews are complete. The Informatics and Analytics Core has grown from 50% coverage of a single research analyst to include 3 part-time analysts, a data manager, a navigator, and clinician informaticists supported by a combination of fee-for-service and sustained CTSI investments. Informatics Core staff are onboarded and trained with “dotted line” reporting to the LACDHS Data Governance leadership (this role included the LACDHS OPA Director and Population Health Senior Managers as the LACDHS organization has evolved). Training has often included assisting on research and nonresearch projects working alongside LACDHS staff. After completing training, research analysts attend regular internal meetings of LACDHS health system analysts allowing for peer-to-peer communication and visibility into new developments and change control practices that impact data sourcing, quality, and availability. Division of effort between an informatics team based at Harbor-UCLA and LAC+USC medical centers was developed to incorporate assignments predicated on a combination of methodological expertise and investigator’s academic affiliation; services are not restricted to LAC+USC or Harbor-UCLA-based investigators. The USC and UCLA CTSI funding enables pro-bono informatics consultations to navigate (and potentially make small modifications to) existing self-service tools and assets, including the report library and the i2b2 cohort discovery tool.¹⁸ The sum of these efforts and opportunities strengthened the academic partnership with LACDHS toward research informatics goals.

Regulatory compliance, privacy protection, IRB approval, and ceding

LACDHS-affiliated researchers and staff may have primary appointments in LACDHS medical centers, colocated research foundations (eg, The Lundquist Institute), or academic organizations (eg, Charles Drew University), resulting in a complex network of 8 IRBs. An IRB ceding agreement was executed in 2016 and revised in 2018 (Supplementary Appendix S3), allowing for mutual reliance among LACDHS hospital and academic affiliate IRBs, accelerating IRB review. As with any research data warehousing program, IRB approval is paramount to compliance with the Common Rule and Health Insurance Portability and Accountabilty Act (HIPAA). The LADCHS Notice of Privacy Practices was carefully crafted with the safety-net patient population and mission in mind.¹⁹ Some additional harmonization was required, including the Lanterman-Petris-Short (LPS) Act and the Confidentiality of Medical Information Act.^20,21 To accommodate state regulations like LPS, standard operating procedures for data provisioning required the creation of data filters, IRB forms, and research analyst checkpoints related not only to specific confidential data elements, Protected Health Information, or Personally Identifiable Information but also policy-based filters for specific types of providers and facilities where services are rendered that do not apply to private systems. Similarly, LACDHS helpdesk forms for data requests include structured data elements that capture components related to research (excerpted in Figure 3; Full Document in Supplementary Appendix S4). In this way, technology has evolved to support the policy framework. Additionally, procedures were initially unclear with respect to how the 4 hospitals’ disparate patient contact practices would operate. This ultimately resulted in a uniform and carefully researched Guidance Statement which, after extensive deliberation, was published in 2021 (Supplementary Appendix S2). This ensured an enterprise-wide informatics program informed by regulatory and ethics requirements.

Figure 3.

ROB Changes to Data Request Form Snapshot (full document in Supplementary Appendix). ROB: Research Oversight Board.

Research products

A number of projects taking advantage of the Informatics Core have produced successful grant applications and results in program evaluation, quality improvement, and clinical research. Examples include evaluation of local and state policy as well as institutional quality improvement initiatives.^22–27 Some of the research products have been incorporated into national public health toolkits.²⁸ These studies were frequently ranked in high alignment with the LACDHS mission. This is not to the exclusion of clinical research, in which the Informatics Core and collaborating data staff may facilitate full-service or self-service study planning or so-called “big data” extracts (eg,^29–32). While LACDHS continues to conduct clinical trials, the ROB framework may have resulted in shifting the balance of research products, particularly those supported by the Informatics Core, toward high-impact products that influence policy and practice.

Challenges

A few key items may inform similar initiatives. We found that while the rating system was a crucial component, the federated nature of LACDHS showed that the system leaders’ judgments were not necessarily consistent across the enterprise. To address this, we conducted an informal inter-rater-reliability exercise collectively rating 3 examples in an ROB meeting, allowing for internal discussion and updates to documented criteria for each category. Since these updates, rating consistency has not been flagged as a concern, even as new raters have been assigned. The queue time for the rating process has not been centrally tracked, as each rating organization maintains its own process. For example, LAC+USC ratings are assigned before IRB review is complete, but there have been examples of complex, enterprise-wide projects with external collaborators and unclear affiliation assignments that experience delays of several weeks. Similarly, each of the different local IRBs had varying policies. For example, some developed detailed checklists for Quality Improvement IRB Exemptions to guide decisions, with others following a more qualitative approach. Each medical center acted as a “laboratory” in which new procedures and policies might be tested and ultimately disseminated. A risk to be managed in the current paradigm is dependency on LACDHS leaders that are also clinicians and researchers themselves, ably stepping into business ownership roles negotiating with vendors and other stakeholders in a way that might be filled by Chief Research Informatics Officer in other organizations. These roles would need to be filled should there be changes in the characteristics of future leaders. Finally, the demand from the LACDHS research community, even for well-aligned projects, is greater than what can be served in a timely manner by currently available staff. Some researchers were unaccustomed to the sustainability model required by the new approach, a challenge shared with other AMCs. The result has been amending our intake consultations for new clients with an introduction by the LACDHS Research Program Director explaining the policy on behalf of DDCA.

CONCLUSIONS

In summary, the key contributor to the success of the program has been the combination of health system leadership that demanded and supported a truly embedded research informatics service core focused on improving healthcare delivery and clinical effectiveness, and academic partners with expertise in informatics enthused about subordinating conventional academic priorities to the needs of the public health system. In particular, synergy assessments are directly incorporated into the process of project initiation and approval, and all research involves close partnership with clinicians and informatics subject matter experts. Furthermore, academic directors for the LACDHS Informatics Program have expertise in data and delivery system science that facilitates system priorities, differing from the traditional AMC focus on clinical trials. Technically and operationally, the LACDHS research data warehouse program is tightly integrated with the service model for the larger health system. In parallel with enterprise-wide policy, each of the medical centers serves as a “laboratory” for emerging policy and practice, successful local initiatives can be disseminated through the ROB and adapted as needed. While not typical, this type of partnership may have value, particularly as academic health systems acquire and merge with independently governed community hospitals.^1,33,34 Tightly coupled governance arrangements, while requiring effort from stakeholders to maintain and establish, have allowed the LACDHS research informatics program to endure transitions in health IT leadership that might disrupt a less well-integrated program. As a result of these factors, the balance of projects and research products have increased academic support for delivery system science, orienting toward healthcare program optimization and evaluation, and a greater impact on policy, population, and community health.

FUNDING

This work was supported by NIH National Center for Advancing Translational Science (NCATS) UCLA CTSI Grant Number UL1TR001881 (PI: Dubinet) and SC-CTSI Grant Number UL1TR001855 (PI: Buchanan) and UCLA CTSI info. No funder had any role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

AUTHOR CONTRIBUTIONS

All authors meet the ICMJE criteria for authorship. DM, KY, RF, and HFY obtained funding. GG collected the data. DM performed the analysis. DM wrote the first draft of the paper. All authors revised the paper. DM takes responsibility for the paper as a whole.

SUPPLEMENTARY MATERIAL

Supplementary material is available at Journal of the American Medical Informatics Association online.