Skip to main content
NCBI home page
JAMIA Open logoLink to JAMIA Open
. 2023 Aug 29;6(3):ooad074. doi: 10.1093/jamiaopen/ooad074

Integrating REDCap patient-reported outcomes with the HealtheIntent population health platform: proof of concept

Juan Espinoza 1,2,3,✉, Maurice Tut 4, Payal Shah 5, Paul Kingsbury 6, Gayathri Nagaraj 7, Daniella Meeker 8, Neil Bahroos 9
PMCID: PMC10463552  PMID: 37649989

Abstract

Objective

Patient-reported outcome measures (PROMs) are critical to drive patient-centered care and to understanding patients’ perspectives on their health status, quality of life, and the overall effectiveness of the care they receive. PROMs are increasingly being used in clinical and research settings, but the mechanisms to aggregate data from different systems can be cumbersome.

Materials and methods

As part of an FDA Real-World Evidence demonstration project, we enriched routine care clinical data from our Cerner electronic health record (EHR) with PROMs collected using REDCap. We used SSIS, sFTP, and the REDCap Application Programming Interface to aggregate both data sources into the Cerner HealtheIntent Population Health Platform.

Results

We successfully built dashboards, reports, and datasets containing both REDCap and EHR data collected prospectively.

Discussion

This technically straightforward approach using commonly available clinical and research tools can be readily adopted and adapted by others to better integrate PROMs with clinical data sources.

Keywords: REDCap, EHR integration, population health, patient-reported outcomes

Background

Modern healthcare often blurs the line between clinical and research data. A study may generate health data that should be incorporated into a patient’s electronic health record (EHR),1 or a study may seek to populate an electronic data collection form automatically from health data in the EHR.1,2 To address this, various solutions have been proposed.3–11 Broadly speaking, solutions can be classified into 3 approaches:

  1. Bring clinical data into a research platform;

  2. Bring research data into a clinical platform (eg, EHR); or

  3. Bring both clinical and research data into a third analytics platform.

Each strategy has its own advantages and disadvantages related to technical, workflow, effort, and compliance considerations. Here, we describe our implementation of the third approach: Bringing Cerner EHR clinical data and research-related patient-reported outcome measures (PROMs) collected using the Research Electronic Data Capture (REDCap) platform into the Cerner HealtheIntent population health management platform (Cerner Corporation, Kansas City, MO).

Real-world evidence project

Our institution has a U.S. Food and Drug Administration (FDA)-funded real-world evidence (RWE) demonstration project focused on pediatric patients with diabetes who use continuous glucose monitors (CGMs). RWE is clinical evidence derived from Real-World Data (RWD) analysis regarding the use, benefits, and risks of regulated medical products.12 RWD are data relating to patient health status and/or the delivery of health care routinely collected from a variety of sources, including the EHR.13,14 For this project, we needed to aggregate clinical data from the EHR, device data generated by CGMs (previously published),4 and PROMs. EHRs have limited native functionality for collecting PROMs,15 so we used REDCap for patient and parent-reported data.16,17 We leveraged HealtheIntent’s data aggregation and standardization capabilities to ingest all data and build clinical, research, and quality improvement dashboards and tools.18

Platforms used

Research electronic data capture (REDCap)

REDCap is a secure, web-based data management platform developed by Vanderbilt University and supported by the National Center for Advancing Translational Sciences.16,17 We use a HIPAA-compliant REDCap instance hosted by our university academic partner to support research (version 12.5.5).

HealtheIntent (HI)

HI is a cloud-based, programmable, source-agnostic platform that can receive data from any EHR, and a wide variety of other clinical and non-clinical data sources.19,20 The near real-time platform includes data warehousing, registry building, care management solutions, and enables health care systems to aggregate, transform, and reconcile data across the continuum of care, creating a longitudinal health record for individual patients. We selected HI because of its integration with our existing Cerner EHR (2018.07).

Objective

To describe our approach to aggregating Cerner and REDCap data in HI to enable real-world evidence research, and encourage the replication of this approach in other settings interested in integrating clinical and research data.

Materials and methods

Setting and resources

This study took place at a free-standing children’s hospital and academic medical center. Relevant infrastructure and required resources used in the project are shown in Table 1. All systems and tools used were already in place. The personnel were supported by Hospital Information Services, the Southern California Clinical and Translational Science Institute, and the RWE project grant.

Table 1.

Team and foundational IT infrastructure relevant to the project.

Application/platform Personnel
Electronic health record (EHR) Clinical informatics analyst—responsible for configurations and maintenance of an EHR
Visual studio professional Research programmer—responsible for programming an SSIS package to transfer data from REDCap API to designated server
REDCap Research coordinator—builds and maintains surveys, data, and survey workflow(s)
Web server in DMZ (demilitarized zone) network Server engineer—sets up or provides access to a web server in the DMZ to receive file(s) in a designated folder path
Rhapsody integration engine Database/integration analyst—transfer file(s) from DMZ web server to a designated HI share drive
HealtheIntent (HI) Cerner platform HI data analyst—ingestion and automation of file(s), visualizes data using Tableau Business Intelligence application
Open in a new tab

RWE study description

This project was executed under the auspices of Children’s Hospital Los Angeles Institutional Review Board, protocol CHLA-19-00028. All subjects provided signed informed consent/assent to participate. Patients 7 years and older using an insulin pump or continuous glucose monitoring for diabetes management were eligible to participate. We collected medical information generated during routine care, including from medical records, surveys, checklists, and other clinical data sources. We also collected medical device data including manufacturer, model, Unique Device Identifier (UDI), date ordered, and device generated data. Lastly, the patients were asked to complete disease- and device-specific PROMs periodically on their smart devices.21 The PROMs were administered via REDCap. The end goal of the project is to demonstrate the successful integration of multiple pediatric RWD sources into a single platform that can be used for RWE generation.

Results

System description

To integrate REDCap data into HI, we developed a SSIS (SQL Server Integration Services) package in Visual Studio that automates the process of loading a JSON file, exported using REDCap’s Application Programming Interface (API), and transferring the file via (s)FTP (secure file transfer protocol) as a flat file (.csv) to a server within CHLA’s network. As a prerequisite to file transfer, our HI technical team validated the file structure configured by the SSIS package to match HI data ingestion standards. The team also provided additional guidance on file naming convention and scheduled delivery frequencies.

SSIS package and sFTP transfer

To create a SSIS package in Visual Studio, the SQL Server Integration Services extension was enabled for SQL Server Data tools (SSDT) functionality. The SSIS package acts as the extract, transform, load (ETL) tool by creating data flow tasks and connection managers and executing (s)FTP transfer. The package executes the following tasks:

  • Transmits credentials to authenticate use of REDCap API

  • Calls for data, transmitted as a JSON file from REDCap API

  • Transforms data structure and file format (.csv)

    • Research specific data transformations

    • HI Data Extraction Requirements

  • Specifies file transfer schedule (twice a day, 5 am and 5 pm)

  • Transmits credentials to authenticate transfer to CHLA server

  • Transfers flat file (.csv) to designated server

Throughout this process, we worked with our Server engineer and Database/Integration Analyst to create the necessary accounts and shares in our CHLA DMZ server to give access for secure file transfer. Once our IT team validated the file transfer, our Database/Integration Analyst began to work on automating the transfer internally.

Integration engine

CHLA uses the Rhapsody integration engine for the standardization and automation for a majority of external clinical data. Once our Server team validated the transfer of the flat file, our Integration team built the automation to pick up the file from the designated server and route it to a specific folder path for HI ingestion. A delivery schedule was also identified, giving the (s)FTP transfer sufficient time before Rhapsody routes the file to HI. To ensure the successful pick-up and delivery of the file daily, specific IT resources receive emails as an auditing measure.

Cerner/REDCap identity matching

To assist with the identity matching, a powerform (Cerner documentation template) was built to enter participants’ research ID and IRB number into the EHR (Figure 1). This Research ID matches the participant ID in REDCap and is used to link REDCap and Cerner data. This makes it possible to avoid entering protected health information like name, date of birth, and medical record number into REDCap. The powerform is completed by a research staff member at the time of consent and takes approximately 30 s.

Figure 1.

Figure 1.

Open in a new tab

Screenshot of Cerner research ID powerform. The form has 3 required field: the IRB study number, the Research ID, and the enrollment (or start) date. A stop date field is available to be populated once the study is completed.

HI data ingestion

Before receiving the flat file from the integration engine, our HI team configured a share drive folder specific for REDCap data ingestion. After the successful transfer to the specific folder, HealtheIntent Data Upload Utility (HIDUU) was utilized to automate the ingestion of REDCap data to the HI application. Once uploaded, various data transformations were conducted to create the schema and define the dataset’s relationship with other datasets/data tables. It is at this point where REDCap data is now available to display within a HI dashboard for reporting purposes.

Figure 2 summarizes the information flow across systems.

Figure 2.

Figure 2.

Open in a new tab

Diagram of data flow across systems. All components from REDCap to the CHLA DMZ server are hosted at our affiliated university partner. The rest are hosted at our institution.

Data capture, display, and use cases

Our HI dashboard includes a routine care cohort of 2134 patients with type 1 diabetes to act as a control, and 10 research subject participants to date (recruitment is ongoing). Data refreshes every 24 h. We can track and compare demographics, healthcare encounters, lab results, ED visits, etc., between the research cohort and the routine care cohort (Figure 3). The endocrinology division uses the dashboard to track quality metrics for the diabetes program, US News and World Reports metrics, and variations in care by insurance type. The research team uses the clinical events data to trigger surveys, track past and upcoming appointments, and track completion of PROMs in a single platform. The dashboard is currently accessed by 4 members of the division’s clinical and administrative leadership, and 5 members of the research team. When called for routine research follow-up, participants have shared that “everything works great, no issues with the survey, it’s easy to fill them out from the links [generated from REDCap].” One patient felt that PROMs were too lengthy at first, but “once I got a hold of it, it was completely fine.” All data originating in the EHR is governed in accordance by existing policies, while REDCap PROM data are governed by the study IRB. This creates the need to manually control access for different users based on their institutional and study clearance.

Figure 3.

Figure 3.

Open in a new tab

Screenshots of HealtheIntent dashboards with both REDCap and EHR data. (A) EHR demographics data for the 2164 patients in the routine care cohort is shown. Using the “Cohort” menu in the top left corners, users can switch to the research cohort and see their EHR demographics data. This can be compared with the demographics data collected separately via REDCap (B) which may be more accurate given its self-reported nature. This includes other demographic variables not routinely captured in the EHR, such as health literacy level.

Discussion

PROMs are critical to both clinical and research applications, but native EHR tools are often inadequate for data collection, particularly for asynchronous and self-reported use cases. REDCap and similar platforms have superior functionality, better user experience, and are simple to program, but integrating the data into secure clinical data environments can be challenging from a technical and data governance perspective. In our approach, we used existing systems and tools to aggregate EHR and REDCap research data into a single population health platform with native integration, normalization, and visualization capabilities. Our first goal was to meet the needs of the FDA RWE project, and second was to create a reproducible architecture to generate regulatory-quality evidence related to pediatric diabetes device use. We hope to encourage other groups to develop similar data-unification approaches.

Our method has the advantage of utilizing mostly off-the-shelf tools, open APIs and other commonplace interfaces (sFTP, SSIS). We estimate that 10–20 engineer/programmer hours were required to build the system. Other researchers have tried different approaches to integrate REDCap data into the EHR and vice versa. One of the options is to use the Informatics for Integrating Biology and the Bedside (i2b2) platform.22 The Cerner EHR can utilize an API call to attach a pdf exported from REDCap, or to import discrete numerical data via csv file.23 Similarly, the Retrieving Form Data (RFD) function in Epic can also allow the EHR platform to display data collected from an EDC platform, but both the EHR and EDC must be RFD Capable.24 Another option is to use the Dynamic Data Pull (DDP) module in REDcap, which requires custom middleware.5 More recently, the REDCap RFD and DDP have been replaced by a Fast Interoperability Resources (FHIR) API interface that does not require middleware.25

While our exact implementation is dependent on HI as the final data platform, the architecture should be easily adapted to other platforms with programmable data ingestion interfaces. One advantage of using HI is that it is already being adopted by health systems for several data-related clinical and operational needs; this project shows that HI can also be leveraged for research applications. Furthermore, regardless of use case, integrating third party data capture tools like REDCap or Qualtrics may face fewer technical and governance barriers at the level of a population health platform like HI as compared to the EHR.

This project is a specific example of the broader category of integrated data repositories (IDR); platforms that ingest data from multiple sources with overlaid analytic tools to facilitate research, analysis, and decision-making.26 There are several use cases for this approach, ranging from research and clinical care to operations and finance. Based on our experience, we would recommend to other teams building IDRs to focus on the foundational components that make visualization and insights possible, including clear documentation of data provenance, developing comprehensive data dictionaries, establishing clear and transparent record linkage mechanisms, and implementing a data quality framework. In the future, we hope to extend this approach to other disease areas like asthma and sickle cell disease and close the loop by incorporating research data into the EHR.

Our project is not without limitations. The use of an FTP via a DMZ server is necessary because our REDCap instance is hosted at our affiliated university partner that is a distinct legal entity. At institutions with their own REDCap instance, connecting directly to the REDCap data API or FHIR API may be simpler, though it will still require appropriate governance approvals. HI and similar platforms represent significant financial investments in both the technology and support staff. Finally, external grants made it possible to both prioritize and fund this effort.

Conclusions

Patient-reported outcomes are key to several clinical and research processes. Non-EHR tools like REDCap are better suited to collecting this data in a dynamic, user-centered way. We demonstrated a straightforward approach to collecting and integrating PROMs using commonly available clinical and research data tools that others can replicate and expand upon.

Acknowledgments

The authors would like to thank Annette Schoen, Benjamin Castillo, David Raya, Ritesh Thakur, Hira Sherazi, Brent Whaling, and Ben Sykes for their technical assistance in implementing REDCap-HI integration.

Contributor Information

Juan Espinoza, Division of General Pediatrics, Department of Pediatrics, Children’s Hospital Los Angeles, CA, United States; Translational Informatics, Information Services Department, Children’s Hospital Los Angeles, Los Angeles, CA, United States; Keck School of Medicine, University of Southern California, Los Angeles, CA, United States.

Maurice Tut, Translational Informatics, Information Services Department, Children’s Hospital Los Angeles, Los Angeles, CA, United States.

Payal Shah, Division of General Pediatrics, Department of Pediatrics, Children’s Hospital Los Angeles, CA, United States.

Paul Kingsbury, Keck School of Medicine, University of Southern California, Los Angeles, CA, United States.

Gayathri Nagaraj, Keck School of Medicine, University of Southern California, Los Angeles, CA, United States.

Daniella Meeker, Keck School of Medicine, University of Southern California, Los Angeles, CA, United States.

Neil Bahroos, Keck School of Medicine, University of Southern California, Los Angeles, CA, United States.

Author contributions

All authors made substantial contributions to the conception and design of this research.

J.E. conceived the study and manuscript, and designed the architecture with D.M. and N.B. P.K., M.T., P.S., and G.N. all contributed to drafting the manuscript. All authors reviewed and revised the manuscript critically for important intellectual content and gave final approval of the version published.

Funding

This work was supported by the Food and Drug Administration under award number P50FD006425 (PI: Espinoza) for The West Coast Consortium for Technology & Innovation in Pediatrics, and by grants UL1TR001855 and UL1TR000130 (PIs: Buchanan/Kipke) from the National Center for Advancing Translational Science (NCATS) of the U.S. National Institutes of Health. The funding sources had no involvement in the development of this manuscript or in the decision to submit the paper for publication. The content is solely the responsibility of the authors and does not necessarily represent the official views of the FDA or the NIH.

Conflicts of interest

None declared.

Data Availability

No new data were generated or analyzed in support of this research.

References

  • 1. Kush R, Alschuler L, Ruggeri R, et al. Implementing single source: the STARBRITE proof-of-concept study. J Am Med Inform Assoc. 2007;14(5):662-673. 10.1197/jamia.M2157 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2. D'Avolio L, Ferguson R, Goryachev S, et al. Implementation of the Department of Veterans Affairs’ first point-of-care clinical trial. J Am Med Inform Assoc. 2012;19(e1):e170-e176. 10.1136/amiajnl-2011-000623 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3. Yamamoto K, Ota K, Akiya I, Shintani A.. A pragmatic method for transforming clinical research data from the research electronic data capture “REDCap” to clinical data interchange standards consortium (CDISC) study data tabulation model (SDTM): development and evaluation of REDCap2SDTM. J Biomed Inform. 2017;70:65-76. 10.1016/j.jbi.2017.05.003 [DOI] [PubMed] [Google Scholar]
  • 4. Espinoza J, Shah P, Raymond J.. Integrating continuous glucose monitor data directly into the electronic health record: Proof of concept. Diabetes Technol Ther. 2020;22(8):570-576. 10.1089/dia.2019.0377 [DOI] [PubMed] [Google Scholar]
  • 5. Campion TR, Sholle ET, Davila MA.. Generalizable middleware to support use of REDCap dynamic data pull for integrating clinical and research data. AMIA Jt Summits Transl Sci Proc AMIA Jt Summits Transl Sci. 2017;2017:76-81. [PMC free article] [PubMed] [Google Scholar]
  • 6. Mattingly W, Sinclair C, Williams D, et al. A software tool for automated upload of large clinical datasets using REDCap and the CAPO database. JRI. 2018;2(1):7. 10.18297/jri/vol2/iss1/7 [DOI] [Google Scholar]
  • 7. Gabetta M, Mirabelli M, Klersy C, et al. An extension of the i2b2 data warehouse to support REDCap dynamic data pull. Stud Health Technol Inform. 2019;258:21-25. [PubMed] [Google Scholar]
  • 8. Dunn WD, Cobb J, Levey AI, Gutman DA.. REDLetr: workflow and tools to support the migration of legacy clinical data capture systems to REDCap. Int J Med Inform. 2016;93:103-110. 10.1016/j.ijmedinf.2016.06.015 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9. Chen C, Turner SP, Sholle ET, et al. Evaluation of a REDCap-based workflow for supporting federal guidance for electronic informed consent. AMIA Jt Summits Transl Sci Proc AMIA Jt Summits Transl Sci. 2019;2019:163-172. [PMC free article] [PubMed] [Google Scholar]
  • 10. Carvajal C, Vallejos C, Lemaitre D, et al. A REDCap application that links researchers, animal facility staff and members of the IACUC in animal health monitoring. Lab Anim. 2019;53(5):500-507. 10.1177/0023677218815723 [DOI] [PubMed] [Google Scholar]
  • 11. Gesell SB, Halladay JR, Mettam LH, Sissine ME, Staplefoote-Boynton BL, Duncan PW.. Using REDCap to track stakeholder engagement: a time-saving tool for PCORI-funded studies. J Clin Transl Sci. 2020;4(2):108-114. 10.1017/cts.2019.444 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12. Fleurence RL, Forrest CB, Shuren J.. Strengthening the evidence base for pediatric medical devices using real-world data. J Pediatr. 2019;214:209-211. 10.1016/j.jpeds.2019.06.060 [DOI] [PubMed] [Google Scholar]
  • 13. The U.S. Food and Drug Administration. FRAMEWORK FOR FDA’S REAL-WORLD EVIDENCE PROGRAM. Published online December 2018. Accessed July 19, 2022. https://www.fda.gov/media/120060/download.
  • 14. Real-World Evidence. FDA. Published May 16, 2019. Accessed October 2022. http://www.fda.gov/science-research/science-and-research-special-topics/real-world-evidence.
  • 15. Gensheimer SG, Wu AW, Snyder CF; PRO-EHR Users’ Guide Steering Group, PRO-EHR Users’ Guide Working Group. Oh, the places we’ll go: Patient-Reported outcomes and electronic health records. Patient. 2018;11(6):591-598. 10.1007/s40271-018-0321-9 [DOI] [PubMed] [Google Scholar]
  • 16. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG.. Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42(2):377-381. 10.1016/j.jbi.2008.08.010 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17. Harris PA, Taylor R, Minor BL, et al. ; REDCap Consortium. The REDCap consortium: Building an international community of software platform partners. J Biomed Inform. 2019;95:103208. 10.1016/j.jbi.2019.103208 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18. Ehwerhemuepha L, Carlson K, Moog R, et al. Cerner real-world data (CRWD) - a de-identified multicenter electronic health records database. Data Brief. 2022;42:108120. 10.1016/j.dib.2022.108120 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19. Population Health Management solution. Cerner. Accessed July 19, 2022. https://www.cerner.com/solutions/population-health-management.
  • 20. Gray K. Public health platforms: an emerging informatics approach to health professional learning and development. J Public Health Res. 2016;5(1):665. 10.4081/jphr.2016.665 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21. Shah P, Raymond JK, Espinoza J.. Modified E-Delphi process for selection of patient reported outcome measures for children and families with type-1 diabetes using continuous glucose monitors. JMIR Diabetes. 2022;7(4):e38660. 10.2196/38660. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22. Tibollo V, Bucalo M, Vella D, Stuppia M, Barbarini N, Bellazzi R.. Procedure to integrate I2b2 and REDCap: a case study at ICSM. PeerJ 2017;5:e3294v1. 10.7287/peerj.preprints.3294v1 [DOI] [Google Scholar]
  • 23. Hawley S, Yu J, Bogetic N, et al. Digitization of measurement-based care pathways in mental health through REDCap and electronic health record integration: development and usability study. J Med Internet Res. 2021;23(5):e25656. 10.2196/25656 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24. Nordo AH, Eisenstein EL, Hawley J, et al. A comparative effectiveness study of eSource used for data capture for a clinical research registry. Int J Med Inform. 2017;103:89-94. 10.1016/j.ijmedinf.2017.04.015 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25. Cheng AC, Duda SN, Taylor R, et al. REDCap on FHIR: Clinical data interoperability services. J Biomed Inform. 2021;121:103871. 10.1016/j.jbi.2021.103871. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26. MacKenzie SL, Wyatt MC, Schuff R, et al. Practices and perspectives on building integrated data repositories: results from a 2010 CTSA survey. J Am Med Inform Assoc. 2012;19(e1):e119-e124. 10.1136/amiajnl-2011-000508 [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Data Availability Statement

No new data were generated or analyzed in support of this research.

Articles from JAMIA Open are provided here courtesy of Oxford University Press

RESOURCES