The ecosystem of apps and software integrated with certified health information technology

Abstract

Objective

This study: 1) characterized the app market by EHR app gallery and type of app; 2) tracked changes in the EHR app galleries from the end of 2019 through 2020; and 3) examined how apps connect to EHR data systems, and if the apps support the HL7 FHIR standard.

Materials and Methods

We developed a program that gathered data from the public app galleries hosted by Allscripts, athenahealth, Cerner Corporation, Epic Systems Corporation, and SMART. Data collection for this study began in December 2019 and ended December 2020. The program was run 11 times during this period, and the data collected were used to generate the findings and trends observed in this study.

Results

The total number of unique apps increased from 600 to 734 during the study period. The most common types of apps marketed were intended for administrative (42%) and clinical use (38%). About 1 in 5 apps (22%) described support for the FHIR standard. Support for FHIR varied by intended functionality and gallery.

Discussion

This study provides early insights into the number of third-party apps that are connecting to EHRs, what services they provide, and if these connections use standards-based application programming interfaces (APIs).

Conclusion

It is a federal government priority to improve the access and use of electronic health information, including third-party apps that can introduce competition as well as best-of-breed functions and user experiences. This study shows that there is room for growth, and variation exists among some of the largest EHR developers.

INTRODUCTION

The integration of software, applications, and devices with electronic health records (EHRs) is an important process to broaden the utility of clinical data stored in EHRs. Software integration also expands the tools health care providers can use to diagnose and treat patients and gives patients and their caregivers new ways to access their electronic health information. Application programming interfaces (APIs) enhance software development and permit predictable, automated communication between different computer systems, enabling easier access and exchange of data. Health care digitization can be further achieved through broader availability of APIs within EHRs. In particular, standardized APIs can harmonize API connections across different EHRs and facilitate third-party software integration.¹

APIs are common in health IT software development, and standardization of these APIs is a federal policy goal. Different API configurations across multiple EHR connections can increase costs and create barriers that can make it harder to exchange data at scale and more difficult for competitive services to enter the market. Therefore, the 21st Century Cures Act (Cures Act) of 2016, required health IT developers to publish APIs that allow “health information from such technology to be accessed, exchanged, and used without special effort through the use of APIs or successor technology or standards, as provided for under applicable law.”² The Office of the National Coordinator for Health Information Technology (ONC) Cures Act Final Rule, published in May 2020, implements key provisions of the Cures Act to advance interoperability and support the access, exchange, and use of electronic health information.³

The ONC Cures Act Final Rule establishes a number of conditions to promote transparent and pro-competitive business practices that facilitate the adoption and use of APIs by third-party app developers. These conditions establish fees that EHR developers are permitted to charge API users and require the publication of specific business and technical documentation necessary to interact with certified API technology. The rule also created a new ONC Health IT Certification Program certification criterion, Standardized API for Patient and Population Services (45 CFR 170.315(g)(10)) that requires use of the Health Level Seven (HL7) Fast Healthcare Interoperability Resources (FHIR) data exchange standard to enable third-party app developers to connect to certified EHRs. In recent years, health IT stakeholders have reached broad consensus about the utility of the FHIR standard, and health IT market leaders have moved toward support of FHIR.⁴ This article seeks to understand whether new federal policies will lead to increased use of the FHIR standard among the third-party app developers who integrate their technology with EHR data systems.

BACKGROUND AND SIGNIFICANCE

The ONC Cures Act Final Rule and the federal law it executes draw inspiration from the findings and recommendations of the JASON report, A Robust Health Data Infrastructure.⁵ The report recognized the promise of health care digitization and the interoperability challenges that limit its achievement. Among its recommendations to achieve a robust health data infrastructure, the report urged that EHR developers be required to develop and publish APIs for medical records and demonstrate “that data from their EHRs can be exchanged through the use of these APIs and used in a meaningful way by third-party software developers.” In other words, meaningful exchange of data can only occur when differing software systems are designed upon common standards for storing and transporting medical record information, and there are “APIs that allow third-party programmers (and hence, users) to bridge from existing systems to a future software ecosystem that will be built on top of the stored data.”

Substitutable Medical Apps & Reusable Technology (SMART), a 2010 awardee of the ONC Strategic Health IT Advanced Research Projects (SHARP) program, laid the groundwork necessary to enable a shift to a flexible health IT environment where APIs would allow integration of apps into EHRs.⁶ SMART demonstrated the value of substitutable apps to support use cases associated with clinical care and research, and the team worked with HL7 to develop a prototype API—SMART on FHIR—that used FHIR to connect apps to EHRs. By 2014, EHR developers had demonstrated use of SMART on FHIR in real-world clinical settings.

SMART project leads, Mandl et al, recognized the opportunities of opening EHRs up to third-party innovation in their 2015 article, “Driving innovation in health systems through an apps-based information economy.”⁷ The authors asserted that EHR developers opening up their platforms to third-party integration would be a mutually beneficial enterprise, where third-party developers’ innovations could reduce EHR developers’ needs to update and deploy new functionalities for their EHR. They posited that a robust app ecosystem would nurture the development of software that could be used for population health, data integration from fitness trackers, remote patient monitoring, and other health care activities. This vision was shared by EHR developers and users who formed the Argonaut Project to develop standards-based APIs, using the HL7 FHIR data exchange standard, to make it easier to develop applications that share electronic health information with EHRs and other health IT software.^8–9

Given that federal health IT policy and EHR and other heath IT developers’ business practices are shaped by these concepts, it is important to understand the actual progress of these efforts. Federal rule-making finalized in 2020 requires that certified API technology within EHRs must conform to the FHIR data exchange standard and that API developers must provide public access to the documentation associated with these APIs to facilitate third-party integration of apps and software with the EHR. While developers of certified health IT have until December 2022 to comply with the slate of new API certification requirements, there is some early insight into EHR developer policies for third-party integration and the types of third-party apps that are integrating with EHRs.

The California Health Care Foundation (CHCF) commissioned 2 surveys, first in 2016 and then in 2018, that surveyed application developers about their experiences working with EHR developers to integrate their software with EHR data systems. In 2016, the survey found that 85% of respondents said integration with an EHR was vital or “nice to have” for their product or service, and 70% had attempted an integration with an EHR company.¹⁰ Respondents found the integration process to be immature and EHR developer resources for third-party developers to be limited. In 2018, 98% of respondents said integration with an EHR was vital or “nice to have” for their product or service, and 85% had attempted an integration with an EHR.¹¹ Respondents in 2018 found that EHR developer resources for third-party developers and EHR developer APIs had improved. Results from both surveys showed app developers use various methods to integrate with EHRs.

In 2018, NORC at the University of Chicago conducted a study of the app galleries managed by individual EHR developers that market applications for their clients to connect to their EHRs.¹² NORC manually examined the galleries managed by 3 leading EHR developers: Allscripts, Cerner Corporation, and Epic Systems Corporation as well as the SMART App Gallery, managed by Boston Children’s Hospital. Researchers found that the galleries consisted of various applications that provide administrative, care management, patient engagement, and clinical research functionalities. The number of applications discoverable in these galleries also varied. Their study found a combined total of 271 applications available in these galleries (as of October 2018).

NORC’s research, like the CHCF survey, found that EHR developers were providing resources and public APIs to third-party developers to connect to their data systems, but that standards-based APIs, including FHIR, were not as widely used. They note that at the time of their study, ONC had not required standards within the adopted API certification criteria nor had a version of FHIR with normative requirements been finalized. Since then, such a FHIR version—Release 4—was finalized and adopted as a standard for certifying the new Health IT Certification Program criterion, Standardized API for Patient and Population Services (45 CFR 170.315(g)(10)). This could be seen as a stabilizing force to encourage adoption of this standard by EHR developers and use by third-party application and software developers to integrate their technology with EHR data systems.

Our study, similar to the NORC study previously referenced, examined apps discoverable in the galleries hosted by Allscripts, athenahealth, Cerner Corporation, Epic Systems Corporation, and SMART.^13–17 This study is novel from prior studies because it collected and measured this data over time, rather than at a single point in time. Our automated data collection program allows us to more easily collect this data over time. We have continued this data collection beyond the study period to observe changes in these galleries and EHR developer policies for third-party integration as federal policies go into effect.

These galleries provide one way to observe what apps are integrating to EHRs. As we look to foster a robust health data infrastructure and an interoperable health system, it is important to know more about how third-party apps are connecting to EHRs, what services they provide, and if these connections are happening in an interoperable way. This study provides an early look at what we see as a maturing market for third-party integration, but in need of more standardization and harmony.

The objectives of this study were to: 1) characterize the app market by gallery and type of app, 2) track changes in the app galleries from December 2019 through December 2020, and 3) examine how these apps connect to EHR data systems and if the apps support the FHIR standard.

MATERIALS AND METHODS

We developed an automated, programmed process that gathers data from the public app galleries hosted by Allscripts, athenahealth, Cerner Corporation, Epic Systems Corporation, and SMART (R statistical software was used to program and automate the data pulls and web scraping of the public websites. The specific R packages used were: (1) rvest, (2) httr, and (3) rjson.). The program either queries publicly available APIs or scrapes the hypertext markup language (HTML) of the website for this data. The data are then structured in a uniform format, aggregating data from the 5 galleries. Data collection for this study began in December 2019 and ended December 2020. The program was run 11 times during this period (The specific dates of each data pull were: 12/30/2019; 02/28/2020; 04/06/2020; 05/06/2020; 06/16/2020; 07/14/2020; 08/18/2020; 09/16/2020; 10/28/2020; 11/20/2020; 12/15/2020). Minor adjustments were made to programmatic scripts to respond to changes to the layout or design of some of the public websites over time.

This process, similar to an automated method implemented by Ritchie & Welch,¹⁸ ensures consistency in data collection, auditing of the process, and reproducibility. We did not manually collect or enter data in our study’s dataset, which can lead to human errors or inconsistences in recording observations. Although other researchers cannot go back in time and pull the same data now, they can reproduce the process to compare newly collected data with those data collected in prior runs. Descriptive and bivariate analyses were conducted to characterize apps discovered through these galleries, identify the intended use or functionality of these apps, and estimate use of the HL7 FHIR data exchange standard. Two-sample t-tests were performed to test for statistically significant differences (P < .05).

As part of this project’s regular data collection, we used a text analysis program to crawl the marketing materials discoverable through each app gallery to help identify the intended use or functionality of apps and to estimate the proportion of apps that describe support for FHIR (R Statistical Software was used to perform text analysis. The specifc R packages used were: (1) tidytext, (2) topicmodels, and (3) SnowballC). To assess intended use or functional category, we used text analysis to identify the most commonly used terms in each app description. These terms were then assigned functional categories by our research team. Categories assigned to apps by their respective gallery were also used to supplement this data. We chose 5 high-level functional categories and 16 specific subcategories to characterize each app. These categories were primarily based off prior research^12,18 but modified to fit our data. See Supplementary AppendixTable 1 for a list of terms used to define each functional category.

Table 1.

Number of apps by functional category

Administrative (42%)	Clinical Use (38%)	Care Management (31%)	Patient Engagement (20%)	Research (5%)
Scheduling and Check-In (n = 208)	Automated Tasks (n = 129)	Information Management (n = 94)	Patient Education (n = 129)	Clinical Research (n = 37)
Billing and Payment (n = 153)	Population Health (n = 99)	Disease Management (n = 44)	Patient Satisfaction Surveys (n = 71)
	Telehealth (n = 63)	Care Planning (n = 44)	Secure Messaging (n = 63)
	Clinical Decision Support (n = 53)	Patient Monitoring (n = 30)	Patient Access (n = 13)
		Medication Management (n = 17)
N = 306	N = 278	N = 228	N = 148	N = 37

Notes: (1) Automated Tasks include: data visualization tools, risk calculators, and natural language processing. (2) Applications may be included in multiple functional categories, therefore, numbers may not add up to the total. (3) Table represents the number of apps discovered in the online app galleries hosted by Allscripts, Athenahealth, Cerner, Epic, and SMART from December 30, 2019 to December 15, 2020 (N = 734).

Apps that described support for the FHIR standard were added to apps discovered in the SMART App Gallery (all apps discoverable in the SMART gallery support the SMART on FHIR standard) to create a list of apps that support FHIR. We performed quality checks to assess the accuracy of our text analysis method at identifying apps that supported FHIR. We manually reviewed the marketing materials and public websites for 25 random apps that described support for FHIR and 25 random apps that did not describe support for FHIR. We used these results to: (1) compare changes in FHIR support among all apps at the end of 2019 compared to the end of 2020; (2) identify apps that did not describe support for FHIR in 2019 but did so in 2020; and (3) measure any change in the overall support for FHIR among apps discoverable in these galleries.

RESULTS

Characteristics of the app market

From December 2019 through December 2020, the total number of unique apps and developers discovered through these 5 galleries increased by about 20% from 600 to 734 apps and 517 to 610 developers, respectively. Figure 1 shows the total number of apps that were added or removed for each gallery during 2020. Three galleries had a net increase in the total number of apps (Cerner, Epic, and SMART), while 2 galleries had a net decrease in apps (athenahealth and Allscripts). The Epic App Orchard saw the largest net increase (43%), adding a net of 169 apps during the study period.

Figure 1.

Number of apps added and removed from each app gallery in 2020.

Notes: (1) Added = all apps discovered for the first time in each gallery during 2020. Removed = all apps, previously discovered, not found during final December 15, 2020 data pull. Current = number of apps discovered on December 15, 2020. (2) Applications may be listed in multiple galleries, therefore added or removed applications in a given store may not be reflected in the “All 5 Galleries” group if that application is listed in another gallery. The “All 5 Galleries” group reflects unique applications. (3) Figure represents the number of applications discovered in the online app galleries hosted by Allscripts, Athenahealth, Cerner, Epic, and SMART from December 30, 2019 to December 15, 2020.

Table 1 shows the total number of apps by app type or intended use across the 5 galleries. Administrative apps were the most common (42%). These apps’ primary functions were to help with activities such as scheduling or check-in, as well as billing and payment. Clinical use apps were the second most common (38%). These apps helped perform automated tasks, population health, telehealth, and clinical decision support. The remaining apps fell into 3 categories: care management (31%), patient engagement (20%), and research (5%). We found a number of apps that fit into multiple categories based on their functionalities, demonstrating that apps may perform a variety of tasks.

FHIR measurement

As discussed in Materials and Methods, we determined FHIR support through a text analysis of app descriptions pulled from each individual gallery. We verified that all the apps identified through our text analysis as supporting FHIR did, in fact, support FHIR. We also found that 2 of the 25 apps that our automated method classified as not supporting FHIR, did describe support for FHIR upon manual review of information on their public website. This information was not in the app description in the gallery where we discovered the app. This indicates that our process to identify apps that support FHIR may currently underestimate the total number of apps that support FHIR.

At the end of 2019, 112 unique apps (19%) across all 5 galleries described support for FHIR. Figure 2 shows that, as of December 2020, 161 unique applications (22%) supported FHIR. There was not a statistically significant increase in the percent of apps identified as using FHIR during the study period (P = .14). We also examined the increase in FHIR support among apps over time to see if it reflected a real increase in apps that support FHIR, or if the proportions, instead, reflected the exit of apps from these galleries that don’t support FHIR.

Figure 2.

Number of apps that support FHIR for each gallery.

Notes: (1) Added = all apps discovered for the first time in each gallery during 2020. Removed = all apps, previously discovered, not found during final December 15, 2020 data pull. Current = number of apps discovered on December 15, 2020. (2) Applications may be listed in multiple galleries, therefore added or removed applications in a given store may not be reflected in the “All 5 Galleries” group if that application is listed in another gallery. The “All 5 Galleries” group reflects unique applications. (3) Figure represents the number of apps discovered in the online app galleries hosted by Allscripts, Athenahealth, Cerner, Epic, and SMART from December 30, 2019 to December 15, 2020.

As shown in Figure 1, at the end of 2020, 83 of the 600 unique apps discovered through 2019 were no longer discoverable. Seventy-nine of these apps (95%) did not support FHIR. Of the remaining 517 apps discovered in 2019 that remained in the galleries through 2020, 6 apps described support for FHIR for the first time in 2020. So, on net, at the end of 2020, 114 of the remaining 517 apps (22%), first discovered in 2019, supported FHIR.

In 2020, 217 new unique apps were discovered, and 47 apps (22%) described support for FHIR. Together, 22% of the remaining 2019 applications and new 2020 applications described support for FHIR. Much of the percentage change is due to the exit of apps from the galleries that did not support FHIR versus a real increase in apps that supported the data exchange standard. It is noteworthy, however, that the percent of new 2020 apps that supported FHIR was greater than in 2019.

Most of the new 2020 apps were discovered in the Epic App Orchard and SMART App Gallery, so the gains were not even across all galleries. Beyond the SMART Gallery, where all apps support FHIR, we found variation in the number of apps that described support for FHIR among the 4 EHR developer galleries. Whereas 1 in 5 apps in the Epic App Orchard and nearly half of apps in the Cerner gallery supported FHIR, less than 5% of apps in the athenahealth gallery and less than 10% of apps discovered through Allscripts described supporting FHIR.

Figure 3 shows variation among apps that described support for FHIR by the number of galleries an app was marketed in. One in 5 apps listed in only 1 gallery described support for FHIR; this proportion was significantly lower compared to 34% of apps that were listed in 2 galleries (P = .005), and 46% of apps that were listed in 3 or more galleries (P = .004).

Figure 3.

Proportion of apps that use the FHIR standard by the number of galleries where the app was discovered from December 2019 to December 2020.

Notes: *Statistically significant from the “1 Gallery” group. Figure represents the number of apps discovered in the online app galleries hosted by Allscripts, Athenahealth, Cerner, Epic, and SMART from December 30, 2019 to December 15, 2020.

Figure 4 shows that FHIR support varies by app type or intended use. Administrative apps were found to describe support for FHIR less than other types of apps. This may be due to the type of data exchanged and functionalities performed by these apps.

Figure 4.

Proportion of apps that use the FHIR standard by functional category.

Notes: (1) Functional categories are not mutually exclusive. (2) Figure represents the number of apps discovered in the online app galleries hosted by Allscripts, Athenahealth, Cerner, Epic, and SMART from December 30, 2019 to December 15, 2020.

DISCUSSION

This study helps characterize the market for third-party apps that integrate with EHR data systems and identifies changes in the market from 2019 through 2020 in advance of the implementation of ONC’s Cures Act Final Rule. The data show that while the total number of apps that integrated with EHR systems increased from 2019 to 2020, the proportion of apps that support the FHIR data exchange standard—about 1 in 5—remained relatively unchanged.

The FHIR data exchange standard represents 1 major advance toward consensus on how to exchange patient health records. Support for the standard is growing among EHR developers.⁴ The data show that support is growing among app developers, too, but modestly. One explanation for the modest growth is that some types of apps are less likely to use or describe support for FHIR. For example, administrative apps, designed to help medical practices with scheduling, patient check-in, or bill pay, were the most common type of app marketed in 2020. Fewer than 1 in 10 of these apps described support for FHIR compared to other apps designed for clinical use or care management, which supported FHIR at higher rates.

Additionally, variation in FHIR support may be because FHIR apps are currently being developed around a set of specific use cases, or that FHIR resources are limited to the exchange of certain data elements. This is especially true for earlier versions of the FHIR standard. As the number of FHIR resources and supported data elements increases and EHR developers’ support for these FHIR resources grows, so too may the use cases supported by these apps.

We also found that FHIR-based apps were more likely to be found when listed across multiple galleries. The proportion of apps supporting FHIR was 2 times greater among apps that were listed in 3 or more galleries (46%) compared to apps listed in 1 gallery (20%). This may indicate that FHIR-based apps can connect to multiple EHR systems more easily compared to apps that rely solely on proprietary APIs. These will be important data to track. If API standardization makes integrations easier, there will be more interoperable data exchanged and more end users will be able to connect to their data.

These findings show that as integration of apps with EHRs continues to grow, use of FHIR among these apps grew modestly. As federal regulations are implemented and support for FHIR grows, we will continue to measure these changes to understand the impact of these policies and the effect of standardization on the market.

CONCLUSION

This study provides baseline data in advance of the ONC Cures Act Final Rule’s compliance dates. Now that FHIR is required as part of the ONC Health IT Certification Program, it will be important to measure its use both among EHR developers and third-party apps that integrate with these EHR data systems. A study of these app galleries is one way to measure these changes. However, there are limitations with this data source, as the apps discoverable in each gallery may not be representative of all apps that have integrated with these EHR data systems.

For example, apps in a given gallery may only represent apps integrated with the EHR that are specifically marketed to the EHR developer’s customers. Since an EHR developer’s client base is primarily health care providers and their organizations, these galleries may not accurately reflect the number of available apps used by patients or consumers of health care. Patient-facing applications, for example, may be more widely marketed in smartphone app stores, such as the Apple or Google app stores.

In addition, app developers may also have business relationships or partnerships with specific hospitals, health systems, or practice groups where they implement one-off integration solutions. These apps may use the EHR’s APIs but only serve an individual customer, and the app developer has no need to market the app more broadly. These apps may not be included in the EHR app galleries, and, therefore, are not represented in our data.

Observed changes across these galleries may also reflect unmeasured differences in EHR developer business and third-party integration policies that could affect the number of apps discoverable in their public galleries. For instance, some developers may charge varying fees to app developers for placement in their gallery. EHR developers may also require that all apps undergo integration testing or be validated prior to being listed in a gallery.

Other research conducted by the authors in parallel to this study found that some apps that do not specifically describe support for FHIR in marketing materials do, in fact, use FHIR APIs to connect their app to some EHR data systems. While EHR developers do provide access to certain data or functionalities using proprietary APIs, there is some initial evidence that certain data or tasks are only accessible through FHIR APIs or there’s a choice between a FHIR and a proprietary API.

We also recognize that FHIR support by technology developers is not binary. Although this study estimates FHIR using a binary definition (whether or not a developer describes support in an app’s marketing materials), during real-world implementation the FHIR standard exists on a spectrum with multiple standard versions and API specifications. This includes whether or not an app that supports FHIR uses an API that conforms to the US Core Data for Interoperability (USCDI) data standard or EHR-specific API capabilities.

This study may, therefore, underestimate the total number of apps that integrate with EHRs today and the number that support FHIR (and in what way they use FHIR). While this limits our ability to make conclusive inferences about the overall state of the app ecosystem and the use of FHIR among these apps, we do believe our study and methods, which leverage publicly available data, provide insights into this ecosystem and will allow us to monitor changes over time. New data sources and research will be needed for deeper understanding of third-party app integrations and the effect of federal policy.

For example, a recent landscape survey by Jones et al assessed the plans of various organizations to implement the SMART/HL7 Bulk FHIR Access API, an API that specializes in the access and integration of data from multiple patients. They found that organizations implemented the FHIR API for a wide array of use cases.¹⁹ This API differs from those used by apps to integrate with EHRs for a single patient but does show a rise in current and planned implementations, which could portend an increase in the use of FHIR among app integrations.

Our study parallels efforts by the federal government to improve the access and use of electronic health information using third-party apps. While progress has been made, there is still room for growth, and variation exists among some of the largest EHR developers. Monitoring the impacts of the rule will be important to providing insights into whether the goals to expand patient access to their electronic health information will be realized.

FUNDING

This study was conducted at the Department of Health and Human Services. The views expressed in this article are those of the authors, and no official endorsement by the Office of the National Coordinator for Health Information Technology or Department of Health and Human Services is intended or should be inferred.

AUTHOR CONTRIBUTIONS

WB contributed to the conception, design, data acquisition, and interpretation and critically revised the manuscript. CJ contributed to the design, data acquisition, and interpretation, and critically revised the manuscript.

SUPPLEMENTARY MATERIAL

Supplementary material is available at Journal of the American Medical Informatics Association online.

DATA AVAILABILITY STATEMENT

The data underlying this article is original data generated in the course of the study. It was collected from publicly available websites using web-scraping and other automated tools that collect structured data from the web. Data will be shared on request to the corresponding author.

CONFLICT OF INTEREST STATEMENT

None declared.