Experiences with information blocking in the United States: a national survey of hospitals

Abstract

Objective

The 21st Century Cures Act Final Rule’s information blocking provisions, which prohibited practices likely to interfere with, prevent, or materially discourage access, exchange, or use of electronic health information (EHI), began to apply to a limited set of data elements in April 2021 and expanded to all EHI in October 2022. We sought to describe hospital leaders’ perceptions of the prevalence of practices that may constitute information blocking, by actor and hospital characteristics, following the rule’s applicability date.

Materials and Methods

Cross-sectional analysis of a national survey of hospitals fielded in 2021. The analytic sample included 2092 nonfederal acute care hospitals in the United States. We present descriptive statistics on the perception of the prevalence of information blocking and results of multivariate regression models examining the association between hospital, health information technology (IT) developer and market characteristics and the perception of information blocking.

Results

Overall, 42% of hospitals reported observing some behavior they perceived to be information blocking. Thirty-six percent of responding hospitals perceived that healthcare providers either sometimes or often engaged in practices that may constitute information blocking, while 17% and 19% perceived that health IT developers (such as EHR developers) and State, regional and/or local health information exchanges did the same, respectively. Prevalence varied by health IT developer market share, hospital for-profit status, and health system market share.

Conclusions and Relevance

These results support the value of efforts to further reduce friction in the exchange of EHI and support the need for continued observation to provide a sense of the prevalence of information blocking practices and for education and awareness of information blocking regulations.

INTRODUCTION

There is substantial concern that health care providers, health information networks and exchanges, and health information technology (IT) developers (including EHR vendors and other developers of health IT certified by the Office of the National Coordinator for Health IT [ONC], part of the U.S. Department of Health and Human Services) face limited incentives and in some cases disincentives to share electronic health information (EHI) with patients and other organizations, especially their competitors.^1–5 In 2016, Congress included provisions in the 21st Century Cures Act that prohibited practices likely to interfere with, prevent, or materially discourage access, exchange, or use of EHI, known as information blocking practices.⁶ One high profile example of practices that may constitute information blocking is the delayed release of test results to patients⁷; however, information blocking could also occur if an actor did not share information with healthcare providers, such as hospitals, that were appropriate recipients of the information.

Following passage of the 21st Century Cures Act in 2016, the applicability date of information blocking regulations was initially set for November of 2020. After a delay, the applicability date of information blocking regulations contained in the HHS 21st Century Cures Act Final Rule occurred in April 2021.⁸ From April 2021 to October 2022, the scope of EHI for information blocking purposes was limited to provide more time for organizations subject to the information blocking rule to grow more experienced with the information blocking regulations.^8–10 Finally, as of October 6, 2022, all EHI is in scope.¹¹ As described by the regulation, failure to exchange EHI and failure to make EHI available for use could constitute information blocking under certain circumstances. However, whether a practice (an act or omission) constituted information blocking depends on the unique facts and circumstances of the practice.¹² Despite the delays described above, some healthcare organizations and clinician groups have expressed concerns that they were not ready to meet the expanded regulatory requirements regarding sharing EHI.^13,14

Limited existing empirical evidence is available on the prevalence of practices that may constitute information blocking and their correlates from the perspective of the healthcare delivery system.^5,15,16 The available evidence provides insight into potential information blocking both before³ and after¹⁶ implementation of the Cures Act Final Rule but often relies on anecdote, reflects the views of one type of actor (health information exchanges, which hold a specific view of the overall market), or reflects only the views of individuals sufficiently informed and activated to submit a complaint. Additional information from other perspectives on the prevalence and correlates of information blocking from could therefore augment this evidence.¹⁶

Similarly, initial evidence has indicated changes in some behaviors related to information blocking following issuance of the final rule and regulations,¹⁷ but the broader impact of the information blocking regulations on the prevalence of practices that may constitute information blocking is still uncertain. Among clinicians and healthcare organizations, substantial research indicates that market position relative to competitors and broader market factors such as the degree of market consolidation shape information sharing behaviors.^18–20 Subsequently, some changes in practices that could be considered information blocking may depend on the extent to which local dynamics incentivized those practices in the first place.

For policy makers, insights into the perceived prevalence and variation in information blocking could inform the development of future regulatory strategies, educational materials, and stakeholder outreach related to information blocking. These insights might also help inform market participants of how they might better facilitate the sharing of EHI. This knowledge may be particularly important for entities such as health IT developers, healthcare providers, and health information exchanges and networks that may wish to respond to concerns around their business practices. For instance, if there was consistent evidence that entities perceive health IT developers to be engaging in information blocking through high prices, that could lead developers to scrutinize their pricing structures and price transparency practices more closely. Additional data on practices that could be considered information blocking could also facilitate measuring changes in perceived behaviors by establishing a baseline against which trends could be estimated as the market adapts to regulatory requirements.

We therefore sought to contribute to the existing literature on the prevalence of practices that may constitute information blocking by examining hospital leaders’ perceptions. Drawing from a national survey of hospital leaders, we sought to describe perception of the prevalence of practices that may constitute information blocking, both overall and by specific practice, and to identify potential drivers of that perception, including the reporting hospitals’ primary health IT developer, characteristics of the hospital’s market, and characteristics of the hospital itself.

DATA AND METHODS

Data

We used data from the American Hospital Association (AHA) IT Supplement Survey fielded between April and September 2021, an annual survey of hospitals on their IT capabilities and experiences. The AHA IT supplement is sent to Chief Information Officers and completed by those individuals or their delegates. In this article, we employ a metonym to describe respondents as “hospitals” to reflect that the responses of these leaders likely reflect the views of their broader organization, and that one response was gathered per hospital. We combined this data with information on hospital characteristics from the 2020 AHA Annual Survey, the most recent year available. Data is available for purchase from the AHA at https://www.ahadata.com/aha-data-resources. This study was determined to be exempt from review by an institutional review board.

Responses to the IT supplement were weighted to reflect the distribution of hospital characteristics in the AHA Annual Survey. Each hospital was assigned a likelihood of responding based on the hospital size, region of the country, ownership type (for-profit, government, nonprofit), teaching status, multihospital system membership status, urban or rural location, and the presence of a cardiac intensive care unit. Weights then comprised the inverse of the probability of responding to better reflect the population of hospitals.

To calculate information on the hospital’s market, we combined information from the AHA annual survey with the 2019 Medicare Hospital Service Area File. This allowed us to identify the ZIP codes in which each hospital treated fee-for-service Medicare patients, a proxy for the hospital’s market.

Perceived information blocking

Hospitals were asked to indicate whether they observed information blocking by various actors. We interpret responses to reflect their perception of practices that may or may not actually constitute information blocking as defined by regulation. Specifically, hospitals were asked whether they observed 4 stakeholders engaging in information blocking: health IT developers (the survey referenced Enterprise EHR Developers and Developers of Certified Health IT, which encompasses most EHR Developers and are one of the actors under the information blocking regulations), healthcare providers, national networks, and state, regional and/or local health information exchanges (the last 2 stakeholders are grouped together and defined as health information exchanges/networks in the information blocking regulations). For the first 2 stakeholders, hospitals were also asked in what forms they had observed or experienced information blocking. For health IT developers, hospitals were asked if they had experienced information blocking via price; contractual language; artificial technical, process or resource barriers; and refusal to exchange patient information (for instance, refusing to exchange patient information with competitors). For healthcare providers, hospitals were asked if they had experienced information blocking through artificial technical, process or resource barriers; refusal; or strategic affiliations. For each item, hospitals could report often/routinely, sometimes, or rarely/never observing or experiencing information blocking. We recoded responses into a binary form comparing hospitals that reported often/routinely or sometimes versus those that reported rarely/never. For each item, we treated hospitals that reported “Don’t Know” as missing data and excluded them from the subanalysis related to that question.

Primary EHR

The AHA survey includes a question that asks hospitals to report their primary EHR, providing 13 different EHR developers, including Other, as options. We categorized primary EHRs into 2 groups based on market share. The first group included 3 EHRs with at least 20% of the market as estimated using our sample after adjusting for survey weights: Epic, Cerner, and Meditech. This group accounted for 80% of hospitals. The second group included all other EHRs, the largest of which accounted for 9% of hospitals. In total, this group accounted for the remaining 20% of hospitals.

Local market characteristics

For each hospital, we calculated the market penetration of for-profit hospitals and multihospital systems within the market served. Prior work has identified relationships between hospital for-profit status, health system membership and engagement in health information exchange, and argued that these relationships were driven by competitive strategy.^1,18,21 We therefore hypothesized that hospitals in markets with greater market penetration of either for-profit status or health systems would be more likely to report perceived information blocking.

To describe market penetration by these 2 organization types, we calculated the proportion of all inpatient days from the ZIP codes of the responding hospital that were treated by for-profit hospitals and multihospital systems while excluding patients treated by the focal hospital or its multihospital system. For example, if a hospital served 2 ZIP codes, and 2 other hospitals, 1 nonprofit and 1 for-profit, provided 20 and 10 patient days of care in one of those ZIP codes, and those same hospitals served the second ZIP code, where they provided 50 and 20 days of patient care, the aggregate for-profit market penetration for ZIP codes served by the responding hospital would be ((10 + 20)/(20 + 10 + 50 + 20)=30%). For ease of reporting, we divided hospitals into groups each consisting of approximately the same number of hospitals—based on the proportion of the local hospital market (excluding the focal hospital and its multihospital system where appropriate) consisting of multihospital systems (Low = 0–76%, Medium = 76–90%, High = 90–100%) or for-profit hospitals (Low = 0–2%, Medium = 2–9%, High = 9–90%).

We calculated 2 metrics to capture issues related to market competition. First, we calculated the Herfindahl-Hirschman index to capture the competition or concentration of the market, with each hospital’s market defined by the ZIP codes in which the hospital’s patients resided, and the market share of each hospital in the market defined by the proportion of all inpatient days from those ZIP codes treated at each hospital (or multihospital health system; Low Concentration = 0.12–0.31, Medium = 0.31–0.40, High = 0.40–0.88). Second, we calculated a measure of market concentration for hospitals’ primary EHR. Given that there are relatively few such EHRs, and in many markets few EHRs are used, we calculated the proportion of the market treated at hospitals using the market-leading EHR (Low = 24–58%, Medium = 58–77%, High = 77–100%).

Hospital characteristics

We included several additional characteristics that we hypothesized would relate to the extent to which hospitals perceived others to engage in information blocking. Those variables included hospital size (0–99 beds; 100–399 beds; 400+ beds), ownership status (nonprofit, government, or for-profit), multihospital system membership, critical access hospital status, and rural versus urban location.

Analytic plan

We first provided the proportion of hospitals that reported that each of 4 entity types engaged in perceived information blocking. We similarly describe the proportion of hospitals that reported perceiving that health IT developers and healthcare providers engaged in various forms of information blocking.

We next described the proportion of hospitals that reported each form of perceived information blocking in terms of the national market share of their primary EHR. For simplicity, we divided the sample into hospitals using one of the 3 market leading primary EHRs and those that use any other primary EHR. Within each group, we report the median and range of reported rates of perceived information blocking and each form of information blocking.

We next assessed the relationship between market conditions and perceptions of healthcare provider information blocking using a multivariable Poisson regression model with heteroskedastic robust standard errors. We examined the association between perceived information blocking by healthcare providers and the market share of health system hospitals, the market share of for-profit hospitals, the overall market concentration, hospital size, ownership status, multihospital system membership, critical access status, and urban/rural location. In a robustness test, we included the primary EHR each hospital used as an additional covariate to test whether the relationship between hospital characteristics and reported rates of perceived information blocking by healthcare providers depended on the hospital’s primary EHR.

Finally, we examined the relationship between market characteristics, hospital characteristics, and perceptions of information blocking by health IT developers. Specifically, we estimated the association between perceived information blocking by health IT developers and the market share of the market-leading primary EHR as well as the hospital characteristics listed above using a multivariable Poisson regression to estimate the independent relative risk of perceived information blocking by health IT developers based on market and hospital characteristics while holding other characteristics constant.

RESULTS

The analytic sample comprised 2092 hospitals in the United States that responded to at least 1 survey item related to their perception of information blocking. This represented 88% of hospitals that responded to any part of the AHA IT Supplement and 47% of all nonfederal, acute care hospitals in the United States. After weighting, respondents remained more likely to be large hospitals than nonrespondents (weighted 11.7% of respondents vs 5.5% of nonrespondents) and to be members of multihospital systems (69.0% of respondents vs 52.7% of nonrespondents). Respondents were less likely to be small hospitals (14.8% of respondents vs 24.1% of nonrespondents).

Perceived prevalence of information blocking

Overall, 42% of responding hospitals indicated perceiving that at least 1 of the 4 actors (health IT developers, healthcare providers, national networks, and state, regional and/or local health information exchanges) either sometimes or often engaged in information blocking. About 22% of hospitals indicated perceiving that more than 1 of the 4 actors engaged in information blocking. The largest number of hospitals, representing 36% of respondents, indicated perceiving that healthcare providers engaged in information blocking (Figure 1).

Figure 1.

Percent of responding hospital that perceived each actor sometimes or often engaging in information blocking. Note: Hospitals that did not respond or indicated “don’t know” to each question are excluded from the numerator and denominator of each measure.

Hospitals perceived that healthcare providers and health IT developers engaged in specific methods of information blocking at varied rates. A substantial proportion of hospitals—over one-quarter—reported that healthcare providers engaged in perceived information blocking through each of the 3 specific methods included on the survey (Figure 2). Overall, 43% of hospitals indicated that providers engaged in information blocking through at least 1 specific method and 19% indicated that providers engaged in information blocking through all 3 listed specific methods. Fewer hospitals reported that health IT developers engaged in perceived information blocking, with just 10% perceiving that health IT developers engaged in information blocking through refusal to exchange patient information, compared to 26% of healthcare providers allegedly engaging in information blocking by the same method. Overall, 37% of hospitals indicated that health IT developers engaged in 1 of the 4 listed specific method of information blocking, with 11% of hospitals reporting health IT developers engaged in 3 or more methods.

Figure 2.

Percent of responding hospital that perceived developers and other healthcare providers engaging in varied methods of information blocking. Note: Response rates vary from 1875 to 1937. Each figure includes the unweighted N and the weighted proportion of respondents after adjusting for nonresponse. The figure indicates the % of respondents reporting observing each actor “Often” or “Sometimes” engages in each method. Hospitals that did not respond or indicated don’t know to each question are excluded from each measure.

Relationship between developers and perceived information blocking

Hospitals using market-leading EHRs were substantially less likely to report that they perceived health IT developers overall engaged in each form of information blocking than were hospitals using other developers (Figure 3). Fourteen percent of hospitals using one of the market-leading EHRs perceived that health IT developers engaged in information blocking compared to 32% of hospitals that used nonmarket leading EHRs. This trend persisted across the 4 specific forms of potential information blocking asked about in the survey. Rates of perceived information blocking across EHR within each market share group were relatively consistent with the exception of price, on which 42% of hospitals using one market-leading EHR reported perceived information blocking through price compared with 23% of hospitals using any of the 3 market leading EHRs.

Figure 3.

Rates of reported information blocking by developer market share and method. Note: Sample size varies due to nonresponses to individual questions. Eighty percent of hospitals in the data use one of the top 3 EHRs while 20% use another EHR. Only EHRs that were used by at least 1% of hospitals are included in the reported EHR-specific range. Differences between weighted percentage are statistically significant at P < .05 for all rows. ^†To assess overall information blocking relevant to other actors, the survey asks “To what extent have you observed the following stakeholders engaging in information blocking behaviors?” and one response category is “Certified Health IT Developers (eg, Enterprise EHR vendors)”. For the other 4 items, the survey asks, “In what form(s) have you observed or experienced information blocking by Enterprise EHR vendor(s)?” The different description of the entity may explain why reported rates of specific methods of information blocking are higher than overall reported rates of information blocking.

Association between market and hospital characteristics and perceived healthcare provider information blocking

In multivariate analyses, hospitals in markets with medium or high health system penetration were 16–8% more likely to report perceived information blocking by health care providers (Relative Risk = 1.16; 95% CI=1.00–1.35) (RR = 1.18; 95% CI = 1.01–1.38; Table 1). For-profit hospitals were 196 percent more likely to report perceived information blocking by healthcare providers (RR = 2.96; 95% CI = 2.61–3.37). Hospitals that were members of multihospital systems were somewhat less likely to report perceived information blocking, though this relationship was not statistically significant at conventional thresholds (RR = 0.85; 95% CI = 0.71–1.01). These associations remained stable in a robustness test that included the primary EHR used by each hospital (Supplementary AppendixTable S1).

Table 1.

Predictors of hospitals reporting healthcare providers engaged in information blocking

Variables	Likelihood hospital reported perceiving other healthcare providers often or sometimes engaged in information blocking
	Relative risk	95% CI
Health system market share other than reporting hospital system (ref: low)
Medium	1.16	(1.00–1.35)
High	1.18	(1.01–1.38)
For profit market share other than reporting hospital system (ref: low)
Medium	1.10	(0.95–1.28)
High	1.03	(0.88–1.20)
Market concentration (ref: low)
Medium	1.00	(0.85–1.18)
High	1.04	(0.88–1.22)
Hospital size (ref: small)
Medium	1.10	(0.93–1.29)
Large	1.17	(0.96–1.43)
Ownership status (ref: nonprofit)
Government	1.11	(0.90–1.37)
For-profit	2.96	(2.61–3.37)
Multihospital system member (Ref: independent hospital)	0.85	(0.71–1.01)
Critical access hospital (Ref: noncritical access hospital)	1.14	(0.91–1.42)
Rural (Ref: urban)	0.94	(0.78–1.12)
Constant	0.28	(0.19–0.39)
Observations	1949

Note: Confidence intervals calculated using heteroskedastic-robust standard errors.

Abbreviation: Ref: reference category for each categorical variable.

Association between market and hospital characteristics and perceived health IT developer information blocking

Several market and hospital characteristics were related to the proportion of hospitals that perceived information blocking by health IT developers (Table 2). Hospitals in markets where the market leading developer held a medium or large share of the hospital market were substantially less likely to report information blocking (RR = 0.73; 95% CI: 0.57–0.94; RR = 0.73; 95% CI: 0.55–0.96) than hospitals in more competitive markets (ie, where the leading developer was less dominant). Hospitals in multihospital systems were less likely to report that developers engaged in perceived information blocking (RR = 0.62; 95% CI: 0.49–0.79). There was also a substantial relationship between hospital size (RR = 0.71; 95% CI: 0.48–1.05) and perceived information blocking and rural location (RR = 0.80; 95% CI: 0.61–1.04) and perceived information blocking, though these did not meet traditional thresholds of statistical significance.

Table 2.

Predictors of hospitals reporting health IT developers engaged in information blocking

Variables	Likelihood hospital reported perceiving developers often or sometimes engage in information blocking
	Relative risk	95% CI
Local market leading EHR market share (ref: low)
Medium	0.73	(0.57–0.94)
High	0.73	(0.55–0.96)
Hospital size (ref: small)
Medium	0.81	(0.61–1.09)
Large	0.71	(0.48–1.05)
Ownership status (ref: nonprofit)
Government	1.05	(0.80–1.37)
For-profit	1.13	(0.79–1.62)
Multihospital system member (Ref: independent hospital)	0.62	(0.49–0.79)
Critical access hospital (Ref: noncritical access hospital)	1.16	(0.87–1.56)
Rural (Ref: urban)	0.80	(0.61–1.04)
Constant	0.32	(0.24–0.45)
Observations	1948

Note: Confidence intervals calculated using heteroskedastic-robust standard errors.

Abbreviation: Ref: Reference category for each categorical variable.

DISCUSSION

We report on a national survey of hospitals fielded just following the applicability date of the information blocking provisions of the Cures Act Final Rule. In total, 42% of hospitals reported that one or more entities engaged in practices that they perceived to constitute information blocking, indicating sharing of EHI may be impeded by these practices and potentially negatively impacting clinical care. Potential information blocking practices can also undermine efforts to increase care efficiency and slow the growth of innovative technologies by introducing additional friction into the nation’s health IT infrastructure.²² These data indicate the importance of continued stakeholder education on the information blocking regulations to facilitate compliance and continued collaborative efforts to reduce barriers to exchanging health information.

Notably, hospitals most often identified other healthcare providers as engaging in practices that may constitute information blocking. This finding contrasts with evidence from 2 surveys of health information exchanges, conducted in 2015 and 2019, that found that health IT developers were substantially more likely to engage in practices that may constitute information blocking than were health care providers.^5,15

There are 2 likely explanations for this discrepancy. One explanation is that health IT developers have improved their information sharing behaviors or altered contractual terms to better allow information sharing in recent years, perhaps in anticipation of information blocking regulation and the continued policy focus on enabling data sharing. Another explanation is that perceptions of information blocking are related to competitive dynamics of different stakeholders. For example, our hospital respondents were more likely than previous health information exchange respondents to perceive information blocking by other providers, perhaps because hospitals more directly deal with healthcare providers in circumstances where competitive dynamics (eg, competition for market share) may be at play. A parallel dynamic may exist in data from claims submitted to ONC’s Report Information Blocking Portal, in which the plurality of claims appeared to have been submitted by patients, and the plurality of claims cited health care providers as possibly blocking information.¹⁶ Because patients engage more often with health care providers (a category of information blocking “actor” that includes hospitals) than with health IT developers or health information exchanges, they may be more likely to perceive health care providers (including hospitals) as information blocking. In contrast, health information exchanges work to provide services for healthcare providers—and many health information exchanges have achieved sustainability through the support of hospitals that are inclined to share information—but can perceive health IT developers as competitors or impediments to growth.²³

Nearly 20 percent of hospitals indicated that health IT developers had engaged in perceived information blocking. Though lower than reported perceived information blocking by healthcare providers, these data indicate that practices that “feel” like information blocking by health IT developers are not inconsequential, and this was especially the case for hospitals not using market leading EHRs. These hospitals were substantially more likely to report that health IT developers engaged in information blocking. This is consistent with work that has shown that physicians using EHRs with lower market share were less likely to engage in interoperability.^24,25 Developers with fewer customers likely need to spread the cost of enabling exchange over a smaller customer base and may therefore seek to minimize additional work to enable interoperability or to more aggressively maximize revenue for that work. It is also possible that this correlation is driven by the practices of larger developers used by other organizations, which may not facilitate exchange with smaller developers. We sought to evaluate this possibility in post hoc analysis based on whether the hospital used the most dominant EHR in their local market. We measured whether hospitals that used an EHR other than the dominant one in their local market were more likely to report that developers engaged in practices that may constitute information blocking than hospitals using the local market leading EHR. After controlling for whether the hospital used a national market-leading EHR or did not, we found no difference in reported information blocking based on whether the hospital used the local market leading EHR (see Supplementary AppendixTables S2). This contrasts with prior evidence that indicates that hospitals using local market-leading EHR reported greater information exchange than other hospitals in their market.²⁶

In multivariable analysis, for-profit hospitals were almost 3 times as likely to report experiencing information blocking from other healthcare providers than were nonprofit hospitals. We speculate that health care providers may have been reluctant to share information with for-profit hospitals because of concerns about their approach to practice or competitive orientation.^27,28 One possibility is that for-profit hospitals are more likely to report perceived information blocking because their more aggressive approach towards obtaining information from other entities. Notably, the share of the market dominated by for-profit hospitals was not correlated with rates of perceived information blocking, which might indicate that our finding is not driven by the information sharing (ie, sending or making available) related behavior of for-profit hospitals.

Limitations

As defined in the 21st Century Cures Act and implementing regulations, practices are determined to constitute information blocking through investigation by the U.S. Department of Health and Human Services (HHS), particularly the HHS Office of Inspector General. Throughout, we have been careful to use the phrase “practices that may constitute information blocking” because hospital reports in a survey do not necessarily reflect instances of information blocking. Ultimately, whether information blocking occurred would depend on the full facts and circumstances of a specific case. For example, it is possible that respondents may conflate the inability to provide information through a specific format or interoperability network as information blocking, even when that behavior may or may not meet the regulatory definition. Finally, our data is cross sectional, intended to highlight prevalence and correlates, and should not be interpreted as directly supporting causal relationships.

CONCLUSION

Although there is concern among some providers regarding the transition to all EHI being in scope for assessment of information blocking, national survey data indicates that a substantial proportion of hospitals feel they have experienced practices that may constitute information blocking, highlighting the potential value of the regulation in promoting information exchange to support patient care. Monitoring trends in these perceptions could indicate whether the information blocking regulations are reducing perceptions of information blocking practices that may interfere with the access, exchange, or use of EHI.

FUNDING

This study was not supported by external funding.

AUTHOR CONTRIBUTIONS

JE and VP conceived of the study. JE performed data analysis and drafted the manuscript. DH and VP performed critical revisions. All authors reviewed the results and approved the final version of the manuscript.

SUPPLEMENTARY MATERIAL

Supplementary material is available at Journal of the American Medical Informatics Association online.

Conflict of interest statement

None declared.

DATA AVAILABILITY

The AHA data used in this study is available for purchase from the AHA at https://www.ahadata.com/aha-data-resources.