Measuring Ambulatory Racial and Ethnic Neurologic Disparities With the Axon Registry

Abstract

Background and Objectives

The primary objective is to examine potential racial and ethnic (R/E) disparities in ambulatory neurology quality measures within the American Academy of Neurology Axon Registry. R/E disparities in neurologic US morbidity and mortality have been clearly documented. Despite these findings, there have been no nationwide examinations of how ambulatory neurologic care affects these negative health outcomes.

Methods

This was a retrospective nonrandomized cohort study of patients in the AAN Axon Registry. The Axon Registry is a neurology-specific outpatient quality registry that collects, reports, and analyzes real-world deidentified electronic health record (EHR) data. Patients were included in the study if they contributed toward one of the selected quality measures for multiple sclerosis, epilepsy, Parkinson disease, or headache during the study period of January 1, 2019–December 31, 2019. Descriptive analyses of patient demographics were performed and then stratified by race and ethnicity.

Results

There were a total of 633,672 patients included in these analyses. Separate analyses were performed for race (64% White, 8% Black, 1% Asian, and 27% unknown) and ethnicity (52% not Hispanic, 5% Hispanic, and 43% unknown). The mean age ranged from 18 to 55 years, with 61% female and 39% male. Quality measures were chosen based on completeness of R/E data and were either process or outcomes focused. Statistically significant differences were noted after controlling for multiple comparisons.

Discussion

The large proportion of missing or unknown R/E data and low overall rate of performance on these quality measures made the relevance of small differences difficult to determine. This analysis demonstrates the feasibility of using the Axon Registry to assess neurologic disparities in outpatient care. More education and training are required on the accurate capture of R/E data in the EHR.

Neurologic diseases are common in the United States and are a major cause of morbidity, mortality, and health inequities.^1,2 Despite these findings, there have been no nationwide examinations of how ambulatory neurologic care affects these negative health outcomes. By far, stroke carries the highest disparate burden in negative health outcomes by race-ethnicity.³ However, many other neurologic diseases, such as traumatic brain injury, epilepsy, multiple sclerosis (MS), Alzheimer disease, Parkinson disease (PD), and pain syndromes, affect the lives and well-being of many Americans. Appropriately trained neurologic care providers are paramount to optimizing medical treatment and care for all across the spectrum of race and ethnicity. Measuring the ability of neurologists to implement quality measures at the point of care, without creating or exacerbating disparities, is critical.

In 2015, the American Academy of Neurology Institute (AAN) created the Axon Registry to leverage real-world patient data to help neurologists and neurologic care providers improve diagnosis and treatment. The Axon Registry contains curated and validated ambulatory neurology quality performance data.^4-7 The initial analysis of US neurologic quality of care using the Axon Registry data found performance variability across providers and measures and intrinsic characteristics of measure design that affected measure performance.⁸

Large clinical quality databases like the Axon Registry are well positioned to examine many types of disparities in the United States. The specific goals of this work were to examine the utility of the Axon Registry in identifying racial and ethnic (R/E) disparities in the provision of ambulatory neurologic care in the United States and describe R/E disparities in quality measure performance.

Methods

Source: 2019 Axon Registry Quality Measures

This is a retrospective, nonrandomized cross-sectional study of patients in the American Academy of Neurology Axon Registry, which is a neurology-specific patient registry that collects, reports, and analyzes real-world deidentified electronic health record (EHR) data. In 2019, there were 49 quality measures in the Axon Registry. Because the goal of this analysis was to evaluate quality performance based on race and ethnicity, we reviewed all quality measures for completeness of patient race and ethnicity data and selected measures for MS, epilepsy, PD, and headache based on a 50% threshold for completeness of race and ethnicity data and commonality of these diseases. Patients were included in the study if they contributed toward one of the selected quality measures.

Data Availability

The underlying patient or provider identifiable data submitted to the Axon Registry for health care operations in the normal course of clinical care are not available due to privacy and contractual restrictions. The aggregate, deidentified data used for this study are available on request from authorized investigators. The Axon Registry measure glossary can be found online.⁹

Analyses

Descriptive statistics of demographic information were conducted on patients who contributed to at least 1 quality measure for any of the 4 neurologic conditions chosen (MS, epilepsy, PD, and headache). Relevant characteristics included sex, birth year, census region, payer information, race, and ethnicity. We reported these values by including number and percentages for count variables and mean and SD or medians and quartiles for continuous variables, as appropriate. We also report the percent provider performance for the specific quality measures.

Unadjusted comparisons across race and ethnic groups were conducted using the patient population described above. To enable an analysis of the individual quality measures across both race and ethnicity, patients with incomplete or unknown sex, age, patient ID number, race, or ethnicity information were excluded from the study cohort (n = 1,488,388).

To assess whether there were significant differences in completion of the quality measures by race and ethnicity, patients were first stratified by race and then by ethnicity. The analysis was performed in 2 steps for each stratification. First, a χ² analysis was run for each quality measure to determine whether there was a significant difference between all race or ethnicity groups (p value = 4.6e-3). Second, if a difference was found, a pairwise comparison between race (or ethnicity) categories within that quality measure was performed using a χ² analysis. Race or ethnicity pairs that showed a significant difference for each measure were reported. Significance was defined as a p value of 1.0e-3 using a Bonferroni correction for multiple comparisons (0.05/49 hypotheses). Statistical analysis was performed using Python 3.7 and the SciPy Stats package version 1.19.5.

Standard Protocol Approvals, Registrations, and Patient Consents

Patient and provider data are collected, used, and secured in a lawful manner by the Axon Registry to assist with the health care operations of the participants. The Privacy Statement for AAN-Generated Axon Registry Publications is available online.¹⁰ Only deidentified data derived from the Axon Registry were accessed and analyzed for this study. This secondary data analysis of deidentified data is exempt from independent review board review.

Results

Quality and performance measurements for each of the 4 neurologic disease areas are described in Table 1. As shown in Table 2, 633,672 patients were included in this analysis, the majority of whom were White (63.7%). Black (8.0%) and Asian (1.4%) individuals represented a minority of participants. As summarized in Table 3, 4.7% of patients were identified as Hispanic. Race was categorized as unknown in approximately 26.8% of patients in the Axon Registry, and ethnicity was unknown in 43%. Slightly over 61% of Axon Registry patients were female, and a plurality of patients were aged 65 years or older (43.6%). The geographic analysis revealed the greatest number of patients in the South region (37.7%), with the lowest number from the West region (3.7%). Commercial insurance was the most common type of coverage at 30.4%, roughly equal to the sum of patients with Medicare or Medicare Advantage. Medicaid participants represented 6.7%, and uninsured were 0.2%. Full demographic data are presented in Tables 2 and 3, with supplementary tables displaying color-coded disease-specific quality performance measure trends by race in eAppendix 1 and subdivided by neurologic diagnosis in eTables 1B–1E and 2B–2E, links.lww.com/CPJ/A401.

Table 1.

Description of 2019 Axon Registry Quality and Performance Measures

Table 2.

Baseline Characteristics of Patients in the Axon Registry, Stratified by Race

Table 3.

Baseline Characteristics of Patients in the Axon Registry, Stratified by Ethnicity

The analysis of pairwise comparisons across race and ethnicity categories revealed a trend toward statistical significance by race (Table 4) and ethnicity (Table 5). However, the number of total patients available by race and ethnicity for analysis varied widely within and between measures. Race and ethnicity were captured for more than 50% of patients for all measures analyzed. Overall numbers of Asian and Hispanic patients were very small compared with White and Black patients; therefore, it was difficult to draw conclusions for those races and ethnicity. Between White and Black patients, there were trends to suggest that Black patients received lower-quality care in headache and some quality measures of PD, but overall, the effect sizes were small (Table 4). As shown in Tables 4 and 5, only 2 quality measures were performed at a rate greater than 50% (Axon 23 and Axon 13). Other quality metrics were performed less than 25% of the time across all races/ethnicity. The only ethnicity quality measure analysis that revealed a trend toward statistical significance was Quality Payment Program (QPP) 419, overuse of imaging in headache (Table 5). White patients were less likely to receive imaging for primary headache (favorable). No other statistically significant differences were identified in quality measure utilization by ethnicity.

Table 4.

Disease Quality Measures in the Axon Registry, Stratified by Race

Table 5.

Disease Quality Measures in the Axon Registry, Stratified by Ethnicity

Discussion

In this first analysis of the Axon Registry focused on neurologic health disparities, we examined race and ethnicity data extracted from the EHR to assess for quality measure differences based on the long-standing negative effect of these 2 complex social constructs on care and health outcomes. We found substantial levels of EHR data missingness related to patient race and ethnicity, overall modest or low provider performance scores for multiple measures across all groups, and modest differences in the provision of care for several comparisons of performance among different R/E groups. The clinical importance of these small differences in provision of care could not be determined. This analysis demonstrates the feasibility of using real-world quality metrics to assess disparities in care and corroborates previous work demonstrating high levels of race and ethnicity data missingness in the EHR.

As has been noted in analyses of other real-world databases,¹¹ the significant amount of race and ethnicity data missingness in our analysis places considerable constraints on our insights into US neurologic disparities. We found that race or ethnicity was not reported in 46% of encounters, which may be a result of confusion between the 2 fields at the point of data entry or lack of consistent data capture at the point of care. Although not a specific focus of our analysis, other patient characteristics such as sexual orientation, gender identity, and related social determinants of health are also inconsistently documented in the electronic medical record.¹² There is a clear need to improve consistency and accuracy of recording these data for reliable analysis of neurologic disparities, which itself is a prerequisite for interventions to address those disparities.

The overall low rates of provider performance on quality measures mirror findings from a recent global cross-sectional analysis of Axon Registry data.⁸ Although there were some differences in this analysis compared with that report, likely attributable to differences in methodology, the overall similarity is unsurprising given the examination in both studies of a 2019 cross-section of Axon Registry performance and use of similar measures. These findings underscore the broad opportunity to improve provision of ambulatory neurologic quality in the United States.

In studies such as ours that aim to identify, measure, and understand disparities in neurologic care, it is crucial to ensure accurate and reliable information regarding race and ethnicity in data sources. Unfortunately, this information may be missing or incorrect in the medical record. Information on race and ethnicity may be collected in various ways: self-report during new patient encounters, abstracted from the medical record of a previous encounter, or entered by office staff based on physical appearance. The complexities in defining race and ethnicity can make capturing and measuring them difficult. Perceptions of others are often incorrect or inconsistent, and the interactions between self-identification and social identification of race and ethnicity are complex in their effects on health care. One study found that being classified by others as White was associated with significant advantages in health Status, regardless of how the individual self-identified.

Self-reporting of race and ethnicity is the recommended manner to collect this information.¹⁴ Automated data collection would benefit from additional quality control measures to verify race and ethnicity information. Examples include manual review of the data or comparison of automated data in patients with multiple encounters. In addition, there are large unmet opportunities to consistently and uniformly gather other data elements that would inform important assessments of neurologic disparities, including sexual orientation, gender identity, socioeconomic data, and others.

There is a critical need for education and tools supporting the accurate capture of race and ethnicity data in the EHR. Stakeholders including specialty societies, patient advocacy organizations, compliance agencies, and EHR vendors should support the development of educational materials for staff, families, and patients, in addition to training modules on the reasons and need to collect this information. There is a need for a uniform process and clear practice workflow with clear accountability for staff training and addressing patient and family concerns. EHR vendors should establish comprehensive and standardized choices for race and ethnicity recording.

There are limitations to this work. Extensive race and ethnicity data missingness in quality measures limited our ability to identify differences in ambulatory neurologic quality of care. The high levels of missingness in race and ethnicity identification may mask meaningful disparities in provisions of care. Other work has demonstrated that disparities exist in specific disease categories,¹⁵ and this analysis took a narrowly scoped view of neurologic disparities based on the available variables of interest (primarily race and ethnicity). Axon Registry participants represent a relatively small proportion of patients from providers delivering neurologic care in the United States and therefore may not be representative of how most patients are treated. Also, because these data were deidentified, we were not able to assess whether regional or practice-level variances contributed to overall performance scores or data missingness. The disparities identified in this analysis presume that missing data would have been equally distributed among identified race and ethnicity variables, which may not have been the case. A more definitive understanding of disparities will require more completeness of data. Another limitation in these analyses is the low number of Medicaid and uninsured patients. These populations are known to be at high risk for health disparities.

Future analyses could consider examining the trends in measure performance identified in this analysis, specifically potential variables that could account for group differences. Examples could include geographic, payer, or other factors that may be associated with differences in performance across different race or ethnic categories. A deeper understanding of the factors associated with disparities in care is a prerequisite for developing strategies to close disparities in care at the patient, practice, community, and national level.

This analysis demonstrates the feasibility of using the Axon Registry quality measures to assess neurologic disparities in care. These results from a real-world clinical quality database demonstrate considerable data missingness and highlight the importance of accurately capturing race, ethnicity, and other patient variables to facilitate future research and closure of gaps in care.

Appendix. Authors

Study Funding

The authors report no targeted funding.

Disclosure

R.T. Benson serves as a voluntary member of the AAN Registry Subcommittee and has nothing to disclose. The views expressed are the authors' own and do not necessarily reflect those of the National Institutes of Health, the Department of Health and Human Services, or the United States Government. S.M. Benish serves as voluntary Chair of the AAN Registry Subcommittee and Member of the AAN Quality Committee and on the AAN Board of Directors and has nothing to disclose. G.J. Esper has received personal compensation in the range of $500–$4,999 for serving as a consultant for NeuroOne Technology Corporation, has received personal compensation in the range of $500–$4,999 for serving as a consultant for Pfizer, has received personal compensation in the range of $500–$4,999 for serving as an Expert Witness for Mitchell Law Group, and reports no other disclosures that are relevant to this manuscript. The institution of B.M. Kissela has received research support from the NIH/NINDS and NCATS. B.M. Kissela reports no other disclosures that are relevant to this manuscript. N. Rosendale has received personal compensation in the range of $500–$4,999 for serving as an Editor Associate Editor or Editorial Advisory Board Member for Neurology. The institution of N. Rosendale has received research support from the UCSF Academy of Medical Educators. N. Rosendale reports no other disclosures that are relevant to this manuscript. E.T. Marulanda-Londono and O.A. Hope have nothing to disclose. T.T.A. Pham is a member of the AAN Registry Subcommittee. T.T.A. Pham has nothing to disclose. M. Roe, A. Torres, A. Lien, and S. Kauwe are employees of Verana Health and have nothing to disclose. K.B. Lundgren and A. Mante are employees of the American Academy and have nothing to disclose. B. Schierman is an employee of the American Academy and has nothing to disclose. L.K. Jones has received publishing royalties from a publication relating to health care and serves as a voluntary member of the Board of Directors with the Mayo Clinic ACO and American Academy of Neurology Institute. L.K. Jones reports no other disclosures that are relevant to this manuscript. Full disclosure form information provided by the authors is available with the full text of this article at Neurology.org/cp.