Inaccuracies in provider directories persist for long periods of time

Abstract

A growing literature has identified substantial inaccuracies in consumer-facing provider directories, but it is unclear how long these inaccuracies persist. We re-surveyed inaccurately listed Pennsylvania providers (n = 5170) between 117 to 280 days after a previous secret-shopper survey. Overall, 19.0% (n = 983) of provider directory listings that had been identified as inaccurate were subsequently removed, 44.8% (n = 2316) of provider listings continued to show at least 1 inaccuracy, and 11.6% (n = 600) were accurate at follow-up. We were unable to reach 24.6% (n = 1271) of providers. Longer passage of time was associated with reductions in directory inaccuracies, particularly related to contact information, and to a lesser degree, with removal of inaccurate listings. We found substantial differences in corrective action by carrier. Together, these findings suggest persistent barriers to maintaining and updating provider directories, with implications for how well these tools can help consumers select health plans and access care.

Introduction

Provider directories, which consumers use to select and contact in-network clinicians, have been shown to contain high rates of inaccuracies.^1-5 These inaccuracies may contribute to delayed care as well as health inequities,^6,7 and ultimately challenge network adequacy regulations.^8-10 Regulators have sought to impose requirements such as the No Surprises Act of 2021 upon carriers to keep their directories verified and frequently updated as a result.^8,11-13 The Act imposes requirements for insurers to verify and update their provider directories at least every 90 days.

The extent to which insurers have responded to these policies, and how long directory inaccuracies persist, is unknown. To assess the persistence of provider directory inaccuracies, we fielded a secret-shopper survey in the Pennsylvania Affordable Care Act (ACA) Marketplace. The survey sought to re-contact 5170 providers a minimum of 117 days after a previous secret-shopper survey identified directory inaccuracies. (The results from this survey have not been published at this point.) We found that 19.0% (n = 983) of the providers we searched for in directories had been removed at follow-up. In addition, 44.8% (n = 2316) of providers continued to show at least 1 inaccuracy, while 11.6% (n = 600) were now listed accurately. We were unable to reach 24.6% (n = 1271) of providers (see Figure S1).

Data and methods

We fielded a follow-up secret-shopper survey in the Pennsylvania ACA Marketplace, which, as of 2023, is served by 8 private carriers, including local plans as well as national and regional ones serving multiple markets. In a previous secret-shopper survey fielded from March 30 to August 31, 2023, we identified 5453 providers who were listed inaccurately due to the following: (1) errors with contact information issues (eg, provider not working at the number listed, phone line being disconnected), (2) being erroneously listed as in-network, or (3) being listed under the wrong specialty. These categories are in line with previous research on provider directory inaccuracies.^1-3,5,14 The original survey included a variety of specialties for adults (cardiology, dermatology, endocrinology, gastroenterology, neurology, obstetrics-gynecology [OB/GYN], primary care, psychiatry, psychology) and children (general pediatrics and pediatric mental health services from psychiatrists, psychologists). Of the 5453 providers identified as having directory inaccuracies, we were able to re-survey 5170 randomly chosen providers from December 11, 2023, to January 8, 2024 (see Figure S2). (We were not able to survey all providers due to budget constraints.) The subsequent contact occurred between 117 and 280 days after the initial survey contact.

The re-survey process was conducted by randomly assigning a provider to 1 of several research assistants. A given research assistant searched for the provider in the online directory of the respective carrier. If the research assistant was unable to find a listing (n = 983), the research assistant marked the listing as removed. If a provider was found in the directory, the research assistant proceeded to make contact at the listed number, taking on the scripted role of an individual seeking an appointment for a family member or friend (n = 4187). Callers were randomly assigned information such as caller name, patient name, patient gender, patient date of birth, and patient medical need. Callers verified whether provider directory information was correct in terms of network participation, specialty, and contact information. In order to maintain their disguise as real shoppers, callers terminated calls once the first inaccuracy was identified. No actual appointments were scheduled. Callers successfully contacted 2916 providers and could not verify information for an additional 1271 listings. (Callers were unable to verify information because they only reached answering machines, were put on hold for more than 5 minutes, or the provider's office refused to provide any information.)

In order to assess differences between carriers and specialties, we conducted tests of proportion to assess whether (1) a provider had been removed from the directory, (2) whether there was an inaccuracy for a provider, and (3) whether a provider was verified as accurate. We also assessed differences by carrier and specialty for the 3 categories of inaccuracies (contact information issues, inaccurate specialty listings, provider was out-of-network). We utilized t tests to assess differences in the mean number of days passed for persistent inaccuracies. Last, we estimated linear probability models to assess the association between the passage of time and our outcomes of interest.

Results

Provider removal

Of the 5170 providers we sought to verify, 983 providers (19.0%) had been removed from the provider directory by the time of the second contact. Importantly, we found substantial differences by carrier (Figure 1 and Tables S1 and S2). For example, whereas Carrier C removed 35.1% of the previously inaccurately listed providers (n = 342/975), this was only the case for 8.9% of providers (n = 41/462) for Carrier B (P < .001). There were more modest differences in provider directory accuracy across specialties (Figure 1 and Tables S3 and S4). The percentage of removed provider listings ranged from a low of 13.0% in dermatology (n = 32/247) to a high of 22.7% for OB/GYN (n = 86/379) (P < .003).

Figure 1.

Percentage of providers removed, unable to verify, or verified as accurate or inaccurate, overall and by carrier and specialty. Source: Data based on a survey of 5170 providers. Of these, 4187 were listed in the online provider directories of 8 Pennsylvania Affordable Care Act (ACA) Marketplace carriers. Data collection was confined to the ACA market. The survey was conducted from December 11, 2023, to January 8, 2024. Adult specialties included were cardiology, dermatology, endocrinology, gastroenterology, neurology, obstetrics-gynecology [OBGYN], primary care, psychiatry, and psychology. We also surveyed pediatrics and pediatric mental health services from psychiatrists and psychologists.

Providers listed inaccurately

We were able to successfully contact 2916 of the 5170 providers (56.4%) for whom we attempted to verify provider directory information. Of these 5170 providers, 2316 (44.8%) were listed with at least 1 inaccuracy at the time of our second survey. We again identified substantial differences across carriers (see Figure 1 and Tables S5–S8), with rates of inaccuracies ranging from 35.0% (n = 229/654) for Carrier G to 53.0% (n = 245/462) for Carrier D (P < .002). Inaccuracies related to contact information were most common (n = 1860, 36.0%). These errors ranged from a low of 26.7% (n = 60/225) for Carrier F to a high of 45.0% (n = 208/462) for Carrier B (P < .001). Incorrect specialties listed were identified for 9.1% of contacted providers (n = 472), ranging from a low of 6.3% (n = 41/654) for Carrier G to a high of 11.3% (n = 26/230) for Carrier A (P < .002). Last, 3.7% (n = 189) of contacted providers were out-of-network, ranging from 1.7% (n = 15/909) for Carrier E to 6.7% (n = 80/1202) for Carrier D (P < .001).

Rates of inaccuracies also varied by specialty (see Figure 1 and Tables S9–S12), with a low of 38.2% (n = 190/497) in pediatric psychiatry to a high of 52.2% (n = 204/391) for pediatrics (P < .001). Errors in provider contact information ranged from 29.1% (n = 127/436) for primary care to 42.7% (n = 150/351) for gastroenterology (P < .001), whereas inaccurate specialty listings ranged from 6.4% (n = 32/497) for pediatric psychiatry to 14.4% (n = 63/436) for primary care providers (P < .001). Last, cardiologists were least likely to be inaccurately listed as in-network (n = 5/329, 1.5%) as compared to 6.9% (n = 17/247) of dermatologists (P < .002).

Among those providers who continued to be listed inaccurately, the mean number of days between survey contacts was 190 (median: 189) days, with a range of 117 to 279 days (Figure 2 and Tables S13 and S14). For individual carriers, the mean number of days ranged from a low of 189 days for Carrier D to a high of 194 days for Carrier A (P < .270). In terms of specialties (Figure 2 and Tables S15 and S16), differences in means ranged from 183 days for child psychology to 194 days for gastroenterology (P < .005).

Figure 2.

Number of days between survey contacts for providers with persistent inaccuracies, by carrier and specialty. Source: Data based on a survey of 5170 providers. Of these, 4187 were listed in the online provider directories of 8 Pennsylvania Affordable Care Act (ACA) Marketplace carriers. Data collection was confined to the ACA market. The survey was conducted from December 11, 2023, to January 8, 2024. Adult specialties included were cardiology, dermatology, endocrinology, gastroenterology, neurology, obstetrics-gynecology [OBGYN], primary care, psychiatry, and psychology. We also surveyed pediatrics and pediatric mental health services from psychiatrists and psychologists.

Providers listed accurately

Of the 5170 providers we sought to re-survey, 600 (11.6%) were subsequently listed accurately with regard to contact information, specialty, and in-network status. The percentage of accurately listed providers (see Figure 1 and Table S17) ranged from a low of 9.0% (n = 46/513) for Carrier H to a high of 13.3% (n = 87/654) for Carrier G (P < .022). Last, between 6.8% (psychology [n = 38/558]) and 18.2% (dermatology [n = 45/247]) of providers (see Figure 1 and Table S18) were verified as accurately listed (P < .001).

Association between passage of time and outcomes

Last, we estimated 7 distinct linear probability models with our various outcomes serving as the dependent variables. Our primary independent variable was the passage of time (in hundreds of days) between the 2 survey contacts. We also controlled for carrier and specialty effects (see Table S19). We found that, for every 100 days between contacts, the probability of removal of listings previously identified as inaccurate increased modestly by 0.035 (P < .017). At the same time, the probability of finding at least 1 inaccuracy or inaccurate contact information decreased by 0.045 (P < .017) and 0.037 (P < .040), respectively. The models with statistically significant findings also demonstrated substantial carrier effects.

Discussion

Inaccuracies in provider directories appear to persist for a substantial period of time across all carriers and specialties analyzed here. These findings run counter to the requirements specified by recent federal statutes that carriers verify and update provider directories at least every 90 days. Many states have similar requirements.¹¹ However, we found that the passage of time was associated with moderate reductions in directory inaccuracies, particularly related to contact information, and with modest increases in the removal of inaccurate listings. We also found substantial variation in provider directory inaccuracies, particularly by carrier, in line with existing literature.^1-5 We found similar variation in corrective action by carrier and specialty, with a relatively low proportion of inaccurate provider listings having been rectified at second contact. Contributing factors may include inadequate administrative capacity to verify and update provider directory information and insufficient enforcement of state and federal regulations.⁴ Providers participating in insurance networks may also experience challenges responding to different processes, documentation, and timelines for responding to multiple plan directory requests, which could also explain some of the observed findings.

The prolonged duration of provider directory inaccuracies has implications for patients, by making navigation of the health care system more challenging, delaying access to care, and increasing the likelihood of out-of-pocket costs.¹⁵ If consumers select plans based on faulty information, persistent inaccuracies may also prevent consumers from accessing their preferred providers. Last, the duration of provider directory inaccuracies confounds efforts to respond to enrollees’ needs. For example, insurance regulators that rely on directories to monitor and assess provider networks may produce unreliable estimates of access to care and network adequacy.

While our findings pertain to single state, many carriers operate in multiple states, and the challenge of directory inaccuracies is widespread across markets and regions.^1-5 From a policy perspective, our findings highlight the need for further tools to reduce provider directory inaccuracies. Differences in rates of corrective action across carriers suggest potential administrative, operational, and health information technology levers that may facilitate more accurate and timely verification.¹⁰ Finally, regulatory action alone—without commensurate support and strong enforcement mechanism—appears to have limited efficacy.^10,11

We note several limitations. First, our analyses were focused on a single state and 1 market, reducing generalizability. However, this state is geographically and socioeconomically diverse and has a large population, and several of the carriers serve multiple states and markets. We note that we did not find any particular pattern between local, regional, or national carriers. Second, we only measured accuracy of provider listings at 2 distinct time points and cannot assess the exact date of provider directory changes. Third, because we sought to ensure that our survey approximates real-world shoppers as closely as possible, callers terminated calls once any inaccuracy was identified. Because callers did not collect information on all potential inaccuracies, we may have underestimated the extent to which directory inaccuracies exist.

Conclusion

In this secret-shopper survey of providers participating in Pennsylvania's ACA Marketplace plans, we found evidence of persistent provider directory inaccuracies that well exceed corrective timelines set by the No Surprises Act. Future research should focus on identifying reasons for variation in corrective action, including removal of inaccurate listings and amendment of erroneous information, at the carrier and provider levels.

Supplementary material

Supplementary material is available at Health Affairs Scholar online.

Funding

This work was funded by the Pennsylvania Insurance Department and the Robert Wood Johnson Foundation. Dr. Zhu reports funding from the National Institutes of Health, the National Institute of Health Care Management Foundation, and the American Psychological Association, which is unrelated to this work.