Modeling underservedness: An evidence‐based approach to managing access to care

1. INTRODUCTION

Paramount to population health is adequate access to care. Health systems can offer the highest quality care, but if their patients cannot access it, the benefits are lost. The National Academy of Medicine (previously, the Institute of Medicine) deems access to care—whenever needed and through whatever delivery method—a critical dimension of care quality. ¹

Adequate access to care leads to improved health outcomes. ² To this end, the Veterans Health Administration (VHA) strives to provide timely, high‐quality care to Veterans within a budget established by Congress. Conversely, inadequate access can result in longer waiting times and increased Veteran mortality. ³ Congress and VHA have launched several initiatives, such as the passage of the Veterans Access, Choice, and Accountability Act of 2014, to improve access to VHA care and increase Veteran access to non‐VHA care when necessary. ⁴ Over time, these initiatives did improve VHA access while private sector waiting times remained stagnant. ⁵

However, these VHA policies, new and old, did not address every access barrier Veterans face because they relied on traditional access metrics. Traditional metrics have limitations. For example, waiting times alone do not tell the whole access story. Many factors, both those that can be changed and those that cannot, influence access to care. Adequately accounting for these nuances requires a new metric, one that builds on waiting times but gives clearer insight into changes in supply of and demand for care.

The passage of the Veterans Affairs Maintaining Internal Systems and Strengthening Integrated Outside Networks Act of 2018 (MISSION Act) provided an opportunity to develop such an access metric. ⁶ Section 401 of the law directed VHA to establish criteria to measure and mitigate underservedness at each Veterans Affairs Medical Center (VAMC) nationwide. The Partnered Evidence‐based Policy Resource Center (PEPReC) responded to this mandate in partnership with the Office of Veterans Access to Care (Access Office). ⁷ PEPReC defined underservedness as an imbalance between supply of VHA care and expected Veteran demand for care and developed a statistical model to identify VAMCs at risk of being underserved in primary care or various specialties.

While the impetus for developing this new access metric was to comply with a congressional mandate, its reach extends beyond Section 401 of the law. The underserved model and scores have influenced virtual care resource allocation in Section 402 of the MISSION Act, mental health staffing predictions, budget forecasting, and pilot programs in local clinic management. In each new application, the underserved model proves more useful than traditional metrics because it allows VHA leadership to see their supply management options in the context of expected demand for services, adjusting for factors beyond VHA control and supporting a clearer focus on local management performance. What is more, the same concepts are applicable beyond VHA; supply and demand modeling can ensure any health system is making evidence‐based decisions when assessing access to care for its patients.

2. BACKGROUND ON PEPReC

PEPReC is a Quality Enhancement Research Initiative resource center focused on providing VHA leadership with evidenced‐based solutions to policy questions. The center works closely with VHA leadership to identify and mitigate issues surrounding access to care, quality of care, and Veteran satisfaction. Led by economists, PEPReC is well suited to assess the imbalances between supply of and demand for VHA care, as required by the MISSION Act. This paper will explain PEPReC's novel approach to measuring access, with an emphasis on policy implications. Future research will evaluate the impact of this approach on tangibly improving access.

3. NEED FOR A NOVEL APPROACH

When discussing this work with stakeholders, the first question is often why VHA needs a novel measure for access. If underservedness is just another way to look at access to care, VHA could easily use waiting times as a proxy. Waiting times are widely accepted in both VHA and the private sector. However, just using waiting times to identify underserved VHA facilities is a simplified approach to a complex problem.

First, a plethora of variables affect waiting times, both measurable and unmeasurable. But the nuances of these relationships are not captured in the raw data and would not be accounted for if VHA strictly used waiting times to measure underservedeness. What is more, VHA waiting times are quite good when compared with the private sector. ⁸ In other words, Veterans are often able to get in to see a VHA provider faster than most can get in to see private sector providers. Using waiting times to measure underservedness could lead to an inflated view of Veteran access to care.

Secondly, access to care is a complex concept, not limited to the number of days one waits to see a provider. Congress understood this when it passed the MISSION Act. The law mandates that VHA consider several different variables when measuring underservedness, such as the Veteran‐to‐provider ratio at a VAMC, waiting times, and local access to care in the community surrounding a facility. Waiting times must be considered, but not exclusively. Thus, even if stakeholders preferred assessing underservedness by waiting times alone, that would not comply with the law. By directing VHA to account for a variety of factors, Congress encourages VHA to apply the principles of evidence‐based policy making, developing models that explicitly link specific policy and management levers to program beneficiaries' experiences—to create “metrics that matter.” ⁹

4. UNDERSERVED MODEL + SCORES

In the first year (fiscal year [FY] 18) after the law's passage, PEPReC used the economic principles of supply and demand to develop a statistical model to measure underservedness in primary care. Now, primary care underserved scores have been calculated for the past 3 years, and the model will be applied to both primary care and specialty care in FY22. A working paper describing the modeling approach in detail is available on PEPReC's website. ¹⁰

A key access measure used in VHA, and the dependent variable of interest in this analysis, is the new patient create date waiting time: the number of days between the date the Veteran calls to make his or her first appointment and the date of the corresponding clinic visit. This measure of primary care waiting times has been validated through patient experience surveys. ¹¹

However, as noted previously, the model does not simply use raw waiting times to measure underservedness. Waiting times are affected by various factors that can be categorized into two groups: VAMC capacity (i.e., supply) and Veteran need (i.e., demand). The model estimates empirical relationships between supply and demand variables and waiting times, which are then used to calculate numerical underserved scores. The underserved scores are adjusted, predicted new patient waiting times; they are not raw waiting times but rather empirically estimated, and are adjusted by the dozens of variables included in the model. The relationships between waiting times and the model variables are explained with numerical weights, with a larger positive or negative weight indicating more influence over waiting times. The numerical weights are translated into nonnumerical weights for ease of understanding by stakeholders (see Table 1).

TABLE 1.

Underserved model variable list (FY21 ^a )

Variable name	Variable definition	Variable influence
Clinic efficiency (physicians/APPs b )	Total clinic encounters per day of clinic capacity. An important element of clinic operations, clinic efficiency mediates the relationship between clinic inputs and total encounters produced by the clinic.	− HIGH
Clinic capacity (physicians/APPs)	Total primary care clinic time for primary providers of the PACT c team. This measure accounts for all physicians and APPs who generate workload in primary care clinics and incorporates in‐person, virtual, and unscheduled care.	− HIGH
Medicare Advantage penetration	The percentage of eligible individuals who have Medicare Advantage coverage in the area surrounding a VAMC d .	− HIGH
Percent of established patient appointments scheduled > 90 days	The percentage of established patient visits scheduled 90 days or more in advance.	+HIGH
Number of community care visits	The number of community care visits purchased for Veterans living within 40 miles of a VAMC.	− MED
Household median income	The median household income in the area surrounding a VAMC. Median household income varies across the country and is used to measure affluence.	− MED
Percent with private insurance (males 18–64)	The percentage of males aged 18–64 years eligible for private health insurance coverage in the area surrounding a VAMC.	+ MED
Percent of enrollees priority 7/8	The percentage of a VAMC's Veteran enrollees who are classified as Priority 7 or 8. Veteran enrollees are placed in one (or more) of eight priority groups based on need, affluence, and eligibility for VA care.	− MED
Clinic capacity (non‐physicians/non‐APPs)	The total primary care clinic time for non‐physician/non‐APP members of the PACT teams. This measure accounts for all other providers and staff who generate workload in primary care clinics (nurses, social workers, clinical pharmacists, psychologists, and dieticians) and incorporates scheduled in‐person and virtual care.	− MED
Number of enrollees	The total number of Veteran enrollees within each facility's catchment area.	− MED
Percent of enrollees 65 or older 12	The percentage of a VAMC's enrollee population that is over 65 years old. Medicare eligibility typically begins at 65 years old. This variable is a proxy for the number of VAMC's Veteran enrollees who have Medicare coverage.	− MED
Unemployment rate	The rate of unemployment in the area surrounding a VAMC.	− MED
Nosos risk score 13	The average clinical risk/complexity for a Veteran enrollee at a VAMC. A higher average Nosos score signifies a more complex Veteran population.	− MED
Primary care HPSA e	Identifies HPSAs, geographical areas with an insufficient number of providers based on population size, and an overutilization or inaccessibility of existing providers.	− LOW
PACT panel size	The current capacity of a VAMC's PACT teams. PACT teams are a multidisciplinary team‐based approach to providing primary care. PACT teams are assigned a specific number of Veterans to serve (a panel).	+ LOW
Zillow house price index	Measures changes to housing prices in the area surrounding a VAMC. Higher housing prices may be associated with higher monthly rents.	− LOW
Average drive time to primary care	The average drive time to a primary care clinic for a VAMC's Veteran enrollees.	− LOW
PACT return visit rate	The frequency with which a primary care clinic's established patient population returns to clinic each year.	+ LOW
Mental health program complexity	The complexity of the mental health (MH) services provided at a VAMC. MH program complexity is scored in two parts: the percentage of a VAMC's Veteran enrollees who use MH services and the number and types of MH programs available.	− LOW
ICU f /surgical program complexity	The availability and complexity of both ICU care and surgical care provided at a VAMC.	− LOW
Complex clinical program complexity	The number of complex clinical programs provided at a VAMC. Twelve clinical programs, such as blind rehabilitation or polytrauma, are included and increase the administrative and clinical complexity of a VAMC.	− LOW

Note: This table lists the variables included in the FY21 primary care underserved model, including variable definitions and variable influence over underservedness.

^a Fiscal Year.

^b Advanced Practice Provider.

^c Patient‐Aligned Care Team.

^d Veterans Affairs Medical Center.

^e Health Provider Shortage Area.

^f Intensive Care Unit.

To continually improve VHA's estimation of underservedness, PEPReC recalculates scores and rankings each year, improving the model each time. Consequently, details of the model can change. For example, in the second year of analysis, we were able to source more granular data for several variables compared with what was available the first year, making the overall scoring mechanism more sensitive to variations over time. The Access Office and PEPReC also regularly brief VHA national and local leadership, requesting feedback on model design. These briefings have been fruitful and have led to model changes, including new variables and new methods of measuring existing variables. The ultimate goal is to have a dynamic model, responsive to changes in supply of and demand for VHA care, that is statistically sound and useful to national and local VHA leadership.

5. IMPACT OF UNDERSERVED MODEL + SCORES

The underserved model and scores allow VHA to systematically identify imbalances in the supply of and demand for VHA care nationwide. One benefit of using statistical modeling to identify underserved VAMCs and, subsequently, underserved populations, is that it fosters evidence‐based policy making and equitable resource allocation. This is a critical priority for VHA leadership, reinforced by the passage of the Foundations for Evidence‐Based Policymaking Act in 2019. ¹⁴

Each year, after running the model and producing underserved scores, VAMCs are ranked relative to each other. The most underserved VAMCs are required to submit action plans to the Access Office explaining how they plan to mitigate underservedness (using personnel‐, technology‐, and infrastructure‐based strategies) and the Access Office then reports both the facilities and action plans to Congress.

However, the impact of the underserved model and scores extends beyond responding to the congressional mandate in Section 401 of the MISSION Act. This novel approach to access to care is also used to guide resource allocation in Section 402 of the MISSION Act, mental health clinic operations forecasting, budget forecasting, and local clinic management. This approach is also applicable to other health systems.

5.1. Section 402

Section 402 authorized VHA to pilot Mobile Deployment Teams (teams of telehealth providers providing primary care or mental health care) at underserved facilities to increase capacity and access to VHA care. These teams are targeted in response to the needs identified in Section 401 and are managed regionally. Mobile Deployment Teams are nested within VHA's existing Clinical Resource Hubs.

A Clinical Resource Hub is a virtual care resource center physically located at one VAMC but serving many other facilities through telehealth. Telehealth providers at the hub virtually treat patients within a specific geographical region. Clinical Resource Hubs have been particularly helpful in rural areas where the supply of VHA and non‐VHA care is often low, but they also represent an opportunity for VHA to rapidly respond to geographical shifts in demand for care.

5.2. Mental health clinic operations forecasting

Beyond congressionally mandated resource allocation, PEPReC uses the underserved model to assist VHA mental health operations leadership with clinic operations forecasting. ¹⁵ , ¹⁶ PEPReC applies the economic principles of supply and demand to assess the relationship between mental health clinic capacity and efficiency and Veteran engagement with services and mental health outcomes. Leadership uses these findings to investigate the ideal size and function of a mental health clinic and how VHA can best allocate and utilize resources nationwide. This work has been well received, suggesting the underserved framework continues to gain traction as a new way to assess access to care.

5.3. Budget forecasting

In another opportunity to use the underserved model to inform VHA policy, PEPReC has been asked by finance leadership to estimate how many additional clinical staff would be needed to reduce VHA waiting times to the proposed MISSION Act access targets. The MISSION Act states that a Veteran is eligible to receive care in the community if VHA cannot provide a primary care appointment within 20 days or a specialty care appointment within 28 days. Thus, leadership is asking how many additional clinical staff would be needed to provide in‐house care for all Veterans who are currently waiting for time‐eligible for community care.

Using the underserved model and the supply and demand framework more broadly, PEPReC estimated the necessary staffing increases (and associated budget increases) to meet the MISSION Act access standards. PEPReC's estimates were included in the FY21 and FY22 budgets and submitted to the Office of Management and Budget. ¹⁷ Using statistical modeling to forecast staffing and budget needs is an evidence‐based way to anticipate and mitigate future mismatches between supply and demand for VA care.

5.4. Local clinic management

Complementing the impact on the national level, the underserved model has inspired a local clinic management pilot. Specifically, PEPReC launched a pilot program in FY21 to test the effectiveness of a clinic efficiency reporting dashboard on improving clinic function and access to care. Clinic efficiency, as defined in the underserved model, assesses how well a clinic utilizes its limited resources to care for its Veterans. It is a significant predictor of access to care, and improvements to clinic efficiency can enhance Veterans' experiences and bring more care back in‐house rather than sending Veterans to community providers.

The primary goal of the pilot is to assist local clinic managers in understanding the complex dynamics of clinic efficiency at their facilities. Participating facilities undergo intensive data review with PEPReC, but the results are used only to identify potential areas for improvement rather than to prescribe a particular solution. The use of the clinic efficiency dashboard is locally driven, with different facilities or clinics utilizing it differently to meet specific management goals.

PEPReC works closely with regional and local leadership to refine the empirical methods of the pilot and better understand the relationship between clinic efficiency and operational outcomes. This pilot represents an opportunity to improve data reporting and inform evidence‐based policy making and management at the local level. In turn, the pilot gives VHA leadership another way to address the MISSION Act's access standards.

6. BEYOND VHA

The lessons learned from the underserved model extend beyond VHA and are applicable to other health care systems. The underserved model shows that access to care is a multifaceted concept, and attempts to improve it, regardless of health system or patient population, will require a multifaceted approach. By accounting for changes in both supply of care and demand for care through statistical modeling, leaders can more accurately predict access bottlenecks and mitigate them with tailored solutions.

For example, an urban academic medical center may wonder how to improve access for specialty care patients who come from the surrounding rural areas. A small community health center may seek to understand if hiring a full‐time provider is worthwhile based on demand for care. A large regional health system may want to study the tradeoffs between prioritizing access for new and established patients. All these scenarios could be viewed as separate operational questions, but they also serve as opportunities to better understand underlying supply and demand dynamics and select the most appropriate solutions, including changes in scheduling practices, staffing, virtual care, and productivity. Using the underserved framework and a similar statistical modeling approach, VHA and other health systems or providers can view access to care as more than a target metric; instead, it is a dynamic policy challenge responsive to evidence‐based solutions.

7. CONCLUSION

Section 401 of the MISSION Act requires VHA to identify and mitigate underservedness nationwide. Under the Access Office's leadership, PEPReC developed a statistical model based on the economic principles of supply and demand to measure underservedness in primary care and specialty care, ranking VAMCs annually by relative underservedness. While the underserved model and scores fulfill the congressional mandate, they also support evidence‐based and equitable resource allocation beyond the law. The underserved model serves as a new and more analytic approach to measuring access to care, separating potentially manageable supply issues from demand factors beyond VHA control and promoting access through improved national budget and staffing forecasting and local clinic management. Going forward, this foundation will allow VHA to keep Veteran access to high‐quality health care a critical policy priority and incorporate social determinants of health into overall Veteran population health management.

Pearson E, Kirsh S, Pizer SD. Modeling underservedness: An evidence‐based approach to managing access to care. Health Serv Res. 2022;57(Suppl. 1):77‐82. doi: 10.1111/1475-6773.13906