Measuring What Matters: Development of Standard HIV Core Indicators Across the U.S. Department of Health and Human Services

“If you cannot measure it, you cannot improve it.”

—Lord Kelvin

As noted by public health leaders, efforts to define and monitor quality in systems that deliver population health services have lagged behind comparable efforts in systems that deliver clinical health services.¹ One potential reason for this disparity is the observation that the aims of medical care are arguably more apparent—and certainly more immediate—than those of public health, focusing on the well-being of individuals rather than on the health of entire populations. Moreover, while quality improvement is predicated upon the ability to measure and monitor inputs and outputs in a consistent way,² data collected by various public health systems are often not standardized, with inconsistent case definitions, specifications, and time frames.^3,4 Hence, it is difficult to analyze data across multiple public health systems to improve the coordination of prevention services, intervention effectiveness, and cost-effectiveness.⁵

The challenges in assessing public health quality have particular relevance to federally funded human immunodeficiency virus (HIV)/acquired immunodeficiency syndrome (AIDS) programs. For the most part, federal programs supporting the prevention, diagnosis, and treatment of HIV have arisen, been funded, and been administered independently from one another. Hence, they typically capture data in different ways, using varied terminology standards, time frames, and reporting systems.⁶ In practical terms, this variability means that despite the plethora of required indicators and reporting requirements, federally funded systems of HIV prevention, care, and treatment—even those providing services to the same individuals—currently cannot meaningfully integrate critical performance data across the continuum of HIV care.⁷

To overcome this limitation, the National HIV/AIDS Strategy (NHAS) called for the standardization of data collection and grantee reporting requirements as a means of promoting a more coordinated national response across HIV/AIDS programs that are funded and administered by different federal agencies and offices.⁸ The NHAS also called for the federal government to simplify grant administration activities and streamline grantee reporting requirements for federal HIV programs.

BACKGROUND

Initial movement toward a more coordinated federal response to HIV/AIDS started shortly after the NHAS was released in mid-July 2010. At that time, the Department of Health and Human Services (HHS), along with the other key federal partners (i.e., Department of Justice, Department of Labor, Department of Housing and Urban Development, Department of Veterans Affairs, and the Social Security Administration), began to develop required operational plans. These plans outlined specific actions to advance the strategy along with steps “to strengthen coordination in planning, budgeting for, and evaluating domestic HIV/AIDS programs within and across agencies.”⁹ One of the specific actions to which HHS committed in its operational plan was “to develop common measures and evaluation strategies to assess process and outcomes as they relate to the goals of the NHAS.”¹⁰ HHS acted upon this commitment on April 11, 2012, when Secretary Kathleen Sebelius issued an action memorandum to the heads of HHS' operating divisions and staff offices directing that they had to:

• Within 90 days, finalize a set of common, core HIV/AIDS indicators;

• In the subsequent 90 days, work with the Office of the Assistant Secretary for Health (OASH) to develop an implementation plan to deploy the core indicators, streamline data collection, and reduce reporting burden by at least 20%–25% for HHS grantees providing HIV services; and

• Within 15 months of reaching consensus on the common core indicators and their specifications, fully deploy the implementation plans.

Within OASH, the Office of HIV/AIDS and Infectious Disease Policy (OHAIDP) worked with colleagues across HHS to identify principles to guide the development of a set of standardized, HIV core indicators. It was agreed that the core indicators should:

• Provide a coherent assessment of HHS progress toward achieving NHAS goals;

• Constitute the smallest set of core measures that could reasonably provide a meaningful assessment of federally funded HIV prevention, treatment, and care services;

• Rely, to the extent possible, on existing data;

• Be grounded in the latest scientific evidence;

• Be deemed of adequate quality for use in policy and program decision-making; and

• Reflect the input and perspectives of funders, grantees, and other stakeholders.

To inform the development of the proposed indicators, OHAIDP began by cataloguing the existing HIV indicators as well as their corresponding data systems and elements that are used within each of the HHS agencies and staff offices providing HIV services. Additional guidance was derived from a review of HIV prevention, treatment, and care metrics and their standardization by the National Quality Forum (NQF), the National Committee for Quality Assurance, other federal entities (e.g., Department of Veterans Affairs and Department of Housing and Urban Development), and the private sector.^11–13 This review confirmed that HIV measures in use across HHS were characterized by substantial variation in definitions and specifications that manifested as noncomparable numerators, denominators, and reporting frames. For example, the assessment revealed five different ways that HHS programs were asked to tally a positive HIV diagnosis; four ways in which the initiation of antiretroviral therapy (ART) was to be recorded; three different measures of viral load suppression; and two measures each of early HIV diagnosis, linkage to HIV medical care, and sustained engagement in HIV medical care (Personal communication, Andrew Forsyth, OHAIDP, April 2012).

To reach agreement on a core set of HIV indicators and establish standard data definitions and data elements for each, OHAIDP then convened a cross-agency workgroup that included representatives from the Centers for Disease Control and Prevention (CDC), the Health Resources and Services Administration (HRSA), the Indian Health Service, the Substance Abuse and Mental Health Services Administration (SAMHSA), the National Institutes of Health, and the Centers for Medicare & Medicaid Services (CMS). To ensure that the discussions were not merely limited to a federal perspective, two technical consultations, one in September 2011 and the other in June 2012, provided an opportunity for external stakeholders from academia, state and local health departments, community-based organizations, and national policy organizations to contribute to this critical series of discussions.¹⁴ Another important resource that informed these discussions was the Institute of Medicine report, “Monitoring HIV Care in the United States: Indicators and Data Systems.” This expert committee was convened, at the behest of the Office of National AIDS Policy, to identify core indicators related to continuous HIV clinical care and access to supportive services. The report identified 14 core indicators for HIV clinical care and mental health, substance abuse, and supportive services. It also detailed potential sources of data to assess the indicators.⁶

RECOMMENDED HIV CORE INDICATORS

On July 24, 2012, HHS Secretary Sebelius approved a package of seven core HIV indicators that were developed based on the aforementioned inputs.¹⁵ Figure 1 lists the seven core HIV indicators along with a description of their numerators and denominators. Six of the indicators are directly related to the so-called “HIV treatment cascade” (i.e., the depiction, at a population level, of the proportion of HIV-infected people in the U.S. who are receiving continuous HIV care resulting in viral suppression).^16,17 The seventh indicator, which addresses housing status, is indirectly related to the treatment cascade¹⁸ but emphasizes the importance of a social determinants approach to HIV and public health.

Figure 1.

HHS-endorsed core indicators for monitoring HHS-funded HIV services

HHS = Department of Health and Human Services

HIV = human immunodeficiency virus

AIDS = acquired immunodeficiency syndrome

ART = antiretroviral therapy

Summarizing the continuum of HIV treatment, beginning at diagnosis and extending through viral suppression, provides a useful model to describe the progress and challenges that we face, as a nation, in achieving the NHAS goals. As shown in Figure 2, CDC estimates that in 2009 (the most recent analysis available), only 25% of the more than 1.1 million people living with HIV in America had achieved viral suppression. Maintaining an undetectable viral load is not only associated with better clinical outcomes, it also reduces the risk of transmitting the virus to others.¹⁹ Thus, one of the primary goals for initiating ART is to maximally and durably suppress plasma HIV viral load.²⁰ In fact, the NHAS identifies increases in the proportion of racial/ethnic and sexual minority groups with undetectable (HIV) viral loads as a key measure for assessing our success in reducing HIV-related disparities and health inequities.⁸

Figure 2.

HIV in the United States: the stages of care for the 1.1 million Americans living with HIV, 2009^a

^aSource: Centers for Disease Control and Prevention (US), National Center for HIV/AIDS, Viral Hepatitis, STD, and TB Prevention. Key graphics from CDC analysis showing proportion of people engaged in each of the five main stages of HIV care [cited 2013 May 7]. Available from: URL: http://www.cdc.gov/nchhstp/newsroom/2012/Continuum-of-Care-Graphics.html

HIV = human immunodeficiency virus

ART = antiretroviral therapy

As Figure 2 reveals, substantial attrition occurs at each step of the treatment cascade. Namely, of the nearly 1.1 million people infected with HIV in the U.S., about 18% have not been diagnosed. Further, it is estimated that one-third of the 1.1 million people with HIV have not been linked to care, that nearly two-thirds are not retained in HIV care, and that two-thirds are not prescribed ART. Clearly, to achieve the medical and public health goal of a suppressed HIV viral load, we must intervene at each stage of the treatment cascade by striving to improve the early diagnosis of HIV infection, timely linkage to and retention in care for those who have been diagnosed, awareness of the current U.S. treatment guidelines recommending ART for all HIV-infected individuals,²⁰ and individual and structural support to maintain treatment adherence, including adequate housing.²¹

As they were directed by HHS Secretary Sebelius in April 2012, each of the HHS operating divisions and staff offices providing HIV services has developed an individual plan to deploy core indicators, and implementation efforts are underway. These plans also detail actions that will be undertaken to streamline HIV program data collection and reduce HIV grantee reporting burden by at least 20%–25%. In fact, after reviewing the submitted implementation plans, agencies and offices report an average expected decrease of 39% in the number of required HIV indicators and an average expected decrease in required reporting frequencies of 29%, thus exceeding the initial target. Ongoing engagement with HHS operating divisions is expected to identify additional opportunities for further data streamlining.

PUBLIC HEALTH PRACTICE IMPLICATIONS

While delegating the management of different aspects of comprehensive HIV prevention, care, and treatment to different organizational units across HHS has enhanced implementation capacity, it has also added to the complexity of monitoring across programs in a holistic manner. For example, a client may be receiving HIV prevention services from a CDC-funded program, HIV treatment services from an HRSA-funded program, and substance abuse and/or mental health services from a SAMHSA-funded program. The implementation of these core indicators across HHS will enable the department to substantially improve the interoperability of key data required to monitor and assess HIV efforts across its operating divisions and will support HHS efforts to advance a diverse array of HIV programs and activities that are administered in a variety of organizational units.

A 2013 report published by the HHS Office of the Inspector General (OIG) highlights the need for and utility of common metrics to assess federally funded HIV program activities.²² The objective of the report was to determine the extent to which HRSA-funded health center sites had implemented CDC's 2006 guidelines, which recommended routine HIV testing in health-care settings for all people aged 13–64 years. The report found that financial constraints and other factors affected full implementation of the CDC HIV testing recommendations. In addition, and more directly relevant to the focus of this discussion, OIG made note of the “recently released seven core indicators” and observed that their adoption “would align data collection by HRSA health center grantees with that of other HHS-funded HIV programs and services” and would raise awareness of HIV testing as “critical to reducing the spread of HIV.”²³

Recent national analyses from CDC show continuing demographic disparities along the continuum of HIV care from diagnosis through viral suppression; African Americans and young people in the U.S. are least likely to receive ongoing care and effective treatment for HIV.¹⁷ The implementation of standardized HIV program measures and demographic variables across HHS programs will enable policy makers and program managers to better identify and address these and similar service gaps and will provide the “minimum floor of data” that public health analysts have called for as necessary to assess and respond to health disparities such as HIV/AIDS.²⁴

This brief report also describes the first steps toward minimizing or removing unwarranted, minimally useful, and/or duplicative HIV/AIDS reporting requirements. Requests to reduce HIV program reporting burden have been voiced by directors of state and territorial AIDS programs²⁵ and are called for in the NHAS.⁸ Policy analysts, writing broadly about federal grant reporting requirements, have likewise noted that there has been a pronounced tendency in grants management toward “proliferation of requirements and requests for data” without a “clear correlation between those requirements and grantee performance.”²⁶ As such, our efforts to streamline and better focus HIV/AIDS program reporting requirements are in line with recommendations being raised in other publicly funded service programs.

To paraphrase Winston Churchill, we are nearing “the end of the beginning” when it comes to expanding our capacity to monitor federally funded HIV/AIDS programs in an integrated, streamlined fashion. Substantial collaborative work has gone into identifying the core HIV measures and agreeing on other measures that can be removed or reduced because they are less informative. The expected costs for implementing these recommendations are not anticipated to be excessive but may include resources for technical assistance, training, or programming associated with revised data collection instruments and processes that will be required to deploy the core indicators and reduce data elements and reporting requirements. HHS intends to conduct further work to assess return on investment and the benefits of adopting core indicators and streamlining HIV data collection. Clearly, competent program monitoring is not the sole requirement for ensuring successful health outcomes for HIV or any other public health program. But as we move forward in our collective efforts to achieve the goals of the NHAS, we would do well to remember advice from another citizen of the United Kingdom, Lord Kelvin, who cautioned that, “If you cannot measure it, you cannot improve it.”