Beyond Measuring What Matters to Managing What Matters: Improving Public Health Quality and Accountability in the U.S. HIV Epidemic Response

The National HIV/AIDS Strategy (NHAS), released in July 2010, is visionary in its recognition of the instrumental role of public health data in maximizing the U.S. epidemic response to human immunodeficiency virus (HIV)/acquired immunodeficiency syndrome (AIDS). The ability to measure outcomes along the continuum of HIV care^1,2 is even more significant now that national guidelines^3,4 recommend antiretroviral therapy (ART) for all HIV-infected individuals. This shift demands greater efforts to ensure that all HIV-infected individuals are diagnosed, linked, and engaged in care; begin ART; and achieve durable viral suppression. Increasing the number of people with durable viral suppression is vitally important for individual health with the collateral benefit of reducing HIV transmission.^5–8

If measuring these outcomes is of such paramount importance, why, then, are we not measuring what matters? In most public health jurisdictions, different federal agencies fund, and different local entities provide, HIV services and address social determinants of health. Numerous independently funded public and private clinical care programs, diverse community-based organizations, and distinct public health divisions provide HIV testing, linkage, partner services, postexposure prophylaxis, and treatment. Every grant from every funder has separate reporting requirements, with distinct, often conflicting indicators for the same steps in the continuum of care. As a result, it is difficult to manage what we measure.

While public health officials have attempted to create a holistic system of prevention and care using diverse funding streams, reporting requirements necessitate obtaining data from distinct entities, with non-interoperable databases or electronic medical record systems, and disparate capacities for reporting. Without valuable local information about rates of testing, diagnosis, linkage, engagement, and retention in care, jurisdictions may struggle to best allocate shrinking resources to areas with the greatest need.

In the article in this issue of Public Health Reports titled “Measuring What Matters: Development of Standard HIV Core Indicators Across the U.S. Department of Health and Human Services,” Valdiserri et al.⁹ describe the significant efforts led by the Office of HIV/AIDS and Infectious Disease Policy (OHAIDP) and others within the Department of Health and Human Services (HHS) “to develop common measures and evaluation strategies to assess process and outcomes as they relate to the goals of the NHAS.”¹⁰

The assessment of existing indicators revealed five ways to report a positive HIV diagnosis; three different measures of viral suppression; and two measures each of early diagnosis, linkage to HIV medical care, and engagement in care. After a process guided by shared principles and supported by stakeholder input, OHAIDP ultimately distilled a set of seven core, population-level indicators measuring diagnosis, linkage, retention, ART use, viral suppression, and housing status.⁹ Each HHS agency committed to reduce the number and frequency of reporting other indicators. In a corresponding process, the Office of National HIV/AIDS Policy commissioned an Institute of Medicine (IOM) study on indicators and data systems for monitoring HIV care quality.¹¹

RECOMMENDATIONS FOR MEASURING WHAT MATTERS

While this study is a considerable first step, the following persistent challenges should also be considered. A local health department may need to report on NHAS goals multiple times a year to the Centers for Disease Control and Prevention, providing interim and annual progress reports for the cooperative agreement, jurisdictional HIV prevention plan, and enhanced comprehensive HIV prevention plan, as well as to the Health Resources and Services Administration for the Ryan White Planning Council plan. A client who receives HIV care, case-management, adherence support, mental health, and substance use services from a medical home clinic may need to recertify eligibility separately for each service every six months, jeopardizing the client's continuity in care.

HHS should consider the following recommendations to improve the ability to measure what truly matters.

Reduce duplication and frequency of reporting

Standardizing data elements, definitions of program activities, forms, and data systems could further reduce duplicate data reporting. A thoughtful effort should be undertaken to align due dates of reports containing similar information to different HHS agencies or parts of the same agency and determine the appropriate periodicity of reports required by different agencies.

Simplify financial reporting and grants administration

Administration burden could be significantly reduced through the removal of high-burden/low-value reporting (e.g., six-month eligibility recertification requirements, given high levels of constancy year after year), the creation of uniform budget codes, increased flexibility for budget reallocation, and efforts to reduce onerous billing requirements.

Encourage data sharing while protecting privacy

A recent Government Accountability Office report, while not specifically focused on HIV, reviewed laws, regulations, and policies and found that facilitating data sharing across human services agencies could enhance eligibility verification, case-management processes, and program accountability, while protecting privacy.¹²

Ensure consistency across HHS agencies

Administration burden could be further reduced through consistent written policies to standardize grants management guidance provided by program officers, contractors, and other staff across all agencies.

RECOMMENDATIONS FOR MANAGING WHAT MATTERS

As acknowledged by Valdiserri et al.,⁹ we need to move beyond simply measuring what matters. We must manage what matters by disrupting our current approach to public health program quality and accountability and spurring innovation. Clinical quality improvements, such as the reduction of medication errors or wrong-sided surgeries, required systems-level innovations in organizational culture.^13–15 Similarly, we need to disrupt the way we use surveillance and other data to take public health action and improve population-level health quality. There has been a historical reluctance to use HIV data for public health interventions as is done with other infectious diseases,¹⁶ due to real concerns about protecting the privacy and rights of people living with HIV/AIDS. There are also legal, regulatory, and organizational practices that hinder the use of surveillance data for public health action.

Issue uniform guidelines for using HIV surveillance to promote public health action

Both community and public health attitudes have shifted in favor of using surveillance data for linkage and reengagement.^16–18 Indeed, it has been noted that once the surveillance data are in hand, it is the failure to use those data for improving public health that must be ethically justified.¹⁹ Several examples exist of thoughtful community- and clinician-endorsed innovations to improve linkage and retention by data sharing beyond traditional surveillance firewalls in San Francisco, New York City, Washington, D.C., Seattle, and Louisiana.^20,21 HHS can empower these local efforts by issuing a uniform set of guidelines to support data sharing across federal partners and with external state, local, and community partners for the purpose of public health action. These guidelines should be informed by U.S. open-government initiatives^22,23 that support data sharing as well as global efforts on personal data security.²⁴

Improve the HIV/AIDS surveillance system and other federal data systems to achieve NHAS goals

HHS should implement IOM recommendations to enhance HIV/AIDS surveillance systems by uniformly collecting CD4 cell-count, viral load, and ART data.¹¹ HHS should direct federal agencies to review and modify their data systems to better accommodate NHAS goals.

Harness trends in health information technology (HIT), big data, and predictive analytic methods

The HHS HIV Open Data Project²⁵ is a groundbreaking effort supporting the use of data collection interfaces, information management systems, and other HIT. The effective use of HIT, integration of electronic health record data, and exchange of health-related information has the potential to improve continuum-of-care outcomes.¹¹ Predictive analytic methods using regression analyses of large-scale datasets that combine data from clinical, demographic, insurance, surveillance, and other databases could generate weighted drivers of risk for individuals at the highest risk for failing to link to care or dropping out of care, allowing public health officials to allocate limited resources in a more informed manner than what could be gleaned from individual studies of siloed datasets.²⁶ HHS should continue its leadership in this area and support innovation in HIV open-data approaches at both federal and local levels.

Evaluate laws, regulations, and policies, and incentivize changes to ensure they are evidence-based, consistent with current guidelines, and do not obstruct public health action

Current laws, regulations, and policies may provide barriers to ART initiation or accessing housing. HHS should conduct a thorough review of its program eligibility requirements to ensure that they do not provide an adverse incentive for HIV/AIDS-related disease progression, as certain key benefits may still be linked to an AIDS diagnosis or low CD4 count.

Synthesize data from different HHS programs and provide local jurisdictions with periodic feedback

HHS, as the collector of reported data, has the responsibility to use all the data collected to not only measure what is happening, but also to meaningfully help the programs manage quality. The collected data should be synthesized and reported back to local jurisdictions periodically to facilitate a culture of continuous public health program quality improvement and to hold both individual programs and the entire system accountable to the NHAS goals.

CONCLUSION

We are in an unparalleled time of advancements in HIV prevention and treatment.^8,27–29 However, significant gaps persist in implementing these strategies.^1,2 The clear leadership demonstrated by OHAIDP and HHS is a vitally important first step to ensuring that we measure what matters. Further innovation will be required to ensure that what we measure gets managed, so that we can significantly improve outcomes for people living with HIV/AIDS and begin to eliminate new HIV transmission in the U.S.