Reducing Cardiovascular Disparities through Community-Engaged Implementation Research: A National Heart, Lung, and Blood Institute Workshop Report

Abstract

Cardiovascular disparities remain pervasive in the US. Unequal disease burden is evident among population groups based on sex, race, ethnicity, socioeconomic status (SES), educational attainment, nativity, or geography. Despite the significant declines in cardiovascular disease (CVD) mortality rates in all demographic groups over the last 50 years, large disparities remain by sex, race, ethnicity, and geography. Recent data from modeling studies, linked micromap plots, and small-area analyses also demonstrate prominent variation in CVD mortality rates across states and counties, with an especially high disease burden in the southeast US and Appalachia. Despite these continued disparities, few large-scale intervention studies have been conducted in these high-burden populations to examine the feasibility of reducing or eliminating cardiovascular disparities. To address this challenge, on June 22-23, 2017, the National Heart, Lung, and Blood Institute (NHLBI) convened experts from a broad range of biomedical, behavioral, environmental, implementation, and social science backgrounds to summarize the current state of knowledge of CVD disparities and propose intervention strategies aligned with the NHLBI mission. This report presents the themes, challenges, opportunities, available resources and recommended actions discussed at the workshop.

Introduction

Disparities in cardiovascular health (CVH) and cardiovascular diseases (CVD) remain pervasive in the US and account for a large proportion of the overall health disadvantage suffered by high risk populations.^1–3 Important progress in disparities has been made as shown in Figure 1. For example, the black-white disparity gap in all-cause mortality rate for all ages narrowed from 33% in 1999 to 16% in 2015 and for persons aged 65 years and older, the all-cause death rate in 2015 compared to that in 1999 declined 27% for Blacks and 17% for Whites. This decline resulted in the elimination of the black-white disparity gap in 2010 and a lower death rate in Blacks than Whites thereafter.⁴ For diseases of the heart, the death rate disparity in Blacks relative to Whites declined from 27.6% in 1999 to 22.2% in 2015 (Table 1).⁴

Figure 1. Death rates among Blacks and Whites, by age group (years) – United States, 1999–2015.

Among adults aged ≥65 years, the death rate in 2015 relative to that in 1999 declined 27% for Blacks and 17% for Whites, resulting in a crossover in death rates beginning in 2010, when Blacks had lower death rates than Whites. Reproduced from Cunningham TJ, et al. Vital Signs: Racial Disparities in Age-Specific Mortality Among Blacks or African Americans - United States, 1999-2015. MMWR Morb Mortal Wkly Rep. May 05 2017;66(17):444-456.

TABLE 1.

Death rates per 100,000 population for all causes of death and death from diseases of the heart, percentage changes, and death rate disparities between blacks and whites, by age group — National Vital Statistics System, United States, 1999 and 2015. Reproduced from Cunningham TJ, Croft JB, Liu Y, Lu H, Eke PI, Giles WH. Vital Signs: Racial Disparities in Age-Specific Mortality Among Blacks or African Americans - United States, 1999-2015. MMWR Morb Mortal Wkly Rep. 2017;66(17):444-456.

Cause of death by age group (yrs)	Blacks			Whites			Death rate disparity relative to whites*
	1999 rate	2015 rate	% change (1999 to 2015)	1999 rate	2015 rate	% change (1999 to 2015)	1999(%)	2015 (%)
1. All causes
All ages†	1.135.7	851.9	−25.0§	854.6	735.0	−14.1§	+32.9§	+15.9§
18–34	167.8	141.5	−15.6§	87.5	100.3	+14. 6§	+91.8§	+41.1§
35–49	454.3	311.5	−31.4§	218.2	220.3	+1.0§	+108.2§	+41.4§
50–64	1,346.5	1,046.0	−22.3§	746.5	722.4	−3.2§	+80.4§	+44.8§
≥65	5,712.7	4,176.0	−26.9§	5,186.0	4,286.9	−17.3§	+10.2§	−2.6§
2. Diseases of the heart
All ages	334.3	205.1	−38.7§	262.0	167.9	−35.9§	+27.6§	+22.2§
18–34	12.5	10.7	−14.5§	4.8	5.1	+5.2	+158.7§	+110.3§
35–49	85.3	66.5	−22.0§	37.9	33.3	−12.0§	+125.2§	+99.7§
50–64	378.6	257.5	−32.0§	193.9	148.1	−23.6§	+95.2§	+73.9§
≥65	1,902.6	1,085.5	−42.9§	1,756.7	1,091.8	−37.9§	+8.3§	−0.6

Definitions:

^*Disparity (%) = (Black rate minus white rate) divided by white rate times 100.

^†“All ages” category includes infants and children. Death rates for all ages were age-standardized to the 2000 U.S. projected population.

^§Z-statistic significant at p<0.05 for the rate change from 1999 to 2015 or for the difference between black and white populations.

Despite this progress, Blacks younger than 65 years old and several population groups and communities continue to demonstrate marked cardiovascular disparities, and in some settings, the disparities are widening.^4–12 These disparities arise from differences in major cardiovascular risk factors and causes of death and disability that are preventable^{1–3, 13} and are seen in all stages of the life course beginning in the intrauterine environment^{14, 15} and early childhood^16–18 through young adulthood^19–21 and old age.^22–24 In addition, social determinants of health, such as low socioeconomic status (SES), stress, poor social support, depression, anxiety, and living in disadvantaged neighborhoods contribute to cardiovascular disparities.^25–28

Addressing the disparate population burden of CVD remains a major clinical and public health challenge.^{1, 29} Accordingly, the National Heart, Lung, and Blood Institute (NHLBI) convened a workshop on June 22-23, 2017 that engaged a multidisciplinary group of experts to discuss, develop, and prioritize themes and strategies aligned with the NHLBI mission to reduce disparities using community-engagement and implementation research frameworks.³⁰ The workshop participants included cardiovascular, social, and spatial epidemiologists; survey methodologists; community-based participatory researchers; health disparities researchers; social and behavioral science researchers; experts in the inter-sectionality of community engagement between communities and academic institutions; dissemination and implementation (D&I) science experts; health services researchers; and clinician-scientists with expertise in general internal medicine, primary care, stroke, type 2 diabetes, depression, and heart, lung, and blood diseases. For the purposes of this workshop, disparities was defined as preventable differences in the incidence, prevalence, prevention, treatment, morbidity, and mortality from CVD and related risk factors.^{1, 29} Inherent in this definition are preventable differences in access to health care and the quality of health care delivered.

While CVD disparities by race/ethnicity have been recognized for decades,³¹ geographic disparities have only recently begun to receive more attention.^{32, 33} For example, Roth, et al., showed a 4-fold difference in the mortality from hypertensive heart disease between counties at the 10^th and 90^th percentiles of the average national rate.³³ Substantial county-level differences, in both relative and absolute terms, were also noted for ischemic heart disease and stroke mortality, with the largest concentration of counties with high CVD mortality extending from southeastern Oklahoma along the Mississippi River Valley to eastern Kentucky³³ (Figure 2). Other studies also show large state- and county-level CVD disparities that are likely to worsen, especially in southeastern US and Appalachia. These patterns of geographic disparities in CVD are not surprising given the high levels of poverty³⁴ and high prevalence of CVD risk factors in these regions.³⁵

Figure 2. United States County-Level Mortality from Cardiovascular Diseases.

A: Age-standardized mortality rate for both sexes combined in 2014. B: Percent change in the age-standardized mortality rate for both sexes combined between 1980 and 2014. In panel A, the color scale is truncated at approximately the 1st and 99th percentiles as indicated by the range given on the scale. In panel B, the color scale is similarly truncated at the 1st percentile but not at the 99th percentile to avoid combining counties with decreases in the mortality rate and counties with increases in the mortality rate into a single group. C: Age-standardized mortality rate in 1980, 1990, 2000, and 2014. The bottom border, middle line, and top border of the boxes indicate the 25th, 50th, and 75th percentiles, respectively, across all counties; whiskers, the full range across counties; and circles, the national-level rate. Reproduced from Roth GA, et al. Trends and Patterns of Geographic Variation in Cardiovascular Mortality Among US Counties, 1980-2014. JAMA. May 16 2017;317(19):1976-1992; with permission from the American Medical Association.

Geography-based differences that contribute to CVD disparities are, among others, neighborhood characteristics, access to health-promoting resources, behavioral (e.g. diet, physical activity, and tobacco use), psychosocial (stress, depression, social support), and cultural factors (acculturation, dietary patterns), and access to quality health care.³⁶ These geographical differences need to be understood in the context of growing economic inequality in the US and recognition that these populations are at high risk, at least in part, as a result of discrimination and economic and political disempowerment.³⁷ The tragic explosion of the opioid epidemic bears further witness to the impact of socioeconomic challenges many of these communities face.³⁸

The aim of the present workshop was to go beyond identifying problems. The priority was to search for and examine solutions. Although the NHLBI supported a series of large-scale multi-level intervention studies in the past, there are few recent attempts to examine translation and implementation research of evidence-based programs to reduce or eliminate CVD disparities. In this report, the workshop participants make recommendations for NHLBI to consider and emphasize (1) identification of high burden communities; (2) commitment to best practices in community engagement; (3) implementation of evidence-based interventions likely to be successful in these communities; (4) alignment of community partners and care networks; (5) incorporation of novel research methods and evaluation milestones; and (6) development of approaches for training the next generation of implementation research investigators for community-engaged health disparities research.

Identification of High Burden Communities

As noted above, the existence of significant CVD disparities across the key demographic categories in the US has been recognized for many years.^{3, 31, 39} The major population groups with high rates of CVD in the US conform to an all too familiar pattern, primarily including racial and ethnic minorities, especially African Americans and American Indians/Alaska Natives, and persons living in poverty.^{1, 31, 40} In 2015, the age-adjusted death rate from all cardiovascular conditions was 282 per 100,000 for Blacks compared with 220 per 100,000 for Whites.⁴¹ An even larger risk gradient can be identified by SES.^42–44 In 2013, the mortality rate from CVD per 100,000 was 212 in large metropolitan areas compared with 259 in nonmetropolitan (rural) areas.⁴¹ Generating a framework for community-based interventions, however, requires both a comprehensive assessment of the burden of social and economic factors likely driving the disparity and the capacity to convert broad demographic descriptors, such as race and SES, into socially relevant geographic units that would self-identify as a community.^{1, 45} Additionally, community-based interventions should take into account the “spillover” effects of the surrounding and wider community given social interactions among individuals in surrounding communities, which can influence the effectiveness of community-based interventions.

Despite the continued growth of data resources, describing the variation of CVD disparities in some communities remains challenging. Vital statistics data and representative sample surveys, like the National Health and Nutrition Examination Survey (NHANES), the Behavioral Risk Factor Surveillance System (BRFSS), and the National Center for Health Statistics (NCHS) Linked Mortality Files and Compressed Mortality Files, provide a richly detailed description of CVD morbidity and mortality and related risk factors by age, sex, race/ethnicity, and – to a lesser extent – geographic region. State and county maps are available for cardiovascular risk factors and both non-fatal and fatal CVD events.⁴⁶ A recent joint effort by the Centers for Disease Control and Prevention (CDC) and Robert Wood Johnson Foundation (RWJF) has led to creation of a rich on-line database with health and social status information for the largest 500 US cities.⁴⁷ Likewise, health departments in large metropolitan areas have vital statistics and risk factor data from more homogenous neighborhoods. In Chicago, for example, data are provided on 77 “community areas”, demonstrating higher CVD burden on predominantly African Americans residing in the south and west side communities, compared to more affluent communities on the north-west and south-west sides; further detail is available on census tracts, zip codes and “neighborhoods”.⁴⁸ Other useful resources recommended for NHLBI to consider are shown in Table 2A.

TABLE 2A.

High Burden Communities^*

Opportunities	Challenges	Strategies/Leverage	Partnerships	Useful Resources
Identify and prioritize high burden communities by county, census tract, zip codes, rural, or non-metropolitan status Prioritize persons living in poverty; those with low literacy Prioritize racial/ethnic minorities, especially African Americans, American Indians/Alaska Natives, and Hispanic/Latino population groups with high CVD risk Recognize examples of community resilience Identify supportive community resources Assess and leverage self-awareness of existing community leaders Assess community needs, readiness, and desire to address CVD disparities Leverage disparities data on country of birth, nativity, and acculturation, limited English proficiency Overlay datasets from different sources, whenever possible	Limitations of county-level data for large metropolitan areas Community distrust of research Loss of precision of census data particularly during inter-censal years in small communities Adverse social determinants of health and multi-morbidity Cultural variation and need for support in health literacy Feelings of helplessness, and hopelessness Dependency on others’ awareness and support Prejudice, discrimination, overt and institutional racism Inadequacy and inconsistency in access to health care Disability from stroke and lower extremity amputation associated with diabetes-magnifying hopelessness and dependency	Insist that researchers recognize the community’s self-identification and needs assessment Create solutions which support lifespan wellness of individuals and communities Mitigate contextual challenges (remote areas) Employ social media in both rural and urban settings Rely on members of the community as cultural and context interpreters Identify community organizations where the patients and their supporters gather—extend care beyond the primary care office Recognize and leverage strength in diversity View community health more holistically Use community-centric grant supplements to support technical assistance to communities Identify resilient communities Improve health literacy	Identify partners at the local and state levels Foster upward mobility of self-determined community leaders Incentivize patients to be better able to self-manage and help others Challenge researchers to help community develop more resilience Have an open, and on-going call for partners Recognize partners and acknowledge their contributions Leverage virtual connectivity to garner energy and trust from like communities Integrate community members into the research at an early stage	Vital statistics data and representative sample surveys e.g.: ○American Community Survey, ○BRFSS ○NHANES 500 Cities: Local Data for Better Health CMS Geographic Variation Dashboard CMS Medicare Chronic Condition Dashboard CDC Atlas of Heart Disease and Stroke County Health Rankings and Roadmaps

BRFSS: Behavioral Risk Surveillance System; CDC: Centers for Disease Control and Prevention; CVD: Cardiovascular Disease; NHANES: National Health and Nutrition Examination Survey.

^*Workshop participants recommended that NHLBI consider these activities

In many instances, county-level data may be uninformative for large metropolitan areas. For example, Cook County, Chicago includes both the city of Chicago (high burden) and a large number of suburban communities (low burden). On the other hand, disparities in sparsely populated rural areas can be easily identified with county maps. Moving down to the level of geographic units traditionally targeted for community interventions, however, can lead to a loss of precision since census estimates may be inaccurate in the inter-census years and the limited number of cardiovascular events to be expected within a single community may make rates unstable. The most robust approach to characterizing communities with CVD disparities might be to create a composite picture using information from broad demographic descriptors (e.g., race/ethnicity), regional or state vital statistics, as well as local survey data. It is ideal for all interventions targeted at small scale population units to begin with intensive efforts to engage the community at all levels, including efforts to validate and interpret available CVD disparity data.

Communities may also manifest disparities in access to quality health care delivery even if their actual rate of CVD is lower than that of other communities. For example, in a risk factor and health status survey in 28 communities located in 17 states, Liao et al. reported that Hispanics had the lowest likelihood of having had cholesterol or glycosylated hemoglobin checked in the preceding year, and the lowest prevalence of taking medications for hypertension.⁴⁹ Influenza and pneumococcal vaccination rates in eligible adults were also lower in black, Hispanic, and Asian or Pacific Islander communities.⁴⁹ In addition, the 2016 National Healthcare Quality and Disparities Report demonstrated multiple examples of worse access to quality health care especially for Blacks, Hispanics, American Indians and Alaska Natives compared to Whites; and also poor access to quality health care for poor persons (defined by the Federal Poverty Level) and those from low-income households⁵⁰

Commitment to Best Practices in Community Engagement

Community-engagement research, including community-based participatory research (CBPR), involves partnerships with multiple stakeholders within a community where all are important players in identifying community problems, designing the intervention, determining how and when to intervene, what data to collect, how to implement and evaluate the intervention, interpret and disseminate the results, and put evidence-based results into practice.^51–53 To reduce health disparities, a community-based approach is considered essential. Important additional principles critical for community-engaged health disparities research include: building trust among research institutions, community partners and other stakeholders; utilizing multi-prong, multi-level approaches, even though this strategy may present challenges in measuring the effect of any single intervention; tailoring interventions to the community context, which requires trade-offs between researchers’ priorities and community needs; identifying and valuing diversity within communities; and using local resources and capacities to ascertain long-term commitments.⁵⁴

Several tools and resources available to support community stakeholders in efforts to reduce CVD disparities were recommended for NHLBI to consider. The RWJF “Action Cycle” for Improving a Community’s Health (Figure 3) is one example that describes key activities for all partners within a community and provides additional resources for taking action. The related County Health Rankings^{55, 56} is another resource that community stakeholders can use to design community needs assessment and program planning. For needs assessment, the rankings are helpful to identify health determinants such as health behaviors, clinical care settings, social and economic factors, and the physical environment. For program planning, it is imperative to identify specific health outcomes. The Health Rankings uses 2 categories: death and health status while alive. A current example of this resource being used is the community health assessment that is required by the Affordable Care Act of hospitals, local public health agencies, and Federally Qualified Healthcare Centers.⁵⁷ These community assessments have led to many creative community solutions that may serve as examples for large scale interventions.⁵⁸

Figure 3. The Robert Wood Johnson Foundation Action Cycle for Improving a Community’s Health.

At the core of the Action Cycle are key stakeholders for taking action. Each step on the Action Cycle is considered a critical component for creating healthier communities and offers a guide that describes key activities within each step that includes suggested tools, resources, and additional reading. In this Action Cycle, “Work Together” and “Communicate” are distal because they are needed throughout the Cycle. Reproduced from The Robert Wood Johnson Foundation County Health Rankings & Roadmaps Action Center, 2017; Take Action Cycle. http://www.countyhealthrankings.org/take-action-cycle.

The workshop discussions affirmed several important opportunities and challenges for NHLBI to consider that are summarized in Table 2B. Crucial among these are strategies to build trust and sustainable partnerships, as shown in the Community-Campus Partnership for Health model.⁵⁹ The workshop also stressed the importance of empowering communities to make decisions on what matters most to their health; ensuring that researchers have the cultural humility and humanity to accept these decisions; embracing the crucial role of social determinants of health; and approaching community health through a comprehensive life-course lens.⁶⁰ The workshop participants recommended that the NHLBI consider strategic partnerships with other NIH Institutes and Centers, with Patient-Centered Outcomes Research Institute (PCORI), Centers for Medicare & Medicaid Services (CMS), RWJF, CDC, and other organizations with experience in funding CBPR. It is reassuring to know that recent large community-based practice network and patient-centered interventions addressing health disparities are yielding some success and important clinical and public health insights.^61–64

TABLE 2B.

Engaging Community Stakeholders^*

Opportunities	Challenges	Strategies/Leverage	Partnerships	Useful Resources
Seek opportunities for building trust Embrace cultural humility and humanity Provide tool-kits and resource guide for community partners Provide training on community engagement strategies and best practices (FAQs, webinars, etc.) Fund more engagement/ relationship research vs. transactional research Seek compromise, not consensus Create research funding opportunities that pair communities with high burden of disease with the funded-researchers who can support them	Program sustainability to include local businesses, government, etc. Lack of shared resources and engagement between researchers and Community Members Increased costs for funding both research and community engagement activities	Create a “Community Engagement” Study Section for grant reviews Add specific grant application review criteria/elements for Community Partners Increase reviewer training opportunities Include more junior and early-stage investigators in the application review process Encourage use of administrative supplements to support lapses in funding community-based research	Promote/reward partnerships for sustainability Create more community engagement partnerships Host workshops/town hall meetings solely for community stakeholders and partners at NIH Engage and promote usage of community advisory boards within funded research projects Create partnership opportunities to improve health literacy	Community Campus Partnership for Health (CCPH) model CTSA Consortium Community Engagement Key Function Committee Task Force on the Principles of Community Engagement Community-Based Public Health Caucus RWJF Culture of Health Community Engagement Models from PCORI CDC, ATSDR Models and Frameworks for the Practice of Community Engagement Loyola Univ. Certification Program on Community Engagement NIEHS Community-Engaged Research and Citizen Science PEPH Resource Center CPBR academy resources AHRQ Activities Using Community-Based Participatory Research to Address Health Care Disparities

AHRQ: Agency for Healthcare Research and Quality; ATSDR: Agency for Toxic Substances and Disease Registry; CBPR: Community-Based Participatory Research; CTSA: Clinical and Translational Science Awards; FAQ: Frequently Asked Question; NHLBI: National Heart, Lung, and Blood Institute; NIEHS: National Institute of Environmental Health Science; PEPH: Partnerships for Environmental Public Health; RWJF: Robert Wood Johnson Foundation; PCORI: Patient-Centered Outcomes Research Institute; PRIDE: Programs to Increase Diversity Among Individuals Engaged in Health-Related Research;

^*Workshop participants recommended that NHLBI consider these activities.

Implementation of Evidence-Based Interventions

Many effective interventions have been developed in recent decades to enhance CVH and mitigate CVD disparities.^{65, 66} For example, the Franklin County Cardiovascular Health Program (FCHP), which is a comprehensive community-based integrated primary health care intervention program, targeted hypertension, cholesterol, smoking, diet and physical activity over a 40-year period and demonstrated significant reductions in CVD mortality in a low-income, rural community of Maine.⁶⁷ Other examples of effective interventions include: the Dietary Approaches to Stop Hypertension (DASH) diet and self-management strategies for at-risk patients with hypertension, diabetes, and other CVD risk factors (at the individual level); peer support interventions (such as those delivered by community health workers) and barber shop/beauty parlor interventions (at the family, peer, and social network level); nurse and pharmacist-delivered care management, and provider communication skills training (at the healthcare provider level); and use of electronic medical records, tele-monitoring, and quality improvement methodology to improve patients’ blood pressure levels (at the organizational level).⁶⁸ Furthermore, early-life intervention programs about healthy lifestyles and CVH that target high burden communities through schools can be effective in reducing CVD disparities in adulthood.^{69, 70} Importantly, active engagement of, and coordination with the healthcare system should be fostered so that safe and effective medications for the treatment and control of CVD and risk factors can be maintained.

Despite the availability of these effective interventions, substantial challenges in reducing CVD disparities in the US population persist. Important gaps in research and translation also remain. If effective evidence-based interventions are to be implemented and disseminated, it is critical that these interventions address the complex, multilevel factors that influence the presence of these disparities and that gaps in translation of these interventions into practice and policy are addressed.^{65, 66, 68, 71} The persisting challenges in eliminating CVD disparities may be attributed, in part, to the paucity of interventions that address social determinants of health disparities.^{72, 73} Indeed, the health impact pyramid described by Frieden suggests that although implementing interventions at all levels of the ecological model can achieve the maximum possible sustained public health benefit, interventions with the greatest impact on population health are those focusing on the physical and social environmental context and on socioeconomic and policy factors.⁷⁴

Bridging the translation gap will require transdisciplinary research that tests the effectiveness of multi-level interventions; compares universal interventions alone to interventions targeting at-risk groups; describes the challenges to program implementation, affordability, sustainability, translation, and dissemination (for example, by tailoring interventions to context and populations); incorporates broad stakeholder engagement in design, implementation, and dissemination of evidence-based strategies; and addresses social structures and policies.^{71, 72, 75, 76} Table 2C describes the opportunities provided, challenges to success, and strategies for leverage, as well as important partnerships and relevant resources to unleash the potential for evidence-based interventions to eliminate CVD disparities that were recommended for NHLBI to consider.

TABLE 2C.

Evidence-Based Interventions^*

Opportunities	Challenges	Strategies/Leverage	Partnerships	Useful Resources
Prioritize CE research as important in the NIH mission Launch a bold, disruptive movement to make HTN a national priority with clearly defined goals Promote cross-training in rural communities to increase healthcare capacity Focus FOA requirements for CE research on risk factors of chronic disease instead of specific pathology Highlight all-cause mortality data, when available, may be as important as CVD mortality	Sustainability after funding ends Few interventions focused at provider, policy, community, family, and org levels Must engage at many levels at once and over a long period for effectiveness Although recruitment and retention efforts are key to research effectiveness and sustainability, they are not carefully scrutinized during review Difficult to conduct research in high-burden, low-resource communities Academic reward structure inhibits CE research	Encourage multi-level, phased, hybrid research designs for CE and IS research Offer more flexibility for CE research grants that encourages testing multiple intervention strategies Align multi-level research approach and partnership efforts with community priorities/ needs Be more engaged with and accessible to communities Include success factors: ○Train the trainer models, super-users ○Use of existing staff ○Peer models ○Clinical champions Engage leadership in entire process	Create more FOAs that span NIH ICs around linked comorbidities Identify community champions as strategic partners for advancing intervention adoption and sustainment	Research to Action Grants PCORI/PCORNet DASH diet CDC Community Guide Database of Abstracts of Reviews of Effects (DARE) Cochrane Reports Other Systematic Reviews

CE: Community-engaged; HTN: hypertension; FOA: Funding Opportunity Announcement; CVD: cardiovascular disease; IS: implementation science

IC: NIH Institutes and Centers;

^*Workshop participants recommended that NHLBI consider these activities

Aligning Community Partners and Care Networks

The number of organizations and partners that might collaborate to eliminate CVD disparities may seem daunting. Hospital and outpatient clinics, office-based primary care practices, pharmacies, health insurance companies, community organizations, and patients have important roles to contribute to decrease CVD disparities. It is crucial to engage all the potential partners as strategic collaborators in implementation research to address the full spectrum of CVD, from prevention and risk factor reduction to diagnosis and treatment. This engagement of stakeholders is necessary to ensure that the implementation research undertaken is relevant and meaningful in the local community.⁷⁷

Several approaches to engaging these strategic partners can help align the mission and goals of implementation research to eliminate CVD disparities. Health Maintenance Organizations, Accountable Care Organizations, and practice-based research networks are models for linking diverse medical clinics, hospitals, and pharmacies across large geographic areas to test, implement, and disseminate locally relevant, meaningful messages and programs to improve CVD care.^78–83 National organizations working with state public health departments and their parallel county organizations as well as healthcare workforce training programs can all find alignment in community-based approaches to CVD risk reduction and treatment.⁸⁴ Engaging and including patients and community members in all aspects of implementation research also aligns care with local culture and community values.⁸⁵

Recently, there has been a resurgence in the desire to link medical care with public health through the creation of local and regional Communities of Solution (COS) that identify the critical links among community, public health, and primary care and call for ongoing demonstrations of COSs grounded in patient-centered care.^{86, 87} The COS begins by identifying the local “problem shed”, in this case, issues related to CVD disparities, and then identifies the “asset shed”, that is, those people, organizations, and health systems that can contribute to a local solution.^{86, 87} The COS combine the key elements of implementation research by aligning strategic partners in developing local solutions to local health problems. In addition to the use of local and regional COS in implementation research, several opportunities and challenges in efforts recommended for NHLBI to consider for eliminating CVD disparities, are identified in Table 2D.

TABLE 2D.

Aligning Networks of Care^*

Opportunities	Challenges	Strategies/Leverage	Partnerships	Useful Resources
Build upon historical research and primary care partnership framework Harness the payor framework for care delivery Establish a lifetime of wellness at individual and group level Identify traditional and new linkages, upstream for prevention, downstream for treatment Magnify impact, when treating one disorder is a means to prevent other diseases Apply a holistic approach with organization and system-level interventions Support the creation of local and regional Communities of Solution	Implementation fatigue Overburdened healthcare system Misalignment between networks and the culture of the community they serve Recruitment and retention of clinical team members Lack of comfort in pursuing participatory approach to patient care	Foster a cultural shift—find a new normal that advances CVD disparities research Understand the cultural context of the science Focus on communication, health literacy Acknowledge potential benefit of existing FQHCs EHRs and 304B pharmacies Use telemedicine as an option to foster self-managed care, mitigate trips to clinical offices Clearly articulate intent and intended benefit Measure to track impact Recognize strength in diversity Pursue community’s priorities Encourage adaptability and balance local adaptation vs. fidelity Use practicalities and efficiencies to inform research design rather than the extremes of research possibilities Understand current workflow and care delivery Understand how community manages change Plan for changes in personnel and resources Adopt a phased research approach to allow for smaller scale testing of the intervention Leverage the PBRN network to foster community engaged CVD research	Engage skeptics Extend reach Foster approach to erode all health disparities Plan for replication Conduct science for the probable, not the possible Foster viable skillsets at the individual and community level Promote Team Science Acknowledge gaps and advancements	AHRQ Practice-Based Research Networks AHRQ Boot Camp Translation: A Tool to Engage Communities for Patient-Centered Outcomes Health Literacy toolkit and Implementation Guide AHRQ Training Modules for New Facilitators and Their Trainers Virginia Commonwealth University Center on Society and Health, Detroit Community-Academic Urban Research Center Practice Facilitation Certificate Program

AHRQ: Agency for Healthcare Research and Quality; CVD: Cardiovascular Disease; HER: Electronic Health Record; FQHC: Federally Qualified Health Center

^*Workshop participants recommended that NHLBI consider these activities.

Incorporation of Novel Research Methods and Milestones

A set of core principles guided the discussion of methods and milestones to evaluate interventions. These principles included: (1) using a variety of research designs to assess the effectiveness of evidence-based interventions;⁸⁸ (2) the methods that implementation research scientist use in communities with CVD disparities must be low-burden and aligned with community priorities;⁸⁹ community partners must be integral co-investigators; (3) the implementation research scientist must strive for equity and fairness in the amount of scientific information produced to understand the causes and solutions to health disparities;^{90, 91} and (4) implementation research represents an essential stage of research for improving population health and eliminating health disparities.

Implementation research hinges upon building partnerships between communities and researchers.⁹² Effective and sustainable partnerships are the result of achieving mutual self-interest, often through the process of cultural exchange.⁹³ Researchers need to be mindful that the failure of an implementation effort comes at a significantly greater cost to the community than to the academic partner. Thus, the methods used need to be responsive to and in harmony with community needs and context and ought to further the goal of creating a sustainable, community-owned practice. Additionally, milestones such as the establishment of systematic engagement of diverse community stakeholders; agreement regarding the method to conduct health needs assessment; consensus regarding community priorities; consensus regarding intervention design; selection of outcomes measures; and sustainability of the program after funding has ended should all be discussed.^94–98

Evaluation of an implementation strategy to create translatable knowledge can be conducted within a single site or multiple sites (clusters) or with a randomized or non-randomized design.^{88, 99} For example, use of the stepped-wedge design, in which all communities or sites receive the intervention in a phased approach, can be a helpful strategy to maintain community engagement when the alternative of potential randomization to a non-intervention comparison group is viewed unfavorably.^100–102 Researchers may use low-burden measures to assess the quality, quantity, speed, and extent of implementation¹⁰³ in the real-world systems and find ways that are practical, feasible, and align with the needs and mission of the service provider or community. Unobtrusive measures,¹⁰⁴ which involve data sources already being collected during service delivery, can be invaluable. Machine learning methods involving automated coding and validation of unobtrusive measures can be used for monitoring and feedback to identify service delivery challenges and how organizations can respond through rapid data-driven decisions.¹⁰⁵

In addition to the County Health Rankings mentioned above, participants discussed other potential evaluation metrics, including, for example, the Life’s Simple Seven metrics promoted by the American Heart Association as key indicators of success of their efforts to promote the CVH of all Americans.¹⁰⁶ Cost, the single most important factor in decisions about program adoption, also remain critically important for spread and sustainment.¹⁰⁷ Because costs can differ substantially across organizations and communities, tools like a cost calculator can be invaluable.¹⁰⁸ Table 2E shows other opportunities, strategies, and challenges in implementation research methods and milestones that were recommended for NHLBI consideration. Important remaining gaps include tools for implementation measurement systems¹⁰⁹ and for calculating statistical power in research designs.

TABLE 2E.

Methods and Milestones^*

Opportunities	Challenges	Strategies/Leverage	Partnerships	Useful Resources
Support methodological/statistical designs Encourage programs that support entire CE research community Focus on implementing established CE research Use validated models and frameworks such as those taught at the IRI Fund studies that integrate research findings into current structures for increased sustainability Encourage paradigm shift around CE research that supports a community-led partnership rather than as investigator-led partnership Create D&I toolbox: promising practices, evaluation tools, fidelity testing, power calculation guide, D&I contacts, etc. Create roadmap curricula: standardized training requirements, IS-related career opportunities Support use of standardized measures of social determinants	In defining CE research priorities and approaches, both scientific equity and health equity need consideration Expectations for assessing implementation fidelity and importance of evaluation/ reporting/info sharing Misalignment between quality measures/ evaluation strategies and current practice requirements that increase administrative burden Evaluation of parameters needed to be explained to community partners	Focus CTSAs on late-stage translational research and incorporate CE cores Recognize that small N studies can be informative Distinguish scaling out versus scaling up Budget impact analysis to assess cost to install, run, sustain Distinguish differences between clinical, implementation, and intervention fidelity Identify un-obtrusive measures for fidelity assessments Communicate sustainability strategies to community Leverage the EHR and other data sources: ○Crosslink across EHR systems ○Leverage CTSA’s data warehouse ○Project REDCap	Form partnerships with IT, FQHCs, and safety net providers Create and sustain implementation demand with investigator/community partner development, CE, and dissemination cores Leverage IRI Leverage NIDA Ce-PIM for implementation research training, IS 101 webinar; use as NHLBI IS Center	Implementation Research Institute (IRI) Center for Prevention Implementation Methodology (Ce-PIM) Academic Centers focused on Dissemination and Implementation Research CDC Prevention Research Centers Project REDCap Partnerships for Environmental Public Health Metrics Manual PCORI’s N of one research resource User guide for N of 1 trials AHRQ Activities Using Community-Based Participatory Research to Address Health Care Disparities

AHRQ: Agency for Healthcare Research and Quality; CE: Community-engaged; Ce-PIM: Center for Prevention Implementation Methodology; CTSA: Clinical and Translational Science Awards; D&I: Dissemination and Implementation; IRI: Implementation Research Institute; NIDA: National Institute on Drug Abuse; NHLBI: National Heart, Lung, and Blood Institute; REDCap: Research Electronic Data Capture

^*Workshop participants recommended that NHLBI consider these activities

Training the Next Generation

Innovative and “out-of-the-box” training models that build upon inter-institutional collaborations could help to attract, develop, and sustain a cadre of implementation research scientist trained in CBPR to reduce CVD disparities. Such collaborations can leverage research infrastructure and investments to train a diverse workforce that is more representative of the community being targeted. For example, the inter-institutional Clinical and Translational Science Awards (CTSA) Program at Emory University and Morehouse School of Medicine (MSM) is a model of two institutions collaborating to train scholars from diverse backgrounds.^{110, 111} Challenges include the limited number of training slots and even fewer trained mentors. The National Institute on Minority Health and Health Disparities (NIMHD) sponsored Clinical Research Education and Career Development (CRECD) scholars could serve as a pipeline of a robust cadre of implementation research scientist focused on health disparities.¹¹² Several CRECD scholars at MSM, the University of Puerto Rico, and across the Research Centers at Minority Institutions (RCMI) have developed successful academic careers addressing implementation research at the intersection of primary care, health care systems, population health and faith communities. These are poised to scale NHLBI implementation research training.¹¹² Programmatic support for teams that will surround these scholars and enable them to pursue implementation research in collaboration with health care systems and community partners was considered crucial.^{113, 114}

Research and training infrastructure such as the National Research Mentoring Network (NRMN) provides evidence-based mentor training and mentee career development to diversify biomedical research.¹¹⁵ Short-term training programs, modeled after the NHLBI and Office of Behavioral and Social Sciences Research (OBSSR)-supported Annual Summer Institute on Randomized Behavioral Clinical Trials¹¹⁶ and the NIH-supported Training Institute in Dissemination and Implementation Research in Health (TIDIRH),¹¹⁷ may help stimulate interest in further training and launch careers. However, more intensive and extended training will likely be necessary to build skills, especially those related to community engagement. Two such programs are currently funded in Dissemination and Implementation (D&I) research, but are not supported by NHLBI. They include the Implementation Research Institute (IRI), funded by the National Institute of Mental Health (NIMH), National Institute on Drug Abuse (NIDA), and the Department of Veterans Affairs (VA); and the Mentored Training in Dissemination and Implementation Research in Cancer (MT-DIRC), funded by the National Cancer Institute (NCI), the VA, and the Cancer Research Network. Both TIDIRH and the IRI training programs have been shown to increase D&I grant submissions and success in obtaining funding.^{117, 118} Unfortunately, available slots in these D&I training programs are insufficient to meet demand within the scientific and practitioner community.¹¹⁹ Schools of Public Health may be especially well positioned to provide the required combination of didactic and experiential training. Other opportunities and resources that could be leveraged to support training and career development in advancing implementation research for the elimination of health disparities recommended for NHLBI consideration, are shown in Table 2F.

TABLE 2F.

Research Training^*

Opportunities	Challenges	Strategies/Leverage	Partnerships	Useful Resources
Establish a road map for career independence in implementation science for health equity Establish a set of core competencies for community engaged research Develop a mid-career K award that includes protected time and specific professional development opportunities Establish training programs for K-12 science teachers Start the implementation research training early in research career Attract the younger generation to the field of implementation science by using their language and/or modes of communication in the messaging	Few and non-uniform implementation science degree programs exist Community engagement is difficult to teach and learn - it requires “real-world” and experiential learning opportunities Building community trust is time-intensive “Light touch” mentoring is not effective High costs to train the trainer/mentor, support under-resourced communities, and sustain the effort Trainee tracking is required to assess impact Support for mentoring relationships that start in high school/college or medical school and last decades.	Invest in mentors and their individual professional development (reboot mid and late career) as they work to attract/retain other trainees and provide protected mentor time Provide administrative supplements for community engagement modeled after the NHLBI Diversity Supplement Program Promote the integration of implementation science into degree tracks at the schools of public health (e.g. MPH and DrPH) Leverage resources and talent from CTSAs, Research Centers for Minority Institutions (RCMI), NIMHD-funded Centers of Excellence, etc. Develop training programs where professional growth of the ESI and the implementation science mentor happen concurrently Support training research awards that allow the funding for the trainee to move with him/her as s/he travels the road to career independence	Develop an initiative that promotes partnership between workforce talent from MSIs and PWIs to focus on implementation research for health equity Engage all key stakeholders through an integrated partnership that includes non-profit agencies, professional societies, community based organizations, state and local government	National Research Mentoring Network Training Institute for Dissemination and Implementation Research in Health (TIDIRH) Implementation Research Institute NHLBI and other NIH Institutes and Centers Diversity Supplement Program Community Leadership Institute of Kentucky (CLIK) Johns Hopkins Bloomberg School of Public Health DrPH Program in Implementation Science RWJF Harold Amos Medical Faculty Development Program Univ. of Washington's Population Health Initiative Research in Implementation Science for Equity (RISE) – A PRIDE Program at UCSF CDC Science Ambassadors High School Scientific Training and Enrichment Program (HiSTEP) NIH Summer Internship Program in Biomedical Research (SIP) NCI CURE program, The Continuing Umbrella of Research Experiences (CURE) NIMHD Minority Health and Health Disparities Research Framework National Junior High School Scholars NIDDK Minority Access to Careers in Science

CTSA: Clinical and Translational Science Awards; DrPH: Doctor of Public Health degree; MPH: Master of Public Health degree; MSI: Minority Serving Institution; PRIDE: Program to Increase Diversity among Individuals Engaged in Health-Related Research; PWI: Predominantly White Institutions; RWJF: Robert Wood Johnson Foundation; UCSF: University of California San Francisco.

^*Workshop participants recommended that NHLBI consider these activities

An important area is training and educational opportunities for high school and middle school students to grow the pipeline of new investigators. Over the years, it has become increasingly apparent that creating a pipeline for young investigators and public health professionals that starts in middle school can be particularly productive. The CDC supports a number of initiatives including the Science Ambassadors Fellowship,¹²⁰ the Science Olympiad,¹²¹ and other career paths to public health¹²² that could be leveraged to grow the pipeline of new racial and ethnic minority investigators.

The workshop participants made a strong recommendation for the NHLBI to consider exploring and encouraging increased use of Diversity Supplements to both engage and train a cadre of Early Stage Investigators (ESI) that are underrepresented in medicine or research (URM/URR) and are also engaged in health disparities research. It was suggested that NHLBI consider prioritizing Diversity Supplements that demonstrate how ESIs from URM/URR groups can advance their career as independent researchers. The cornerstone of the program is teaching these early career investigators how to effectively and equitably engage community partners and community stakeholders in implementation research to reduce CVD disparities.

Importance of Strategic Collaborations

Whereas the focus of this Workshop was on CVD disparities, participants recognized the substantial extent to which the social determinants of CVD disparities are shared with other health conditions. This reality underscores the potential synergy to be gained through wide collaboration. Several Institutes, Centers, and Offices at NIH have resources and programs that can be invaluable in the effort to reduce and eliminate CVD disparities. For example, NIMHD supports a research framework that incorporates multiple health determinants over the life course.¹²³ Based on the fundamental pillars of race/ethnicity and SES, NIMHD supports research that explores how biology, behavior, the built environment, culture and community, and the role of the healthcare system, influence health outcomes. Understanding the importance of community-driven interventions can inform CVD disparities research. NIMHD supports several research activities involving community health workers, the use of mobile technology in risk reduction, and CBPR that utilizes a health behavior intervention to increase community recognition of stroke warning signs and follow up instructions engaging healthcare professionals.

Insights gained from these research activities can be leveraged in planning CBPR for reducing and eliminating CVD disparities. The workshop participants encouraged NHLBI to consider collaborating actively with NIMHD and other NIH Institutes, Centers, and Offices to address research gaps in understanding health disparities and implementation research to reduce CVD disparities. For example, how might enhancing access to health care services like portals for patients, e-referrals and telemedicine reduce cardiovascular disparities? Insights and lessons learned from several NIMHD funding opportunity announcements¹²⁴ may be invaluable to NHLBI as it charts the future for health disparities research as part of its Strategic Vision.

The Prevention Research Centers (PRC) (www.cdc.gov/PRC) provide a unique opportunity for NHLBI, CDC, and Foundations to support CBPR focused on the elimination of cardiovascular disparities. The PRCs constitute a network of 26 Schools of Public Health and Schools of Medicine with preventive medicine residency programs that focus on community-based applied public health research. This research includes both investigator initiated and sponsored research that could be supported by CDC, NIH or a private foundation. Groups of PRCs have come together in a series of thematic networks in the areas of cancer prevention, obesity, physical activity and nutrition, epilepsy, workplace health promotion and healthy aging. However, there is not a specific thematic network focused on the elimination of cardiovascular disparities. Bringing together a robust group of researchers to collectively develop and implement such a research agenda could be particularly fruitful. In addition, the AHRQ Practice Based Research Networks (PBRNs) provide yet another important framework that could be used to support community based multi-sector research with a strong grounding in active community-engagement and community empowerment.

Potential Solutions, Remaining Questions, and Gaps

Tables 2A–F provide a comprehensive and detailed description of both the scope and breadth of the challenges, opportunities, and resources needed for conducting community-engaged implementation research to reduce cardiovascular disparities. Potential solutions for NHLBI to consider fall within four over-lapping domains: 1) support of true community-based, community-participatory implementation research projects; 2) strategies to advance innovative improvements in care delivery within health systems; 3) strategies that address the social determinants of health and structural changes in the built environment in communities; and 4) training a diverse workforce appropriate to the regional population and skilled in the reduction of cardiovascular disparities in strategic partnerships with community stakeholders. A detailed list of specific examples from these domains with enough granularity appropriate for different communities is beyond the scope of this workshop report; however, at the general level, these potential solutions focus on:

1. Supporting highly meritorious community-engaged implementation research initiatives;

2. Harnessing cutting-edge information sources and analytic methods to identify high-burden communities that are receptive to change;

3. Supporting enduring and effective community engagement policies and practices;

4. Developing and testing models for integration and delivery of evidence-based interventions;

5. Nurturing innovative efforts to align community-based organizations, public health agencies, and health care systems;

6. Identifying, vetting, and promoting the use of appropriate methods and metrics for study conduct and evaluation;

7. Implementing scalable approaches for training the needed current and future generations of the workforce; and

8. Supporting the development of strategic partnerships between research investigators, their institutions and centers, and community stakeholders.

While attention to the details inherent in these potential solutions could result in substantial improvement in health and health equity, many important questions and knowledge gaps remain. The best way to engage communities that may be distrustful and skeptical of research projects remains a challenge. While much progress has occurred in documenting and understanding the levels of CVD disparities by sex, race/ethnicity and geography; the impact that environment has on CVD disparities; early life exposures (including intrauterine development); the social and psychological determinants of CVD disparities; and methods to collect accurate data to evaluate if these factors have been modified, major challenges for community research still remain. Furthermore, few interventions have specifically leveraged these influences for reducing cardiovascular disparities in communities. While we have effective interventions to consider for the major CVD risk factors, most of these interventions have been evaluated in isolation. Whether these interventions alone or integrated into more comprehensive CVH promotion programs will specifically work to reduce health disparities remains less clear and untested. There is no readily available compendium of interventions, and few interventions have addressed the whole socioecological structure including patients, family, schools, community organizations, health care providers, health and social policy, and other higher organizational levels.

Successful engagement of communities, key stakeholders, and researchers in sharing resources for community-based research is also a major challenge. The research should align with the culture of the community where it is being conducted, keeping in mind the difficulties with conducting research in high-burden, low-resource communities. Often there may be inadequate and inconsistent access to the health care and public health systems within an overburdened community, particularly if many members are uninsured. A critical question that remains is how to align evidence-based research with current practice requirements without increasing the administrative burden. For example, the identification of simple, yet responsive, measures of environmental and social determinants of health that can be easily incorporated into existing community surveillance programs and electronic health records, would represent a significant advance. In addition, standardized measures of race/ethnicity, SES, education, and other social and environmental determinants of health and health care need to be incorporated into the electronic medical record. Whereas some Schools of Public Health are beginning to train implementation research scientists, the output has not been sufficient to date, in part because many competencies related to community-engagement research are difficult to teach and require substantial time and commitment from both the mentors and trainees. The success of efforts to reduce and eliminate CVD inequities would be enhanced by the development of innovative training programs and venues for sharing best practices.

There are several important themes that constitute remaining questions and gaps in cardiovascular disparities that the workshop did not have time to address. For example, dementia occurs at higher rates in racial and ethnic minority populations in the US.^125–127 Although educational attainment appears to play a very significant role, place of birth and the higher prevalence of vascular disease and risk factors likely also contribute to the greater susceptibility to dementia, particularly among African Americans.¹²⁵ Recent evidence from the Framingham Heart Study showing a downward trend in dementia in the last three decades strongly suggests that it is possible to improve the risk pattern leading to this devastating condition.¹²⁸ Continued attention to these remaining questions and gaps in disparities research is important.

Summary and Conclusions

Despite remarkable declines in age-adjusted rates of CVD mortality in the US over the past five decades, CVD disparities remain pervasive. Few large-scale intervention studies have been conducted to examine the feasibility of reducing or eliminating these disparities. To address this challenge, on June 22-23, 2017, the NHLBI convened experts from a broad range of biomedical, behavioral, environmental, implementation, and social science backgrounds to summarize the current state of knowledge on CVD disparities and propose intervention strategies for consideration that align with the research, training, and education mission of NHLBI. As summarized in this report, workshop participants developed a set of themes that identified challenges, opportunities, and resources needed to inform and advance research to decrease CVD disparities. They also identified novel approaches for community engagement and community-participatory implementation research. As the NHLBI proceeds with implementing its Strategic Vision,³⁰ actions to address the themes that emerged from this Workshop should support the Institute’s mission-focused strategic objectives including the reduction or elimination of cardiovascular disparities.

Non-standard Abbreviations and Acronyms

AHRQ

Agency for Healthcare Research and Quality

ATSDR

Agency for Toxic Substances and Disease Registry

BRFSS

Behavioral Risk Factor Surveillance System

CBPR

Community-based Participatory Research

CDC

Centers for Disease Control and Prevention

CMS

Centers for Medicare & Medicaid Services

COS

Communities of Solution

CRECD

Clinical Research Education and Career Development

CTRIS

Center for Translation Research and Implementation Science

CTSA

Clinical and Translational Science Awards

CVD

Cardiovascular disease

CVH

Cardiovascular health

D&I

Dissemination and Implementation

DASH

Dietary Approaches to Stop Hypertension

ESI

Early Stage Investigators

FAQ

Frequently Asked Question

FCHP

Franklin County Cardiovascular Health Program

FOA

Funding Opportunity Announcement

IRI

Implementation Research Institute

MSM

Morehouse School of Medicine

MT-DIRC

Mentored Training in Dissemination and Implementation Research in Cancer

NCHS

National Center for Health Statistics

NCI

National Cancer Institute

NHANES

National Health and Nutrition Examination Survey

NHLBI

National Heart Lung, and Blood Institute

NIDA

National Institute for Drug Addiction

NIEHS

National Institute of Environmental Health Science

NIMH

National Institute for Mental Health

NIMHD

National Institute on Minority Health and Health Disparities

NRMN

National Research Mentoring Network

PBRN

Practice Based Research Networks

PCORI

Patient-Centered Outcomes Research Institute

PEPH

Partnerships for Environmental Public Health

PRC

Prevention Research Centers

PRIDE

Programs to Increase Diversity Among Individuals Engaged in Health-Related Research

RCMI

Research Centers at Minority Institutions

RWJF

Robert Wood Johnson Foundation

SES

Socioeconomic status

TIDIRH

Training Institute in Dissemination and Implementation Research in Health

URM/URR

Underrepresented in Medicine/Underrepresented in Research

US

United States

VA

United States Department of Veterans Affairs