Monitoring the implementation of the national institutes of Health Strategic Plan for Women's Health and Sex/gender Differences research: Strategies and Successes

Abstract

Building upon the legacy of the previous two National Institutes of Health (NIH) women's health research agenda–setting reports,^1,2 the Office of Research on Women's Health (ORWH) released the third NIH scientific agenda for women's health and sex differences research in September 2010, entitled Moving Into The Future With New Dimensions and Strategies: A Vision for 2020 For Women's Health Research.³ Within NIH, ORWH is part of the Division of Program Coordination, Planning, and Strategic Initiatives, residing in the Office of the Director; ORWH is charged with coordinating women's health research in collaboration with the 27 Institutes and Centers (ICs) that make up NIH, each of which has a distinct mission and identity. Of note, the 2010 research agenda, or strategic plan, is the women's health research agenda for NIH overall, cutting across the missions of all the ICs. As such, it serves as a map to guide new efforts as well as continue collaborations within NIH in order to fulfill the NIH mission to seek fundamental knowledge about the nature and behavior of living systems and to apply that knowledge to enhance health, lengthen life, and reduce illness and disability. Through the framework of the strategic plan, in partnership with the NIH ICs, the Office of the Director, and the Advisory Committees (Figure 1), ORWH leads efforts to meet this mission as it relates to women's health.

INTRODUCTION

Building upon the legacy of the previous two National Institutes of Health (NIH) women's health research agenda–setting reports,^1,2 the Office of Research on Women's Health (ORWH) released the third NIH scientific agenda for women's health and sex differences research in September 2010, entitled Moving Into The Future With New Dimensions and Strategies: A Vision for 2020 For Women's Health Research.³ Within NIH, ORWH is part of the Division of Program Coordination, Planning, and Strategic Initiatives, residing in the Office of the Director; ORWH is charged with coordinating women's health research in collaboration with the 27 Institutes and Centers (ICs) that make up NIH, each of which has a distinct mission and identity. Of note, the 2010 research agenda, or strategic plan, is the women's health research agenda for NIH overall, cutting across the missions of all the ICs. As such, it serves as a map to guide new efforts as well as continue collaborations within NIH in order to fulfill the NIH mission to seek fundamental knowledge about the nature and behavior of living systems and to apply that knowledge to enhance health, lengthen life, and reduce illness and disability. Through the framework of the strategic plan, in partnership with the NIH ICs, the Office of the Director, and the Advisory Committees (Figure 1), ORWH leads efforts to meet this mission as it relates to women's health.

Figure 1.

Graphical representation of the Office of Research on Women's Health's position within the National Institutes of Health (NIH) and interaction with advisory committees. This figure is not intended to serve as an organizational chart. For NIH organizational charts and definitions of the ICs' abbreviations, visit hhttp://oma.od.nih.gov/manualchapters/management/1123/Default.htm.

The 2010 strategic plan represents the culmination of a highly interactive scientific and public partnership that encompassed both looking back for historical perspectives and looking forward to new research opportunities on the horizon. The resulting three volumes³-⁵ comprise the NIH Strategic Plan for Women's Health and Sex Differences Research (SP-WH/SDR) and serve as a framework for research investigations galvanized by cutting-edge technologies and catalyzed by nascent scientific concepts to advance women's health research through interdisciplinary and multidisciplinary collaborations across the entire research spectrum from basic to clinical and translational.⁶

During the strategic planning process, regional topic-based working groups put forward a total of 400 recommendations, from which six crosscutting goals were distilled.⁶ (For an additional description of this process, see the article by Vivian Pinn, MD, in this issue.) Central to the strategic plan is the importance of evaluating sex and gender differences across the research spectrum with an explicit emphasis on interdisciplinary approaches. The goals of the strategic plan are outlined below:

1. Increase sex differences research in basic sciences studies

2. Incorporate findings of sex/gender differences in the design and application of new technologies, medical devices, and therapeutic drugs

3. Actualize personalized prevention, diagnostics, and therapeutics for girls and women

4. Create strategic alliances and partnerships to maximize the domestic and global impact of women's health research

5. Develop and implement new communication and social networking technologies to increase understanding and appreciation of women's health and wellness research and

6. Employ innovative strategies to build a well-trained, diverse, and vigorous women's health research workforce.

Each goal has corresponding objectives, ranging between five and nine objectives for each goal, for a total of 44 objectives across all six goals. As the lead for the coordination of research agenda across 27 distinct ICs, ORWH has endeavored to create a toolkit and metrics to assess implementation of the strategic plan. ORWH is using this toolkit to guide its efforts and inform discussions with the ICs that make up NIH to stimulate women's health and sex/gender differences research.

INTEGRATION OF THE STRATEGIC PLAN IN THE NIH RESEARCH AGENDA

To promote and catalyze integration of the NIH SP-WH/SDR into the overarching NIH research agenda, ORWH has embarked upon a process designed to enhance and strengthen partnerships and establish new routes of communication and modes of collaboration through identification of areas of synergy between the IC's strategic plan and the NIH SP-WH/SDR.

Meeting With Institutes And Centers Directors: Protocol

Each meeting with IC leadership is approached as a brainstorming discussion. By design, the meetings are tailored to the interests of the IC partners in attendance, but the prepared guiding materials are standardized to consistently highlight areas of current collaboration and areas of overlap that could be mined or leveraged for new collaborative efforts. Before each meeting, the IC's strategic plan is reviewed and summarized, and areas of overlap with the NIH SP-WH/SDR are identified. All research co-funded with the IC in the previous fiscal year is summarized as a starting point to characterize successful existing collaborations. Objectives of the NIH SP-WH/ SDR with highest potential for collaboration are selected for discussion by ORWH and are highlighted for the IC partners. All meeting attendees are provided with the following guiding questions, which are tailored to each IC:

• The following are areas of mutual concern and overlap between the NIH strategic plan on women's health and sex differences research and [IC]'s strategic plan(s) and vision. How can we work together to leverage our efforts in these areas? [Language from the IC's strategic plan is provided here.]

• What are the low-cost, readily feasible small steps (“low-hanging fruit“) that can be taken to raise awareness and investigation of sex differences in areas within [IC]'s mission?

• Are there new or ongoing activities within your IC that could provide an opportunity for collaboration with ORWH? If so, what are they, and how can ORWH work to magnify the effect and expand the scope of the research or activities?

• What advice and/or recommendations can you provide for the implementation of the NIH strategic plan on women's health and sex differences research, especially as it relates to optimized approaches in a fiscally constrained environment?

Following the meeting, any action items identified are prioritized and followed up on in a timely manner to ensure that new opportunities are launched.

Meeting With institutes and centers Directors: Selected Outcomes Thus Far

The meetings with IC directors have forged stronger communication pathways, brought about new ways of thinking, and led to significant new routes of collaboration. For example, the ORWH concept for an administrative supplements (to parent grants) program to draw attention to the importance of sex/gender as a basic variable in research was discussed and endorsed at the National Institute for Arthritis, Musculoskeletal, and Skin Diseases (NIAMS) meeting in August 2011. ORWH implemented this concept in fiscal year 2013 with the Administrative Supplements for Research on Sex/ Gender Differences Program, which awards supplemental grants of up to $100 000 USD to existing parent grants for the purpose of adding a new sex/gender–based study population, analysis, or data set. One outcome of the meeting with the National Human Genome Research Institute (NHGRI) was the suggestion that ORWH could inform their selection of sequencing targets under their Large-Scale Genome Sequencing Program. In response, ORWH formed an ad hoc trans-NIH working group on comparative sex and gender research and genomic sequencing made up of members from the Coordinating Committee on Research on Women's Health (CCRWH). The group recommended several diseases and animal models for diseases that exhibit a large sex disparity in incidence, progression, presentation, or outcome of disease. This opportunity to add an element of sex/gender differences research to such a large, well-established program is one success of the IC director's meeting that could be replicated across each IC in future meetings, as well as meetings with advocacy groups, industry groups, and other federal agencies.

EXAMPLES OF THE IMPLEMENTATION OF STRATEGIC PLAN GOALS

ORWH recognizes that the study of women's health requires an interdisciplinary approach to research, bridging basic and clinical science and incorporating new models of collaboration, institutional support, and ways of evaluating those who conduct it. In the past 13 years, ORWH has stimulated novel interdisciplinary research through funding initiatives such as the Specialized Centers of Research on Sex Differences (SCORs) and the Building Interdisciplinary Research Careers in Women's Health (BIRCWH) program. The BIRCWH program and SCOR centers were initiated in 2000 and 2002, respectively, as a result of the recommendations from the second NIH research agenda–setting process for women's health research.² These two ORWH signature programs represented a new paradigm to advance the science associated with women's health and to better understand the contributions of sex and gender to human health and disease. As one of the first actions taken to launch the implementation of the new strategic plan, ORWH reissued the funding opportunity announcements (FOAs) for these two signature programs. Four of the six goals are described below.

Goal 1: Increase Sex Difference Research In Basic Sciences Studies

The SCOR is an innovative interdisciplinary research program established and implemented by the ORWH in 2002 to integrate basic and clinical approaches to sex and gender research across scientific disciplines. In July 2012, ORWH announced a third round of SCOR centers, renamed the Specialized Centers of Research on Sex Differences (from the earlier Specialized Centers of Research on Sex and Gender Factors Affecting Women's Health). To date, there have been a total of 33 awards made to 26 academic institutions. Each award is for a total cost of $1 million USD per year; the grants are for 5 years. Currently, there are 11 active SCOR centers. The principal investigators (PIs) are accomplished scientists in their fields. Each SCOR consists of at least three individual, but interrelated, research projects, each with high scientific merit and clear research objectives and, in the aggregate, devoted to a specific major health area relevant to women's health. The previous two rounds of SCOR PIs' research findings have provided numerous insights into the sex differences observed in pain, including visceral pain and pelvic floor dysfunction; depression, addiction, and substance use disorders; other conditions, including osteoporosis, recurring urinary tract infections, and cardiovascular disease; and general health, including immunological responses and the central nervous system. New to this round of SCORs, the PIs are collecting sets of common data instruments and elements to be standardized and shared.

Goal 3: Actualize Personalized Prevention, Diagnostics, And Therapeutics For Girls And Women

In fiscal year 2012, ORWH provided cofunding to the National Institute for Child Health and Human Development (NICHD) in support of the grant titled Effect of Feeding Buddies in Adherence to World Health Organization (WHO) Prevention of Maternal-Child HIV Transmission (PMTCT) Guidelines in S. Africa, which is part of the NIH/President‘s Emergency Plan for AIDS Relief (PEPFAR) Collaboration for Advancing Implementation Science in PMTCT. The investigators are evaluating the effect of a feeding buddy to support a mother in adhering to PMTCT recommendations in order to establish feasible models of promoting HIV-free infant survival in resource-limited settings. The aims of the project are to determine the effect of a feeding buddy on adherence to exclusive breastfeeding and to determine the effect of a feeding buddy on adherence to antiretroviral (ARV) prophylaxis. Feeding buddies could be one low-cost strategy for strengthening existing efforts to implement the new PMTCT guidelines and could contribute to improving HIV-free survival for children.

Goal 4: Create Strategic Alliances And Partnerships To Maximize The Domestic And Global Impact Of Women'S Health Research

ORWH has collaborated with the NCI on the development and evaluation of the prophylactic virus-like particle (VLP) human papillomavirus (HPV) vaccine that was discovered by investigators at the National Cancer Institute (NCI).⁷ As far back as the late 1990s, ORWH supported efforts to conduct animal studies and an early phase I human trial in the United States that were critical to demonstrating the potential for this vaccine. Subsequently, ORWH supported the multiyear, community-based, randomized phase III trial of the HPV vaccine conducted by NCI in Costa Rica. This is the only publically funded trial of an HPV vaccine.

This research is now being expanded to examine gallbladder and anal cancers and builds upon 10 years of this landmark collaboration. Gallbladder cancer is one of the few nongynecological tumors known to occur with higher frequency in women than in men. The highest rates of this cancer (particularly in women) are observed in Chile. Investigators in the Infections and Immunoepidemiology Branch (IIB) at the National Cancer Institute are evaluating the feasibility of conducting a case-control study of gallbladder cancer in Chile to better understand the causes of this disease. As currently envisioned, the initial pilot effort will define whether such a study would be successful at identifying and enrolling cancer cases, controls with gallstones (an important precursor for this cancer), and controls from the general population. Investigators are working with well-established investigators in the region with a proven track record of conducting epidemiological investigations to maximize the likelihood of success. A case-control study of gallbladder cancer in Chile will permit the elucidation of the role of obesity (and metabolic syndrome more generally), diet, infections, immunological responses, and genetic susceptibility factors in the etiology of this tumor.

Goal 6: Employ Innovative Strategies To Build A Well-Trained, Diverse, And Vigorous Women'S Health Research Workforce.

One objective under this goal is to “lead the way in encouraging institutions to recognize mentoring as an essential component of building career success in their training programs.” The office continues to support innovative ways to encourage collaborative, interdisciplinary research that is team based to improve women's health through the BIRCWH program. BIRCWH is an interdisciplinary mentored career development program for men and women junior faculty in women's health research. In fiscal year 2012, ORWH announced the sixth round of awards to new or competing BIRCWH programs, bringing the total number of academic institutions that have ever received a BIRCWH award to 39. As of November 1, 2012, a total of 493 individuals had participated in the BIRCWH program as scholars. While the BIRCWH program is open to both women and men, 80% of the BIRCWH scholars are women. The majority of scholars have gone on to research independence. Further data on BIRCWH scholars are described in the accompanying BIRCWH article by Nagel et al that is part of this special issue.

The ORWH signature programs (BIRCWH and SCOR) are now serving as major vehicles for advancing women's health and sex differences research that holds the potential to be translated into clinical practice to benefit the health of women and men. Both programs continue to support and are critical to advancing all six of the strategic plan goals, particularly the ultimate goal of actualizing personalized prevention, diagnostics, and therapeutics for girls and women. These programs and cofunding efforts demonstrate how ORWH sets its research priorities and future directions and leverages existing IC research efforts to maximize returns on investment.

DEVELOPMENT OF SUCCESS METRICS AND A TOOLKIT FOR DATA COLLECTION AND VISUALIZATION FOR STRATEGIC PLAN IMPLEMENTATION EVALUATION

In order to collect information on funding and support for activities that implement the Strategic Plan goals and objectives, such as the successes described here, ORWH designed a toolkit that assimilates the information and presents it in an accessible way its use for strategic decision-making. The toolkit is designed as an internal implementation evaluation, rather than a program outcome evaluation. The implementation evaluation toolkit described below will be complemented by an outcome evaluation employing standard metrics such as publications and patents, in order to assess the full impact of the new strategic plan.

Strategic Plan Objective Prioritization

The goal of having the strategic plan guide all new funding, activities, and initiatives required that all 44 objectives under the six goals be prioritized and targeted in a manner that takes into consideration both the costs (financial, feasibility, or other) and the time required to complete each. To this end, ORWH created an action matrix that portrays stratification and prioritization on the major domains of cost and time (Figure 2). Following staff comment, the action matrix has been updated iteratively with input from two Congressionally mandated advisory committees: the Advisory Committee for Women's Health Research (ACRWH, comprised of non-NIH physicians, scientists, and other health professionals who advise the ORWH director) and the CCRWH (comprised of IC directors or their designees to assist the ORWH director in accomplishing its mission). An immediate benefit of this stratification is the discernment of highest-cost and/or longest-term objectives; these objectives will likely have prerequisites to be addressed prior to fulfillment. The objectives designated as shortest term and lowest cost encompass many of these prerequi-sites; ORWH now selects a subset from these identified objectives for discussion at monthly meetings with the CCRWH centered on ways to fulfill this “low hanging fruit.” In this way, ORWH and NIH can both address specific low-cost/short-term objectives and also meet necessary benchmarks for fulfilling longer-term or higher-cost objectives. These interactions with stakeholders, including the monthly meetings with CCRWH, support ORWH's ability to coordinate the spectrum of NIH women's health and sex differences research.

Figure 2.

Action matrix developed by the Office of Research on Women's Health stratifies objectives based on cost/difficulty and time needed to complete each. (Numbers here represent specific objectives: 1.9 is Objective 9 under Goal 1).

^aCost can include financial cost, as well as staff effort, political costs, etc.

Strategic Plan Implementation Evaluation Data Collection And Assimilation

In order to achieve strategic plan implementation, ORWH needed to collect data not previously captured by any existing system or database, including the NIH IMPACII database that houses data on publicly funded awards to investigators and can be accessed only by government staff. The information required for an evaluation of strategic plan implementation includes additional data on programmatic activities beyond direct research funding that entail significant effort in terms of staff time and/or that can have large impacts, such as serving on working groups, delivering scientific presentations, or organizing scientific workshops. ORWH developed a new data collection system to gather information on the full spectrum of program staff activities, from personnel-attended meetings to RFAs cofunded by the office. The electronic SPIE form (Figure 3) allows program staff to code each relevant activity by the strategic plan objective(s) fulfilled. The form also collects data on type of activity, ORWH role in the activity, and any IC or the US Department of Health and Human Services partners (DHHS) in the activity. Using a single click of the mouse, staff can send the forms to the SPIE officer, where data assimilation is automated using free Adobe Acrobat software and reviewed for standardization across staff. Further, a weighted score is coded into each activity, based on a weighting scheme created by the office (from 1 to 100, spanning an attended meeting to a supported RFA). This process implemented by ORWH collects a breadth of information not collected elsewhere in a way that is standardized, efficient, and highly-effective.

Figure 3.

The electronic Strategic Plan Implementation Evaluation (SPIE) form created and used by the Office of Research on Women's Health allows all program activities to be coded by strategic plan objective fulfilled and collects other relevant data.

Strategic Plan Implementation Evaluation Data Visualization.

The creation of the SPIE toolkit by ORWH described above culminated in the use and implementation of a data visualization technique that allows all 44 objectives of the strategic plan to be presented in a single aesthetic and accessible visual (example in Figure 4). Using a modified polar area diagram, the tool is dynamic in that the radius of the circle can correspond to any relevant success metric, from number of activities that fulfill each objective to the amount of funding put toward each objective. The visual instantly conveys successes in implementation of the strategic plan across specific objectives and permits the easy identification of objectives where there are opportunities for increased focus.

Figure 4.

Data visualization tools conceived and developed at the Office of Research on Women's Health allow all 44 objectives to be seen at once across a dynamic set of metrics. Shown is the number of activities fulfilling each objective as of March 2012.

CONCLUSION

For the first time ever, the Report of the Advisory Committee on Research on Women's Health, Fiscal Years 2011-2012: Office of Research on Women's Health and NIH Support for Research on Women's Health prepared in early 2013 includes contributions from each IC that highlights research that specifically fulfills major goals and objectives of the NIH SP-WH/ SDR, demonstrating how this strategic plan has already been interwoven throughout the fabric of the NIH research agenda.⁸ ORWH is now in the second year of data collection concerning strategic plan implementation, which has provided a full fiscal year of information on which to make evidence-based decisions regarding research funding and other programmatic initiatives. In addition, this expanded data set allows a more robust analysis of research trends, successes, and gaps in knowledge. Capitalizing on these expanding analysis tools and data sets, strengthening partnerships with IC colleagues, and continuing input and insight from the ACRWH and CCRWH, ORWH is poised to boldly move into a healthier future for women and girls envisioned by the strategic plan.