Cost Analysis of Prevention Research Centers: Instrument Development

Abstract

The 2014–2019 Prevention Research Centers (PRC) Program Funding Opportunity Announcement stated that “all applicants will be expected to collaborate with CDC to collect data to be able to perform cost analysis.” For the first time in the 30-year history of the PRC Program, a cost indicator was included in the PRC Program Evaluation and a cost analysis (CA) instrument developed. The PRC-CA instrument systematically collects data on the cost of the PRC core research project to eventually answer the CDC PRC Program Evaluation question: “To what extent do investments in PRCs support the scalability, sustainability, and effectiveness of the outcomes resulting from community-engaged efforts to improve public health?” The objective of this article is to briefly describe the development of the PRC-CA instrument. Data obtained from the PRC-CA instrument can be used to generate cost summaries to inform decision making within the PRC Program and each individual PRC.

In September 2014, the Centers for Disease Control and Prevention (CDC) Prevention Research Centers (PRCs) Program started a new 5-year funding cycle (2014–2019) for 26 academic institutions in 24 states.¹ PRCs are located at academic institutions with either an accredited school of public health or an accredited medical school with a preventive medicine residency program.²

Economic evaluation, such as cost analysis, contributes to public health evidence-based decision making and is an integral part of identifying and measuring the impact of public health activities, as well as their scalability and sustainability potential.^3–6 Understanding estimated program costs is as important as identifying the size and characteristics of population reached and project impact when deciding on which public health strategies to implement.^5,6

In response to the increased demand for economic evidence for public health practice, the 2014–2019 PRC Funding Opportunity Announcement (FOA) stated that “all applicants will be expected to collaborate with CDC to collect data to be able to perform cost analysis.”⁷ For the first time in the PRC Program’s 30 years, a cost indicator for the PRC core research project is included in the CDC PRC Program evaluation plan. The cost indicator helps answer the PRC Program evaluation question, “To what extent do investments in PRCs support the scalability, sustainability, and effectiveness of the outcomes resulting from community-engaged efforts to improve public health?” This question is operationalized through the PRC core research project cost analysis (PRCCA), and a PRC core research project cost analysis instrument (PRC-CA instrument) was developed and implemented.

The PRC-CA instrument is a module of a larger data collection and reporting system for the overall PRC Program evaluation, called Prevention Research Centers Program Evaluation and Reporting System (PRC-PERS). PRC-PERS is a centralized Web-based data collection system hosted on a secure CDC server; it has audit trails, analyzable data sets through queries, and reporting for various stakeholders. Here, we describe the PRC-CA, as well as the development and implementation of the PRC-CA instrument.

PRC Core Research Project Cost Analysis

The purpose of the PRC-CA is to measure the cost of the core research project and to obtain baseline data for further economic studies such as cost-effectiveness analysis. We created a logic model (Figure) for the PRC-CA to operationalize data collection through the development and implementation of the PRC-CA instrument. The PRC-CA instrument captures the dollars spent on labor, materials and consumables, travel, and location expenses associated with the PRC core research project (Table).

FIGURE.

PRC Core Research Project Cost Analysis Logic Model

Abbreviations: CDC, Centers for Disease Control and Prevention; PRC, Prevention Research Center.

TABLE.

Prevention Research Centers Cost Analysis Instrument Cost Measures Descriptions

Measures	Descriptions	Pay Category
Labor	The salary and wage of each individual involved with the PRC core research project. Collected on the basis of 2 options: Salary based: Labor cost reported as an individual’s fully loaded salary and percent full-time equivalent spent on a particular activity. A fully loaded salary is the sum of a base salary and any additional compensation such as fringe benefits, endowment accounts, or discretionary funds, which reflect an individual’s full expertise. Hourly based: Labor cost reported as an individual’s hourly wage, working on the PRC core research project.	PRC paid: Faculty and staff who play active roles in the PRC core research project and are directly compensated with PRC dollars. In-kind: Subject matter experts who lend their expertise to the PRC core research project and are not directly compensated with PRC dollars. Volunteers: Any other individuals who took part in the PRC core research project activities and did not get any direct or indirect compensation fortheir contributions. For example, a volunteer might be a student or community member who recruited PRC core research project participants without getting paid.
Materials and consumables	Costs of products or services regularly used and durable or capital goods associated with the PRC core research project.	PRC paid: Any expenses paid directly with PRC dollars. In-kind: Any donated or gifted materials and consumables, travel, and location costs used for the PRC core research project.
Travel	Travel expenses incurred while working on the PRC core research project.
Location	Sum of the percentage of the cost for the primary PRC location attributed to conducting PRC core research project activities and expenses for places or settings outside the PRC office associated with PRC core research project.

Abbreviation: PRC, Prevention Research Center.

PRC-CA Instrument Development

We developed the PRC-CA instrument as a flexible data collection instrument that can support both retrospective and prospective data elements. Data collection systems used to capture research quality data for public health strategies such as the PRC core research project exist.^8–14 We used the Public Health Return on Investment (PH ROI) tool^8,9 and the Substance Abuse Services Cost Analysis Program (SASCAP)¹⁰ to guide the PRC-CA instrument development decisions. Both the PH ROI tool and SASCAP contain elements such as those we included in the PRC-CA instrument and its logic model such as labor, and materials and consumables costs.^8–10 The PRCs played an important role in the PRC-CA instrument development process, as its end users. We piloted the instrument with principal investigators from 9 PRCs by asking them to review it for content, flow, and burden. Then we analyzed the feedback using inductive qualitative content analyses to identify themes of needed improvement for the PRC-CA instrument.¹⁵ Three main themes emerged: clarify the purpose of the PRC-CA instrument; address burden concerns because of lack of flexibility; and provide clear instructions and definitions for the instrument. To address these concerns, we clearly defined the purpose of the PRC-CA instrument: The purpose of the PRC-CA instrument, a module within PRC-PERS, is to systematically collect budget year data on the costs related to the PRC core research project for each of the 5 budget years in the 20142019 funding cycle and to complete a cost analysis as directed by the 2014–2019 FOA. The PRC cost analysis will:

1. Quantify the cost of the PRC core research project;

2. Provide baseline data for further economic evaluation studies, such as a cost-effectiveness analysis; and

3. Develop capacity of each PRC and the CDC PRC Program to use economic evidence to assess the core research project’s effectiveness, efficiency, equitability, scalability, and sustainability.

To address the burden concern, we made data input flexible. PRCs can collect each expense individually as it occurs or as aggregates collected on a monthly, quarterly, semiannual, or annual basis. Finally, we created a companion issue tracking system that allows PRCs to submit any technical issues, content and definition clarification requests, process issues, or any other issues they might encounter with the CDC PRC Program Evaluation Team. Each issue is routed to specific subject matter experts within the team. Any issues or concerns that might be relevant to all PRCs are included in the Frequently Asked Questions section of PRC-PERS. We also defined all data elements within PRC-PERS.

The elements included in the PRC-CA instrument are described in Supplemental Digital Content Appendix A (available at http://links.lww.com/JPHMP/A405). It is important to note that PRCs are different from each other. They use different research types, focus on different health topics, and provide different services. Cost comparisons between different PRCs are not feasible beyond aggregated total cost of PRC core research projects for all PRCs. For each individual PRC, different cost summaries can be obtained, such as cost per participant, or by type (eg, labor or travel).

Implications for Policy & Practice.

Public health profession should consider building its capacity to collect and use economic data; for example, through curriculums in public health training programs or professional development trainings. Implementing a cost data collection system such as the PRC-CA instrument could be useful for other public health programs, because:

• Economic data inform internal budgetary decisions. For example, which category should an individual PRC invest in while developing, implementing, or disseminating its core research project?

• In addition to program effectiveness, economic data strengthen the decision-making process when choosing which evidence-based programs to translate and implement in communities.

• Economic data allow public health programs to demonstrate their impact to stakeholders and decision makers at the local, state, and federal levels.