Proposal for the Development of a Standardized Protocol for Assessing the Economic Costs of HIV Prevention Interventions

Summary

Maximizing our economic investment in HIV prevention requires balancing the costs of candidate interventions against their effects and selecting the most cost-effective interventions for implementation. However, many HIV prevention intervention trials do not collect cost information, and those that do use a variety of cost data collection methods and analysis techniques. Standardized cost data collection procedures, instrumentation, and analysis techniques are needed to facilitate the task of assessing intervention costs and to ensure comparability across intervention trials. This article describes the basic elements of a standardized cost data collection and analysis protocol and outlines a computer-based approach to implementing this protocol. Ultimately, the development of such a protocol would require contributions and “buy-in” from a diverse range of stakeholders, including HIV prevention researchers, cost-effectiveness analysts, community collaborators, public health decision makers, and funding agencies.

Nearly 40 million people worldwide are infected with HIV, the virus that causes AIDS.¹ The scale of this epidemic and the required response are unprecedented in world history. Because economic resources are limited, decision makers at local, national, and international levels face difficult choices between competing needs for treatment and prevention and choices among alternative programs within the treatment and prevention arenas. This article focuses on HIV prevention interventions, but similar considerations also apply to HIV treatment programs.

HIV prevention decision makers must consider myriad factors when selecting interventions for deployment, including but not limited to the appropriateness and acceptability of candidate interventions for the target community, intervention costs, and potential intervention effects.² Because HIV prevention budgets are universally constrained, funding a particular intervention imposes an “opportunity cost” in that the economic resources devoted to that intervention are no longer available to fund alternative interventions. Maximizing the investment in HIV prevention requires balancing the costs of candidate interventions against their effects and selecting the most cost-effective interventions for implementation.³

Determining intervention costs is a necessary first step toward assessing overall cost-effectiveness. HIV prevention intervention trials are a potentially important source of cost estimates for “state of the science” HIV prevention approaches. Many HIV prevention intervention trials do not collect cost information, however, and those that do use a variety of cost data collection methods and analysis techniques. These 2 issues likely are related. HIV prevention intervention trials are complex enterprises, and intervention researchers may be reluctant to devote the additional time, energy, and money required to design their own cost data collection methods and develop their own instruments.

If “off the shelf” data collection methods and instruments were available and were packaged together with user-friendly cost data analysis software, more intervention researchers likely would incorporate cost data collection and analyses into their overall study designs. This argues for the need to develop standardized cost data collection procedures, instrumentation, and analysis techniques. The need for standardization is further underscored by the intended use of intervention cost information; namely, to assist HIV prevention decision makers in the difficult task of comparing one intervention with the next. The current lack of standardization creates “apples” and “oranges” that cannot be directly compared.

A similar need for standardization is evident with regard to the often difficult question of quantifying the impact of an HIV prevention intervention, which is the second element (along with costs) needed to assess the cost-effectiveness of an intervention. Because of space limitations, the present article focuses exclusively on the cost side of the equation. With or without corresponding effectiveness estimates, cost information can provide valuable guidance to decision makers, whose first question often is “Can we afford it?”⁴

This article describes the basic elements of a standardized cost data collection and analysis protocol and outlines a computer-based approach to implementing this protocol. Ultimately, the development of such a protocol would require contributions and “buy-in” from a diverse range of stakeholders, including HIV prevention researchers, cost-effectiveness analysts, community collaborators, public health decision makers, and funding agencies. It is beyond the restricted scope of the present article to describe potential consensus development procedures or to propose concrete recommendations for the protocol itself. Rather, the present article is exploratory in nature. It is meant to provoke thought, outline the terrain, and suggest possible approaches for developing a standardized cost data protocol for use in HIV prevention intervention trials.

STANDARDIZING HIV PREVENTION INTERVENTION COST DATA COLLECTION AND ANALYSIS

There are 3 main steps involved in assessing the cost of an HIV prevention intervention: (1) identification of specific resource items utilized in the intervention, (2) development of appropriate procedures and instruments for collecting cost information about the items identified in the first step, and (3) combining and summarizing the information from the second step to estimate the costs of intervention-related activities and the overall cost of the intervention. The standardized protocol envisioned here would consist of a series of “best practice” recommendations with respect to each of these 3 steps (see the next section for examples of best practice recommendations).

These recommendations could be promulgated through the usual channels (eg, conference presentations, journal articles, possibly a monograph) to reach the target audience of HIV prevention intervention researchers. Adoption of a standardized protocol would enhance the quality and the comparability of HIV prevention intervention cost analyses. By itself, however, the existence of a standardized protocol is unlikely to increase the number of investigators who integrate cost data collection and analyses into their intervention trials.

Designing a cost study and conducting cost data analyses are complex tasks that require a modicum of specific expertise that may or may not be optimally represented on a particular study team. It takes time and effort to identify the intervention-related costs that should be included in the “cost inventory,” to develop cost data collection procedures and to design the associated forms or other instrumentation, to conduct the analyses themselves, and to compile and format summary reports. The guidance provided by a standardized protocol that included best practice recommendations on cost study design, data collection and instrumentation, cost analysis techniques, and ways to summarize and report study findings would reduce the burden on investigators and, presumably, increase the number of studies that collect intervention cost information.

To reduce the burden on study investigators further, an integrated software package could be developed that would assist the study team with the myriad tasks associated with conducting an economic evaluation. The proposed software package would incorporate expert knowledge related to the conduct of HIV prevention intervention cost studies and would codify elements of the standardized protocol (best practices). It would significantly simplify the process of designing and implementing a cost study, which could help to increase the number of investigators who collect and analyze intervention-related cost data.

HIV prevention intervention strategies range from behavioral approaches, such as mass media campaigns or risk reduction counseling, to mainly biomedical approaches, such as male circumcision or sexually transmitted infection (STI) detection and treatment. To accommodate the diverse range of HIV prevention strategies evaluated in current and future intervention trials, the proposed software package should be modular in design and should guide investigators through the selection of individual modules applicable to their particular study designs. A flow chart-based algorithm in the software program would allow investigators to customize the standardized protocol to their particular intervention applications. Decision points might include, for example, “Will participants be provided with behavioral counseling?” A “yes” answer to this question would trigger the behavioral counseling module of the program, which would solicit further information regarding the number and length of counseling sessions; the number and types of staff who conduct the sessions; where the sessions are conducted and associated facility costs; and materials, supplies, and equipment needed in the counseling sessions. The software would be expected to anticipate and suggest resource costs that might potentially be associated with the behavioral counseling component of the intervention and would “work” with the investigator to ensure that the resultant cost inventory is comprehensive, appropriate for the target intervention, and consistent with the standardized protocol.

Despite the wide diversity of possible HIV prevention approaches, the types of resources utilized in HIV prevention interventions generally can be classified as belonging to one of several broad categories, such as personnel costs, materials and supplies, equipment, and facilities. Refining these categories and compiling a comprehensive list of the items that fall into each category is a necessary first step toward standardizing HIV prevention intervention cost analyses. The development of a standardized list of potential elements to be included in intervention cost inventories would allow development of structured data collection methods, forms, and analysis techniques.

Based on the cost inventory, the software would help the investigator to identify procedures to collect the necessary cost information (eg, using time diaries to record staff time spent in various intervention-related activities) and would provide detailed instructions about the implementation of these procedures. The program would generate the necessary cost data collection forms customized to the specific requirements of the intervention study (eg, time diary forms could be broken down by activity, with different activities listed for staff with different responsibilities). Forms could be paper based or computer based. Use of computer-based forms would obviate the necessity to enter the cost data at a later time but might be impractical for gathering certain types of cost information or might not feasible in certain settings. In short, the automated protocol would encourage the use of similar methods and instruments for collecting intervention cost information, which, in turn, would enhance cross-intervention comparability.

The final component of the software package would perform basic analyses of the cost data collected in the intervention trial. For example, it would combine information about the time spent by staff in a particular intervention activity (eg, behavioral counseling) with staff compensation information (salary or hourly wage rate plus fringe benefits) to determine the total personnel costs associated with that activity. This information would then be combined with other costs related to the particular activity (eg, materials and supplies, facility costs) to determine the overall cost of the activity. The total cost of the intervention would then be calculated by summing across intervention activities. Total cost by category (eg, personnel costs, equipment costs) across activities also would be calculated. To increase cross-study comparability, the software would generate cost analysis summaries, tables, and figures in standardized formats suitable for publication.

BEST PRACTICE GUIDELINES

The standardized protocol would include best practice recommendations similar to those advanced by the Panel on Cost-Effectiveness in Health and Medicine^5,6 but tailored to the specific challenges associated with conducting HIV prevention cost studies. Here, we provide initial thoughts on some of the main issues that would need to be addressed by the standardized protocol.

These issues include general questions related to the framing of the cost analysis, such as the study’s perspective and the time frame over which cost data are collected. With regard to the study perspective, the panel recommends that all studies include a “reference case” analysis conducted from the “societal perspective.” This perspective differs from the “provider prospective” in that it includes all costs, regardless of who incurs them, rather than only costs borne by the intervention provider. For example, costs associated with intervention participants’ lost work time, transportation costs, and other expenses related to their participation in the intervention would be included in an analysis conducted from the societal perspective but excluded from a provider perspective analysis. To maximize the usefulness of HIV prevention intervention cost analyses, we recommend that studies collect and report those costs needed to support analyses from both perspectives.

With regard to the time frame of the analysis, in some cases, it may be sufficient to collect a “snapshot” of intervention costs, for example, by collecting costs over a restricted period once the intervention is fully operational. For others, it may be desirable to collect costs over an extended period to capture potential seasonal or other temporal variability.⁷ Recommendations would be developed to identify the circumstances under which one approach or the other might be required.

Given the international application of HIV prevention interventions, the generalizability of cost study findings must be carefully considered. Differing wage rates and the costs of other goods and services make it difficult to apply the results found in one country to programs that are designed for another country. This becomes even more complicated when one has to adjust for currency differences. Preliminary recommendations related to these issues include the following. First, intervention personnel should be identified by job title or professional classification (eg, project manager, physician). Titles that are idiosyncratic to the location of the study (eg, civil service level 14) should be avoided. The applicable wage rate for that job category in the local market should be used to calculate personnel costs and should be included in the cost analysis summary so that others who are utilizing the results can convert the results to their settings. Reporting universally understood job titles would allow decision makers to apply the results to their circumstances by using local pay scales.

To make comparisons more viable, it also is important that costs be broken down to basic levels so that they can be easily converted for use in other settings. For example, cost analyses should report not only the total intervention cost but the cost of each main intervention activity (eg, disaggregate the costs of HIV testing from the costs of counseling in a voluntary counseling and testing [VCT] intervention). Further, personnel costs should be reported as “X minutes of nursing time per patient,” rather than as “Y dollars of personnel costs per patient,” because these broad categories may not be transferable to other settings.

Similarly, it is important to disaggregate supply costs, including purchased services (eg, laboratory services), so that others can apply local costs when adapting study results to local circumstances. An excellent example of this is the cost of antiretroviral drugs, which varies greatly across settings. Because countries have negotiated different prices with drag companies, an intervention that involves antiretroviral medications may be cost-effective in one country but not in another.

Cross-study comparability requires the use of a common stable currency. Local currency can be used to perform the original calculations but should be converted to a stable currency that is regularly used in international transactions (eg, US dollars, Euros) at the exchange rates that are in place at the end of the study.

Clearly, there are many more issues that need to be addressed in the standardized protocol, including questions related to startup costs, training, and program evaluation To enhance the likelihood that the standardized protocol is widely adopted in HIV prevention trials, consensus among HIV prevention economists, investigators, and other stakeholders is necessary with respect to the best practice recommendations advanced in the protocol.

A MAJOR CHALLENGE

Perhaps the most significant challenge to the development and implementation of a standardized cost data collection and analysis protocol for HIV prevention interventions is the potential discrepancy between the intervention costs observed in the context of research trials and the costs of implementing interventions under “real-world” conditions. The main rationale for conducting a cost analysis is to provide public health decision makers with the information they need to prioritize HIV prevention and other health-related intervention efforts. Therefore, the closer the fit between the study intervention and its (eventual) real-world counterpart, the greater is the policy relevance of the cost analysis results. The discrepancy issue is not unique to the question of intervention costs but applies to the effectiveness side of the equation as well, perhaps more so.

With regard to intervention costs, the discrepancy can be minimized by carefully distinguishing between research-related costs and true intervention costs. “Intervention costs” are costs that would be incurred if the intervention were conducted in the real world rather than in a research setting. Only intervention costs should be included in the cost analyses; costs associated with the research objectives of the intervention trial should be excluded. For instance, costs associated with tracking participants for follow-up purposes, data collection and analyses, and assessing the effectiveness of the intervention generally are research related rather than truly intervention related and should not be included in intervention cost analyses.

Transferability of research trial findings to a real-world setting could be enhanced further if, as recommended previously, costs were disaggregated and reported at meaningful levels; for example, minutes of staff time required to perform a particular intervention-related task, together with the staff person’s job classification. This would allow decision makers to substitute relevant local wage rates and other costs for the costs observed in the intervention trial and reported in the cost analysis summary. Importantly, the substitution of real-world costs for the costs obtained in the intervention trial requires the principled development of a model of how the intervention would be implemented in the real world and how this implementation might differ, if at all, from the intervention trial protocol.

SUMMARY AND DISCUSSION

The primary goals of developing a standardized cost data collection and analysis protocol are 3-fold: first, to encourage investigators to collect information about intervention costs; second, to make it simpler for investigators to integrate cost data collection and analyses into their studies; and third, to increase the comparability of cost estimates across intervention studies. Although the development of a standardized protocol is largely independent of its possible implementation in the form of an integrated software package, we believe that success in achieving these 3 objectives would be greatly enhanced if such a computer package were available. Specifically, investigators should be responsive to a user-friendly automated system that incorporates expert knowledge related to the costs of HIV prevention interventions, provides guidance on cost data collection, generates customized forms to facilitate the collection of cost information, and automatically summarizes this information. This would make it easier and less expensive for investigators to integrate cost data collection and analyses into their intervention trial study designs. Acceptance of this system would enhance cross-study comparability through the standardization of cost inventories; cost data collection procedures; and generation of cost analysis summaries, tables, and figures, all in standardized formats.

The development and implementation of a standardized cost data collection and analysis protocol for HIV prevention intervention trials face several significant but surmountable challenges. First, the breadth of possible intervention approaches makes standardization difficult. Standardized cost protocols have been developed for more limited ranges of intervention strategies, however.^8,9 The proposed protocol, which would span behavioral, biomedical, and mixed approaches to HIV prevention, is a bit more ambitious than previous standardization efforts but does not differ qualitatively from these efforts.

Second, the software package envisioned here would need to be highly interactive and flexible enough to allow investigators to substitute their own judgment for the program’s “artificial intelligence.” It also would need to be extendable to accommodate the needs of future researchers, advances in the science of HIV prevention, novel intervention strategies, and revisions to the standardized cost analysis protocol. Finally, it would need to be user-friendly and simple to navigate to encourage its widespread adoption by HIV prevention researchers across a range of disciplines.

Third, the development of a standardized cost protocol, and especially the automation of this protocol, would require a substantial financial investment. It also would require the time and effort of a variety of stakeholders and a commitment to the goal of establishing best practice recommendations through a consensus-building process. Additional commitment on the part of study investigators and funding agencies would be needed to ensure the ultimate success of this undertaking. At present, there is little incentive for investigators to incorporate cost studies into their intervention study designs. The standardized protocol would significantly ease the burden on investigators; however, without explicit encouragement from funding agencies, many investigators likely would be reluctant to increase their study budgets to accommodate the additional costs associated with conducting an economic evaluation of proposed interventions.

Once developed, the cost analysis software could be offered under an open-source license to allow and encourage collaboration in maintaining the software and keeping it up to date; for example, by extending it to handle novel types of interventions. A Web site could be developed to support the software and assist in the dissemination of the software program, updates, and information about the standardized protocol. Through such a site, researchers could download software and documentation, share their experiences, and get answers to their questions regarding how best to apply the standardized protocol. Intervention investigators would be encouraged to upload the results of their cost analyses, which then could be integrated into an intervention cost database. These detailed and standardized data would be invaluable for comparing costs across interventions and conducting meta-analyses of the economic costs of HIV prevention interventions. This Web site would be an important resource not only for intervention researchers but for policy planners interested in projecting the costs of future HIV prevention programs.