Recreating the impact of interventions in the absence of baseline data: challenges for intervention programmes

Condom-promotion interventions have been a cornerstone of the HIV-prevention effort since the mid-1980s.¹ Initial efforts were largely education-based and used simple process outcomes, such as condom-distribution statistics. Programmes rapidly became more sophisticated, and interventions based on behavioural models were rapidly implemented. One of the most challenging issues in designing and evaluating these interventions has been outcome measures. Ideally, behavioural interventions should be evaluated on disease-incident impact measures. Since HIV incidence is uncommon even in high-incident areas, intervention impact has largely used behavioural measures, such as the proportion of sexual acts in which condoms were used. These are by definition self-reported and subject to bias. In HIV/STD clinical settings, and in the context of prevention intervention studies, there is often implicit social desirability to over-report condom use. These measurement issues have a profound impact on intervention effectiveness evaluation.

Approaches to improving validity have included refining survey report methods. This has included intensive training of interviewers, and using self-administered computerised techniques. Using other biological measures, such as other sexually transmitted infections, is not practical in most settings. Macroscopic approaches to evaluate condom-promotion effectiveness have been ecological studies, which evaluate community-based disease trends after implementing large structural interventions. For example, after implementation of the Thailand 100% condom programme, there was a rapid decrease in HIV/STI incidence, which was largely attributed to the intervention.

The Avahan investigators have developed and implemented a series of innovative approaches to the evaluation dilemma in an environment where there was no baseline programmatic data, and where there was rapid ramp-up of an intervention. The intervention involved a large population of commercial sex workers. Longitudinal assessment was possible only for a minority of individuals, recognising that these populations are highly dynamic with substantial in- and out-flow over the course of a longitudinally based assessment. Using an innovative approach, the Lowndes team² interviewed sex workers and obtained retrospective estimates of condom use for each of previous years, and then imputed those results on an aggregate basis to develop trend measures. The team's hypothesis was that increased condom use and proportion of protected sex acts would correlate with implementation of the Avahan intervention programme. This approach has substantial opportunity for reporting and ascertainment bias, and since the interviews assessed previous behaviours over a period of several years, the observations are not independent. Therefore, external validity of these imputed condom use data is required.

The Avahan programme is unique in that other measurements conducted by the study team do provide the external validity In particular, the paper by Bradley³ used operational and programme measures to estimate condom use on a population basis, and forge an estimate of the number of condoms actually available through assessing inventories and distribution data. Furthermore, the authors have clearly indicated that there were no systemic biases which changed over time. The results are validated by the reports from the series of Avahan cross-sectional analyses which were simultaneously conducted among both FSWs⁴ and clients⁵ which clearly indicated that condom use increased over time, especially with paying regular commercial clients.

Field survey research by definition and by practice always involves manipulation of data which is not as clean as one would obtain from a pure randomised controlled trial. Nevertheless, these contributions are incredibly valuable in helping us understand the dynamics of condom-use promotion and behavioural intervention impact as well as the important public-health implications. Whether or not these methods are translatable to other venues remains to be questioned and evaluated. This is a largely illiterate population, which had relatively poor access to health resources, and therefore, one may argue whether there may has been bias either to increase cooperation or to be more honest with their responsiveness. This is also a population where commercial sex work was clearly part of the culture and perhaps not as highly stigmatised as it would be in other venues.

Despite these efforts, it is important to note that the large efforts employed and invested in behavioural intervention had a modest impact but statistically significant over the course of this study. These data highlight the importance of and the necessity to maintain these interventions over time in order to continue HIV prevention efforts.