Monitoring and evaluation of programmes to prevent mother to child transmission of HIV in Africa

Abstract

Many countries are expanding the coverage of programmes to prevent mother to child transmission of HIV. Although the need is unquestionable, Richard Reithinger and colleagues are concerned that without true measures of effectiveness we may not be making the best use of resources

In 2006, an estimated 2.3 million children under 15 years were living with HIV and about half a million babies became infected with HIV before birth, during delivery, or through breast feeding.¹ Prevention of mother to child transmission of HIV is therefore a priority for agencies fighting the global HIV epidemic, but many questions remain about the effectiveness of the current programmes. We use the President's Emergency Plan for AIDS Relief² as an example to examine how programmes to prevent mother to child transmission are monitored and evaluated and to highlight the problems.

Strategy to prevent transmission

Estimates of the efficacy of antiretroviral prophylaxis³ suggest that at least half of the world's children who are at risk of HIV infection might be protected if a mother receives antenatal care, is offered HIV counselling and testing, and, if infected, she and her baby receive prophylaxis. Prophylaxis is the mainstay of the strategy to prevent mother to child transmission.³ Several antiretroviral regimens are recommended in resource constrained settings, although nevirapine (either alone or with other drugs) is usually favoured because it is cheap, easy to administer, rapidly absorbed, and has a long half life.³ Depending on the regimen and the mother's choice of infant feeding, the risk of HIV transmission can be reduced to <2%.³ Whenever feasible, programmes should strive to provide highly active antiretroviral therapy to pregnant women.

Although formula feeding can reduce HIV infection rates among infants,⁴ it is often not acceptable, feasible, affordable, sustainable, or safe in resource limited settings.^{5 6} Programmes should provide counselling and support on various feeding options (including exclusive breastfeeding), highlighting the potential benefits and risks of each.⁷ Other activities include promotion of optimal obstetrical practices; improvement of antenatal, postnatal, and child health services; and treatment of maternal diseases. Community based activities are also often implemented to improve community knowledge of HIV and AIDS and counter negative attitudes to people with HIV.⁸

Routine monitoring and evaluation tools

Because most countries do not have the money or laboratory resources to determine HIV infection in infants using polymerase chain reaction tests, health facility and patient indicators are generally used to assess the effectiveness of programmes (table 1). The effectiveness of the programme is typically measured in terms of the estimated percentage (change) of HIV infected infants who are born to HIV infected mothers.^{9 10} However, table 1 shows that effective prevention relies on a cascade of steps. A pregnant woman must receive antenatal care at a centre offering HIV testing. If she consents to testing she must receive her HIV test result and, if infected, receive appropriate prophylaxis. She must then take the prophylaxis during labour and her infant must receive prophylaxis after delivery. Finally, the mother and infant must receive follow-up care to ensure that they have any necessary treatment and that the infant is tested for HIV, typically in the second year of life when HIV antibody testing is reliable.

Table 1.

 Indicators for monitoring and evaluation of programme to prevent mother to child transmission of HIV used by the President's Emergency Plan for AIDS Relief

	Where information recorded	Location of services and registers
Health facility indicators
No of health facilities providing services to prevent transmission in past 12 months	Programme reports and health facility surveys	Various
No (%) of practising skilled health workers in antenatal care who have received training in preventing transmission in past 24 months
No of health facilities with adequate capacity to monitor and accurately record services
No of maternity facilities with appropriate referrals at the institutional level to link HIV infected women and their infants to care and support services
No of facilities that offer appropriate advice on infant feeding during post-test counselling
No of antenatal clinics providing family planning advice services during post-test counselling
No of condoms distributed in antenatal clinics
Patient indicators
No of pregnant women who attend at least one antenatal clinic visit at a programme site	Antenatal care enrolment register	Antenatal care
No of pregnant women who receive counselling for HIV testing	Voluntary counselling and testing register
No of pregnant women accepting testing for HIV	Voluntary counselling register/HIV testing laboratory register	Antenatal care/labour ward
No of pregnant women testing positive for HIV
No of pregnant women receiving HIV test results and post-test counselling
No of women who receive counselling for recommended infant feeding practices	Voluntary counselling and testing register
% of HIV infected pregnant women receiving complete course of antiretroviral prophylaxis to reduce risk of mother to child transmission	Antiretroviral register/labour ward delivery register
No of infants born to HIV infected mothers who receive cotrimoxazole prophylaxis for the first year of life	Cotrimoxazole register	Maternal and child health
% of HIV infected children born to HIV infected mothers: programme impact indicator	Patient register

The few studies that have attempted to evaluate such programmes show the logistical, managerial, and technical challenges in delivering effective preventive services (table 2).^{5 11 12 13 14 15 16 17 18} Coverage falls progressively the further women are along the prevention cascade. Thus, 12 months after delivery, only a fraction (19% in one study in Malawi⁵) of HIV positive mothers who received antiretroviral drugs will attend health services to have their infant tested for HIV. Clearly, this may have lethal consequences for those children who become HIV positive.

Table 2.

 Studies monitoring and evaluating the effectiveness of programmes to prevent mother to child transmission of HIV

Country	Programme results*
Cameroon12	7871 women attended for antenatal care, with 86% being tested for HIV. 572 (9%) women tested positive for HIV. 427 HIV positive women were “regularly followed-up”; 92% of the 261 women that delivered received nevirapine (including their newborns). 210, 123, and 81 children were HIV tested at 6-8 weeks, 5-6 months, and 15-18 months respectively
Dominican Republic13	42 666 women attended for antenatal care. 54% were tested for HIV, and 581 (2.5%) had a positive result. 89% of the 185 HIV positive women that delivered and 183/186 delivered infants received nevirapine, with 24% of the infants being tested for HIV infection at 6 weeks
Kenya14	3564 attended for antenatal care; 70% were tested for HIV. 348 (14%) tested positive, but only 285 (82%) returned to the health centre for their test results, of whom 152 returned to receive nevirapine at 34 weeks' gestation. 106 took nevirapine during labour. No information on follow-up of infants
Kenya15	Data from two periods. 8231 women attended antenatal services, with 5652 (69%) women being tested for HIV. 5077 (90%) of tested women returned to the health centre for their results, of whom 1207 (24%) were HIV positive. 773 (64%) of women reported taking nevirapine. No information is provided on follow-up of infants
Malawi5	3136 women attended antenatal services, 2996 had pretest counselling, and 2965 (95%) were tested. 646 (22%) women were HIV positive. 288 (45%) returned to obtain nevirapine at 36 weeks' gestation, with 206 returning to the hospital for delivery. 222 of the delivered infants received nevirapine, and 122 returned at 6 months to be tested for HIV
South Africa16	1234 (15%) of 8221 babies delivered had HIV positive mothers. >95% of newborns were given nevirapine; 705 (56%) returned for postnatal follow-up at 2 weeks, but the number missing appointments at 4 months was substantial (79% in the routine cohort of patients)
South Africa17	Of 84 406 women attending for antenatal care 56% agreed to HIV testing; 30% tested HIV positive, 55% of whom received nevirapine. 12 months after delivery, only 50% of mothers who had received nevirapine attended health services to have their infant tested for HIV
Zambia11	Among 8787 women that delivered in surveyed health centres, 2257 (26%) had cord blood that was positive for HIV. 1112 of these HIV positive women knew their HIV status and were given nevirapine to take themselves. Of these, 751 (68%) had nevirapine in their blood at time of delivery. 675 (90%) infants born to these mothers received nevirapine after birth
Zimbabwe18	Of 2298 women who had counselling, 93% agreed to HIV testing, with 74% of those tested returning to collect their test result. Of 437 (20%) women that tested HIV positive, 24% received complete mother-child prophylaxis with nevirapine

*Methods and duration of monitoring and evaluation varied considerably.

The effectiveness of the programmes is compromised if there is a low coverage at any stage of the cascade. Monitoring and evaluation of each stage are thus critical to identify any weakness in the implementation and to develop strategies for improving coverage where necessary.

Challenges of measuring effectiveness

Relying exclusively on the indicators listed in table 1 has several limitations. Firstly, indicator data are commonly collected from several patient registers in the form of unlinked monthly summary reports.¹⁶ Consequently, for example, data on the number of pregnant women who tested HIV positive, received prophylaxis, and then delivered children who also received prophylaxis are not readily available. To obtain linked data, patient registers would need to be thoroughly cross-checked, which would be time consuming and cumbersome without computerisation and planning. The multitude of registers also adds a potential source of human error, with health workers sometimes forgetting to complete them. As a result, different registers may report different denominators.¹⁶

Secondly, the reported HIV prevalence rate in pregnant women is typically derived from those accepting voluntary counselling and testing. These women have a lower prevalence than those who refuse testing, probably because women at risk of HIV are less likely to volunteer for testing.^{11 16}

Thirdly, in countries where same day HIV testing is unavailable, 15-67% of women who are tested do not obtain their test results and, hence, may not receive prophylaxis.^{17 18 19} Even in those countries where rapid testing is available, integration of testing services into antenatal care is essential to ensure that women receive prophylaxis.

Fourthly, and perhaps most importantly, none of the indicators listed in table 1 objectively measure adherence to the prophylactic drugs, and there can be substantial discordance between the proportion of women who agree to prophylaxis and the proportion of women who actually take the drugs. In an extensive surveillance study carried out in Zambia in 2003, only 68% of HIV infected women given a nevirapine tablet during antenatal care to take at the onset of labour had the drug in their umbilical cord blood at delivery.¹¹ Moreover, both the woman and her infant received nevirapine in only 30% of cases despite preventive services being universally available in public antenatal clinics at the time of the evaluation.¹¹ The unexpectedly low level of population coverage highlights the importance of using objective and definitive surrogates to measure the effectiveness of programmes. Once alerted, the Zambian team was able to make improvements at every level of the prevention cascade, increasing coverage as a consequence.

Fifthly, most indicators of programme effectiveness are quantitative rather than qualitative. The quality of services offered at health facilities affects supply and demand for voluntary testing, uptake of antiretroviral drugs, and other activities such as family planning.^{7 11 16 17 18} Also, current indicators do not capture possible adverse effects of the activities, whether related to the antiretroviral drugs or to the revelation of maternal or child infection status (such as, stigmatisation and domestic violence).

Finally, current indicators fail to evaluate effectiveness in terms of coverage and linkage to care for mothers, infants, and family members. As shown above, rates of postnatal follow-up can be very low.^{5 11 12 13 14 15 16 17 18} None of the indicators in table 1 capture postnatal follow-up rates so that, for example, the effect of breast feeding can be accurately assessed. Instead, estimates of infant coverage and infection prevented are modelled on nevirapine uptake data or from a limited number of cohort (research) studies.

Where to go from here?

Ideally programmes should carry out sequential prevalence studies or cohort studies in infants. These studies would reliably estimate infant HIV infections under operational conditions rather than extrapolate figures from clinical trial or “model programme” data. Although these studies could certainly be included in larger programmes, they are labour intensive and costly. For example, infants born to HIV infected mothers would have to be followed up for 15-18 months to rule out transmission. The care benefits of proper follow-up that accrue to mother and infant are substantial, so evaluation costs will be linked to service expenditures. Depending on venue and source of funding, ethical mandates to improve the standard of care for study participants would also reduce the generalisability of evaluations of enhanced prevention programmes and follow-up care. Thus, this type of evaluation would represent a best case scenario. Moreover, cohort studies will not yield data about the programme's coverage unless explicitly designed to do so.

One approach to measuring effectiveness is through anonymous screening of umbilical cord blood samples for HIV antibodies. Positive samples would then be tested for nevirapine (an indicator of maternal nevirapine coverage) using high performance liquid chromatography¹¹ or the cheaper but less sensitive thin layer chromatography followed by high performance retesting of samples with negative results.²⁰ Investigators in Zambia have used cord blood analysis combined with chart documentation of infants' receipt of nevirapine to measure true population coverage.¹¹ When combined with patient level indicator data extracted from a woman's antenatal record at delivery, this approach provides an objective measure of effectiveness both for women who participated in the programme and in the broader population of women delivering in public labour wards.

Collection of patient data must be simplified and linked. In the absence of fully computerised, real time collection of patient health data,²¹ such linkage could be obtained by incorporating a small coded stamp into the medical record of women attending antenatal care (figure). A first generation of such a stamp was used when evaluating nevirapine coverage in pregnant women in Zambia²² and is being completed by health staff at each Zambian programme site during routine antenatal care and labour ward visits. The stamp provides a place to record important data for monitoring and evaluating prevention programmes. After delivery, the medical records are left in the health facilities' maternal and child health department, where the data can be readily entered on to computerised databases for analytical purposes or to aid clinic staff in postnatal outreach of infants born to HIV infected women.

Example of data collection stamp for use on antenatal care and labour records. Spaces shown represent where the dates of the given activity are recorded; multiple lines are provided for documentation of counselling or testing during follow-up visits. CR=counselling refused; CA=counselling accepted; TR=testing refused; TA=testing accepted; R=reactive; NR=non-reactive; I=indeterminate; MGN=mother given nevirapine (during antenatal care); MRN=mother refused nevirapine; MIN=mother ingested nevirapine (at delivery); MRIN=mother refused to ingest nevirapine; IGA=infant given antiretroviral drugs; INGA=infant not given antiretrovirals; IFB=infant feeding breast; IFR=infant feeding replacement

Use of a data stamp, combined with the umbilical cord blood analysis, permits collection of anonymous linked maternal and infant data—for example, by combining data on maternal programme indicators, maternal population drug coverage, and data on infant population drug coverage. This is crucial when attempting to identify shortcomings in the implementation of programmes and to estimate the proportion of children who will become infected with HIV. Moreover, inclusion of a modified version of the stamp in an infant's immunisation card may remind health staff of the need to test the infant for HIV and thereby improve postpartum follow-up rates (at least among those who return for immunisations).

The data stamps must be coded to protect the confidentiality of the mother and infant. Confidentiality can be violated only if a healthcare provider reveals the meaning of the stamp to a third party. This is analogous to a medical provider revealing the contents of a patient's medical record. In fact, a coded stamp would be more secure than standard medical record notes because it could not be interpreted without an explanation, whereas a third party might read records successfully without help. Nevertheless, before stamps are widely implemented we need pilot research to assess their feasibility and acceptability in a given clinical and cultural context.

We recognise that the above approach may be unsuitable for areas where a small proportion of women deliver in public labour wards (such as Haiti), where there is insufficient laboratory capacity to process and store cord blood specimens, and where programmes lack funds for high performance liquid chromatography analysis. The cost of analysing each specimen is around $50 (£25; €37),²⁰ although high volumes should reduce costs. Nevertheless, a coded stamp could still enhance monitoring and evaluation efforts by allowing for simplified, periodic extraction of data. These data could verify the data presented in monthly monitoring reports and identify any shortfalls in the programme.

For programmes that use more complex antiretroviral regimens (such as short course zidovudine plus single dose nevirapine or nevirapine-containing highly active antiretroviral therapy), the absence of nevirapine in the cord blood would not necessarily imply a woman was not at least partly covered, assuming she adhered to the zidovudine portion of the regimen. Although zidovudine can be detected in cord blood,²³ it is a less perfect surrogate for infant infection than nevirapine because it has a shorter half life and detection in the cord blood does not prove full compliance with the longer zidovudine regimen.

Other methods of measuring the effectiveness of these programmes have been reported. In South Africa, for example, facility based case finding approaches have been used to identify infants born to women accessing programme services,²⁴ and mathematical modelling has also been used.²⁵ However, these approaches do not separate each element of the prevention cascade, making it hard to pinpoint a programme's failures and successes.

Conclusion

Currently, less than 10% of HIV infected pregnant women receive antiretroviral prophylaxis.²⁶ Scaling-up of programmes to prevent mother to child transmission of HIV remains a huge challenge. Programmes supported by organisations with a high media profile, such as the President's Emergency Plan for AIDS Relief, are often subject to substantial political and public pressure to report successes rather than failures. Given the amount of public and private sector investments into these programmes and the programmes' potential effect on a country's societal and economic development, we believe that rigorous monitoring and evaluation of operational programmes is a moral and ethical imperative. Evaluation is essential to identify shortcomings in the programme and to conceptualise approaches to improve services. This, in turn, will improve a programme's cost effectiveness and long term sustainability and save the most infant lives.

Summary points

• Programmes to prevent mother to child transmission of HIV are an important part of global AIDS initiatives

• Data on the effectiveness of these programmes are scarce

• Current indicators to monitor and evaluate the programmes are inadequate

• Comprehensive indicators that monitor all stages of the prevention cascade are urgently needed