Abstract
This study evaluates the efficiency of rural health centers in Rwanda in delivering the three key human immunodeficiency virus/acquired immunodeficiency syndrome services: antiretroviral treatment, prevention of mother-to-child transmission, and voluntary counseling and testing using data envelopment analysis, and assesses the impact of community-based health insurance (CBHI) and performance-based financing on improving the delivery of the three services. Results show that health centers average efficiency of 78%, and despite the observed variation, the performance increased by 15.6% from 2006 through 2007. When the services are examined separately, each 1% growth of CBHI use was associated with 3.7% more prevention of mother-to-child transmission and 2.5% more voluntary counseling and testing services. Although more health centers would have been needed to evaluate performance-based financing, we found that high use of CBHI in Rwanda was an important contributor to improving human immunodeficiency virus/acquired immunodeficiency syndrome services in rural health centers in Rwanda.
Introduction
Human immunodeficiency virus/acquired Immunodeficiency syndrome (HIV/AIDS) is endemic worldwide. In 2008 when this study began, 2.7 million new HIV infections occurred, 2.0 million persons died, and 33.4 million persons were living with HIV infections worldwide.1 To address this epidemic, the international community has mounted a substantial response. In 2009, 5.25 million persons accessed life-prolonging antiretroviral therapy (ART), up 1.2 million from 2008. However, the Joint United Nations Programme on HIV/AIDS (UNAIDS) estimated a $10 billion shortfall to meet universal access and noted the need “to enhance the efficiency of programs” and to build “systems for a sustainable response.”2
To address this financial concern for HIV/AIDS, it is important to understand the efficiency of HIV/AIDS programs and the impact of innovative health policies on use of key HIV/AIDS services. The response to HIV/AIDS in Rwanda, where seroprevalence is 2.67%,3 provides an opportunity for such evaluation. The Rwandan government has enacted numerous policies to reshape and restructure the health delivery system. Rwandan health officials understood that effective health care systems were fundamental in providing quality services for populations in need and were critical in addressing the formidable challenges of HIV/AIDS and other diseases.4 Of all policies implemented, performance-based financing (PBF) and community-based health insurance (CBHI) are the most notable.
The PBF started in 2002 and sought to encourage the provision of key health services by rewarding productive health centers. In doing so, policy makers are hoping that health centers would be motivated to deliver more essential medical services, such as HIV/AIDS counseling, facility-based deliveries, and vaccinations. The PBF was implemented first as a pilot study, then expanded to 74 health centers in 2005 and to 85 health centers in 2006.5 The government has since expanded it nationally. Key HIV/AIDS services, including voluntary counseling and testing (VCT), prevention of mother-to-child transmission (PMTCT), and ART for AIDS patients, are among the incentivized indicators.6 With incentives, use of HIV/AIDS services was expected to increase as health facilities sought to gain more revenues by providing more of those services.
In addition to providing incentives to providers using PBF, Rwanda also launched CBHI in an effort to increase the demand for health services at nearly the same time as PBF was implemented. The CBHI increases the involvement of communities in solving health-related issues through community-based programs and reduces consumers' out-of-pocket payment for health services.7 According to administrative data, by 2008, 85% of the Rwandan population was covered by CBHI.8 A 2010 household survey estimated coverage at 68%,9 and both rates represented substantial increases over earlier years.
Although free of charge, use of HIV/AIDS services could be improved from demand-side incentives through several possible mechanisms, First, CBHI increases the demand for primary care, which improves the use of HIV/AIDS services because of increasing integration of maternal and child services (e.g., prenatal care) with HIV/AIDS services (e.g., PMTCT). Second, the improved quality of services from PBF could also attract patients to use HIV/AIDS services. Third, more regular contacts with health providers because of lower treatment expenditure could increase awareness of using HIV/AIDS services.
Despite several evaluations of these policies in Rwanda,4–6,10,11 several gaps remain. First, the previous studies rarely took inputs (i.e., expenses of health facilities) into consideration when evaluating these two policies. By including inputs in the evaluation, the study can assess whether and how these policies improve the efficiency of health service delivery.12,13 Second, some previous evaluations at the national level were unable to distinguish the effects of each of these two policies from each other.5,10,11 Because PBF and CBHI were implemented almost simultaneously in Rwanda, an evaluation of one policy will often overstate its effectiveness if the other one is omitted.
In this study, we use the data collected at 26 rural health centers in Rwanda to 1) evaluate the efficiency of each health center in providing HIV/AIDS services and 2) examine and separate the impact of PBF and CBHI on HIV/AIDS service delivery with multivariate analyses. We selected services for HIV/AIDS as the focus for this study because of this condition's prominence among health problems in Rwanda and relatively good availability of data.
Materials and Methods
Selection of health centers.
The number of health centers providing HIV/AIDS services in Rwanda has increased steadily to improve the geographic accessibility for the AIDS-affected population. The evolution of the number of health centers delivering ART services during January 2003–January 2007 are shown in Figure 1. The 90 health centers providing ART services in 2007 constitute our study population.
Figure 1.
Cumulative number of health centers providing antiretroviral treatment in Rwanda, 2003–2007. Source: Authors' calculations were obtained from TRACnet data.
To analyze the evolving delivery of HIV/AIDS services, we stratified HIV/AIDS health centers based on their starting date of ART. We selected all six rural health centers that started ART services during October 2003–June 2005, and randomly selected 20 of the 38 health centers that began ART services in 2006 for a total sample of 26 rural health centers. This overall sample, planned jointly with a parallel study14 constituted 59% of eligible health centers.
Measurements and data collection.
We followed a classical framework of economic analysis of efficiency using data envelopment analysis15 with direct inputs and outputs. The inputs of production of HIV/AIDS services included personnel and non-personnel HIV/AIDS spending.
Number of personnel.
We obtained a personnel list from each health center and directly asked employees who were still working there to allocate their time between work on HIV/AIDS services and non-HIV/AIDS services. For staff no longer at the health center, we asked the director to estimate their time allocation between AIDS and non-AIDS services, augmented by recalls from their former colleagues still working in the health center. Based on the time allocations, we divided the number of full-time equivalent (FTE) personnel into two categories: number of FTE for HIV/AIDS services, and number of FTE for non-HIV/AIDS services.
Non-personnel HIV/AIDS spending.
To derive this spending, we first calculated the total revenue in each health center by summing up the revenue from CBHI, the revenue from patients' out-of-pocket payments to the health center, and the annualized monetary value of various donated goods (e.g., drugs and test kits) and devices. We assumed that the health center had a balanced budget (the total expenditure equaled the total revenue) because it generally must over the long run as a government or non-profit entity. We then subtracted personnel salaries from the total revenue to obtain non-personnel expenses. To estimate the non-personnel expenses specific for HIV/AIDS services, we separated the donated goods for HIV/AIDS services only (e.g., antiretroviral drugs and rapid HIV/AIDS test reagents) from those for general medical services, which could be spent on HIV/AIDS and non-HIV/AIDS services. We used the ratio of HIV/AIDS personnel to total personnel to allocate the non-personnel expenses for general medical services. Thus, the non-personnel HIV/AIDS spending was calculated as the sum of the annualized monetary value of donated goods only for HIV/AIDS services and the portion of non-personnel general cost attributed to HIV/AIDS services. All expenses and revenues were first expressed in current Rwandan Francs (RWF) in the year of the transaction and then converted to 2008 RWF by adjusting for inflation.
The outputs for this study included the number of clients receiving VCT services; the number of participants in prevention of mother-to-child transmission (i.e., the number of pregnant women and their partners receiving testing and counseling for HIV (PMTCT); and the number of AIDS patients receiving ART. Data on PMTCT and VCT services were collected from health centers. To reduce recall bias, we collected the information on PMTCT and VCT on a quarterly basis and imputed it to a yearly quantity if the health center began offering ART before July 1 of the designated year. The variable showing the number of HIV+ pregnant women initiating ART treatment could also serve as a measure of PMTCT services. Because of the high correlation between numbers of pregnant women initiating ART and the number of pregnant women receiving testing and counseling (correlation coefficient = 0.78, P < 0.001), we selected the latter indicator, which had more complete data, as the proxy for measuring PMTCT in this study. For health centers that started ART after July 1 of the year, that year was removed from the analysis. Although we collected the number of visits of AIDS patients for ART quarterly, for consistency with international standards, we instead used the number of patients receiving ART obtained from the government agency managing HIV/AIDS services, the Treatment Research AIDS Center as the indicator measuring the quantity of ART services. We calculated the annual average number of patients on ART by averaging the monthly reported numbers.
To examine the impact of the two policies (PBF and CBHI) on HIV/AIDS service delivery, we included measures of the two policies.
PBF phase.
The implementation of PBF among health centers in Rwanda occurred in phases. The PBF was first launched in 2002 by international non-governmental organizations.16 In 2006, the World Bank initiated a larger pilot experiment,6 under which participating health centers were randomly assigned to phase 1 or 2. Those in phase 1 were paid based on the volume and the quality of a list of services provided, and those in phase 2, serving as control sites, received a fixed amount of incentive payment. The remaining health centers received no payment (phase 0).
Use rate of CBHI.
Because coverage of the catchment area of each health center by CBHI was not generally documented, we instead calculated a proxy, termed the CBHI use rate. We collected the revenue data from CBHI (the CBHI payments for services received by members) in each health center in RWFs. The proxy, which approximates the share of population using CBHI funding to pay for health care services, was calculated as [revenue from CBHI]/[CBHI premium]/population of catchment areas of the health center.
Most data required for the analysis were collected through a semi-constructed questionnaire designed by the research team. Trained field researchers collected the data at health centers and conducted interviews with administrators of the health center.
If we had excluded all observations with any missing data, we would have greatly reduced the statistical power of the analysis. Instead, we imputed missing values, approximately 2% of total data, by replacing them by the mean of corresponding numbers from two observations for which all other known information matched closest to that for the one with missing values, a hot deck procedure.17
Statistical methods.
Data envelopment analysis.
We used a traditional data envelopment analysis (DEA) model and the DEA/Malmquist model to evaluate the efficiency of health centers. Because DEA/Malmaquist requires strictly balanced data, we used only data from 2006 and 2007 for both DEA models. Performance was defined as the ratio of weighted outputs to weighted inputs. Our traditional DEA approach evaluated how efficiently each health center delivered HIV/AIDS services compared with the best health centers in our study sample in the same year. It used an output-oriented DEA model developed by Charnes and others (Banker, Charnes, and Cooper evaluation) with variable return to scale.18 Unlike conventional parametric approaches, DEA is capable of modeling production with multiple inputs and multiple outputs by comparing each decision making unit to a production frontier defined by the best performers from data, and it has been applied in many settings to evaluate the performance of various types of organizations and programs.19–23 As mentioned above, the inputs were the number of FTE and non-personnel HIV/AIDS expenditures, and the outputs were the quantities of the three HIV/AIDS services. The output-oriented DEA model calculated an efficiency score for each health center in delivering HIV/AIDS services in 2006 and 2007 separately.
Using the same inputs and outputs, the DEA/Malmquist analysis24,25 was applied to estimate the change in efficiency for health centers from 2006 and 2007. Because we had two years of data, there were separate production frontiers for each year. The DEA/Malmquist analysis provides estimates for three components of change. The technical change captures the effect of the shift of production frontiers (frontier shift effect). The technical efficiency change measures the ability of decision making unit to catch up on the shift of production frontiers (catch-up effect) by comparing the efficiency of that unit over the two years when the efficiency is calculated using the production frontier in the same year to which data refer. The total factor productivity change measures the overall efficiency change, which is the product of the technical change by the technical efficiency change. If a specific index is > 1, then the corresponding component increased in efficiency. In contrast, a number < 1 denotes a decrease in that efficiency.24,25
Regression analysis.
To understand the impact of these two policies on providing the three services separately, we used the classic framework of a Cobb-Douglas production function allowing only one output to be analyzed at a time,
 |
where Y represents product (output), L labor input, K capital input, and M other factors.
Applying this framework specifically to this study, the model can be written as
 |
where yit is the quantity of ART, or PMTCT, or VCT in tth year for ith health center, hp equals HIV/AIDS FTE personnel; nhp equals non-HIV/AIDS FTE personnel; hspend equals non-personnel HIV/AIDS spending; PBF equals status of performance based financing (1 = with PBF [phase 1], 0 = without PBF [phase 0 or phase 2]). We used the dummy variable for PBF because the number of health centers with phase 2 PBF was small, approximately 6% of the total number of observations; CBHI equals CBHI use rate; ui equals health center individual effect; and ε equals the random error and time equals dummies of years. For this analysis, we used the available data for 2002 through 2007.
In constructing this model, instead of including dummies for all years, we included a dummy variable for year 2007 only, given that 1) the number of observations from 2002 through 2005 was relatively small, and 2) the major policy initiatives, such as PBF and CBHI, began in 2006 so that the fixed effect in that year, if included, might underestimate the effect of these two policies. Not all the health facilities had values of zeros for the variables of PBF and CBHI for years 2002 through 2005 because a few health facilities in the sample were involved in pilot PBF and CBHI schemes during the period.
Both fixed-effects and random-effects models were conducted and the results were compared by using the Hausman test. We report results only from the fixed-effects model because the Hausman test showed a statistically significant difference (P < 0.05) in coefficients between the fixed and random effects models, suggesting that the fixed-effects model is more appropriate. In addition, the fixed-effects models help resolve the measurement challenges of using the CBHI use rate as a proxy for CBHI coverage by health center because the fixed-effects models consider only the within-variation in each health center.
Results
Descriptive analysis.
Key variables for the analysis of 2006–2007 data are summarized in Table 1. Consistent with the Rwandan national statistics,26 we found that all the health centers in the study had revenue data available from CBHI in 2006 and 2007. The CBHI use rate varied among catchment areas. Across all health centers in our data set, the CBHI use rate increased from 31.71% in 2006 to 40.30% in 2007; these rates are consistent with the increasing trend in CBHI coverage over the period from household surveys and aggregate administrative data.8,9 Similarly, PBF also expanded during the study period. Seventy-seven percent of the health centers in the study had implemented PBF by 2006; by 2007, all the health centers had either phase 1 or phase 2 PBF. We also found that both inputs (the number of personnel and the amount of HIV/AIDS non-personnel spending) and outputs (the quantity of services) increased from 2006 though 2007.
Table 1.
Summary of variables for 26 health centers, Rwanda, 2006–2007*
| Variable | 2006 | 2007 |
|---|---|---|
| PBF (no. study health centers) | ||
| Phase 0 | 6 | 0 |
| Phase 1 | 17 | 21 |
| Phase 2 | 3 | 5 |
| CBHI use rate, % | 31.71 ± 18.27 | 40.30 ± 17.37 |
| No. HIV/AIDS FTE personnel | 5.37 ± 4.07 | 5.86 ± 3.86 |
| No. non-HIV/AIDS FTE personnel | 17.32 ± 9.42 | 16.76 ± 7.33 |
| Amount of non-personnel expenditure on HIV/AIDS (million RWF in 2008) | 13.10 ± 11.79 | 18.60 ± 16.68 |
| No. clients receiving VCT | 2,104 ± 1,141 | 2,829 ± 1,393 |
| No. patients receiving ART | 73 ± 123 | 148 ± 169 |
| No. visits of pregnant women receiving PMTCT | 1,021 ± 271 | 1,028 ± 316 |
Values for PBF are numbers of health centers. All other values are mean ± SD. PBF = performance-based financing; CBHI = community-based health insurance; HIV/AIDS = human immunodeficiency virus/acquired immunodeficiency syndrome; FTE = full-time equivalent; RWF = Rwanda francs; VCT = voluntary counseling and testing; ART = antiretroviral treatment; PMTCT = prevention of mother-to-child transmission.
Data envelopment analysis/Banker, Charnes, and Cooper evaluation.
Efficiency scores for each health center based on an output-oriented DEA variable return to scale model for 2006 and 2007 are shown in Table 2. We found a high average efficiency of health centers in providing AIDS services in 2006 and 2007. In 2006, 11 health centers performed at maximum efficiency in delivering HIV/AIDS services. The geometric mean efficiency of all health centers that year was 82.0%. In 2007, eight health centers performed with 100% efficiency, and the geometric mean efficiency decreased to 73.9%.
Table 2.
Efficiency scores and efficiency changes in 26 health centers, Rwanda, 2006–2007*
| Health center | 2006 | 2007 | TC | TEC | TFPC | ||
|---|---|---|---|---|---|---|---|
| Efficiency | Rank | Efficiency | Rank | ||||
| Bungwe | 1.000 | 1 | 1.000 | 1 | 1.328 | 0.991 | 1.315 |
| Congo-Nil | 0.694 | 21 | 0.776 | 12 | 1.523 | 1.055 | 1.607 |
| Gakenke | 0.787 | 16 | 1.000 | 1 | 1.103 | 1.306 | 1.441 |
| Gisagara | 0.580 | 24 | 0.593 | 20 | 0.850 | 1.186 | 1.008 |
| Kabuye | 0.713 | 19 | 0.522 | 23 | 1.450 | 0.584 | 0.846 |
| Karengera | 0.665 | 22 | 0.565 | 21 | 1.368 | 0.726 | 0.993 |
| Kibilizi-Gisagara | 1.000 | 1 | 1.000 | 1 | 3.247 | 1.000 | 3.247 |
| Kirambo | 1.000 | 1 | 1.000 | 1 | 1.129 | 0.865 | 0.976 |
| Muhura | 1.000 | 1 | 1.000 | 1 | 1.419 | 0.334 | 0.474 |
| Mukono | 0.942 | 12 | 0.721 | 15 | 1.443 | 0.789 | 1.138 |
| Munzanga | 0.498 | 26 | 0.383 | 26 | 1.472 | 0.682 | 1.003 |
| Musha | 1.000 | 1 | 0.532 | 22 | 1.007 | 0.503 | 0.506 |
| Mwezi | 0.553 | 25 | 0.701 | 17 | 1.890 | 0.726 | 1.372 |
| Nyamasheke | 1.000 | 1 | 0.846 | 11 | 1.924 | 0.328 | 0.631 |
| Ramba | 0.828 | 15 | 0.959 | 9 | 1.527 | 0.870 | 1.329 |
| Remera | 0.886 | 14 | 0.741 | 14 | 1.499 | 0.649 | 0.973 |
| Remera-Rukoma | 1.000 | 1 | 0.766 | 13 | 1.604 | 0.564 | 0.905 |
| Rugarama | 1.000 | 1 | 1.000 | 1 | 1.057 | 1.000 | 1.057 |
| Rurenge | 1.000 | 1 | 0.470 | 24 | 1.550 | 0.386 | 0.598 |
| Rusumo | 0.749 | 17 | 0.693 | 18 | 1.941 | 0.605 | 1.173 |
| Rwankeli | 1.000 | 1 | 1.000 | 1 | 3.635 | 1.299 | 4.722 |
| Rwankuba | 0.903 | 13 | 0.655 | 19 | 1.495 | 0.609 | 0.911 |
| Rwaza | 0.584 | 23 | 0.705 | 16 | 1.899 | 1.466 | 2.783 |
| Rwesero | 0.702 | 20 | 0.449 | 25 | 1.740 | 0.538 | 0.935 |
| Rwinkwavu | 1.000 | 1 | 1.000 | 1 | 1.747 | 1.000 | 1.747 |
| Ryamanyoni | 0.719 | 18 | 0.871 | 10 | 1.792 | 0.737 | 1.321 |
| Geometric mean | 0.820 | – | 0.739 | – | 1.557 | 0.743 | 1.156 |
TC = technical change; TEC = technical efficiency change; TFPC = total factor productivity change. TFPC = TC × TEC.
Performance of health centers varied widely (Table 2). In several health centers, Gisagara, Munzanga, Mwezi, Rwaza in 2006, and Gisagara, Kabuye, Karengera Munzanga, Rurenge, and Rwesero in 2007, the efficiency was < 60%. We calculated the average efficiency score for these low-efficiency health centers and found that those health centers would provide at least 67% more AIDS services if they were fully efficient.
There were 58% (15 of 26) and 69% (18 of 26) health centers that did not operate in their full capacity in 2006 and 2007, respectively. For inefficient health centers, the efficiency scores, on average, were 72% and 66% in 2006 and 2007, respectively, indicating that those inefficient health centers would have increased the HIV/AIDS services by 31% ([1–72%]/72%) and 50% ([1–66%]/66%) if they performed as well as efficient health centers.
When comparing crude average efficiency scores between 2006 and 2007, we might conclude that health centers had higher productivity in 2006 than 2007. However, this conclusion is not necessarily sound because the health centers were compared with different production frontiers. If a production frontier shifts outwards too far for some health centers to catch up, health centers may show a lower efficiency score but actually have higher productivity than the earlier year.
Data envelopment analysis/Malmquist evaluation.
We used the DEA/Malmquist model to analyze efficiency changes over time. The magnitude of the technical change, technical efficiency change, and total productivity change (Malmquist index) for each health center and health centers as a whole are shown in Table 2. The results show that from 2006 through 2007, the total productivity of delivering HIV/AIDS services in most health centers improved. The geometric mean of technical change was 1.557, indicating that the production frontier in 2007 moved outwards by 56% compared with that in 2006. The technical efficiency change (0.743) is < 1, showing a decrease of 26% in the efficiency of health centers relative to their best peers in the same year because some health centers could not keep pace with the frontier shift. The total productivity index was 1.156, suggesting an increase of approximately 16% of total productivity of health centers over this period.
Similar to the observations described above when evaluating the health centers separately for 2006 and 2007, the results from DEA/Malmquist analysis also indicated a great variation in efficiency changes. The overall efficiency change ranged from 0.474 (Muhura) to 4.722 (Rwankeli). Some health centers (e.g., Rwankeli and Kibilizi) improved their efficiency substantially.
Multiple regression analysis.
The results shown in Table 3 indicate that CBHI tended to improve PMTCT and VCT services (P < 0.10) but not ART services. Coefficients for the CBHI use rate of 0.037 on PMTCT and 0.025 on VCT mean that if the CBHI use rate increased by 1%, the number of pregnant women or their partners who had been tested for HIV and the number of persons who received VCT services would increase by 3.7% and 2.5%, respectively.
Table 3.
Impact of CBHI, PBF, and other variables on HIV/AIDS services, Rwanda*
| Variable | Log ART | Log PMTCT | Log VCT |
|---|---|---|---|
| HIV personnel | 0.094 | 0.086 | 0.038 |
| Non HIV personnel | 0.056 | −0.001 | 0.009 |
| PBF | 0.294 | 0.870† | 0.459 |
| CBHI use rate, % | 0.015 | 0.037‡ | 0.025† |
| Log of HIV expenditure | 0.730‡ | −0.077 | 0.191§ |
| Year 2007 | 1.065§ | −0.363 | −0.031 |
| Constant | −10.702 | 6.468 | 2.928 |
n = 156. Fixed-effects models were used. CBHI = community-based health insurance; PBF = performance-based financing; HIV/AIDS = human immunodeficiency virus/acquired immunodeficiency syndrome; ART = antiretroviral treatment; PMTCT = prevention of mother-to-child transmission; VCT = voluntary counseling and testing.
P < 0.10.
P < 0.05.
P < 0.01.
Comparison of these coefficients suggests that PMTCT services were more responsive to CBHI than VCT. Two reasons may explain this finding. First, pregnant women may increase their use of medical services as a result of lower co-payments because of CBHI. Second, all pregnant women attending health centers are required to be tested for HIV. The results show the contribution of CBHI in strengthening the linkage between maternal health services with HIV services.
The only service found to be significantly affected by PBF was PMTCT. The PBF was associated with an increase of PMTCT services by 87%. The coefficients of PBF for ART and VCT were 29.4% and 45.9%, respectively, but were not statistically significant. One factor was the limited variation in the independent variable: on average, 73% of the health centers were in phase 1 of PBF. The large coefficients imply that implementation of PBF is perhaps advancing towards one of its designated goals of improving HIV/AIDS services in Rwanda. Conversely, despite the large and positive magnitude, the absence of statistical significance for the coefficients of PBF for VCT and ART suggests a wide variation of performance among health centers after switching PBF status.
Discussion
To our knowledge, this is the first study to measure the efficiency of health centers in delivering HIV/AIDS services in Rwanda. Several studies have examined the variation of the costs of delivering HIV/AIDS services in other countries.27–29 The study found that health centers, on average, performed well overall in 2006 and 2007, and showed an average efficiency of 78%, but with great variation. Similar to findings in Sierra Leone,30 Burkina Faso,31 Zambia,32 and Ghana,33 our results suggest there is room for further enhancement of service delivery at the health center level in low- and middle-income countries. Inefficiency may be caused by improper allocation of resources, limited human resource capacity, inaccessibility of health services for remote populations, and low demand.31 With limited resources for HIV/AIDS, strengthening and improving the efficiency of service delivery through complementary policies becomes increasingly imperative to address the dilemma that the HIV/AIDS epidemic remains rampant while funding for HIV/AIDS is decreasing.
We also found that the overall efficiency increased by 16% in 2007. Some health centers improved substantially over the two years, such as Kibilizi, Rwaza, and Rwankeli. Year 2006 was important in Rwanda because it marked the initiation of important health policies. Both CBHI and PBF were expanded to cover more population for better health. For all the three health centers, we observed a substantial increase of the CBHI use rate; Kibilizi increased from 38% to 84%, Rwaza from 62% to 71%, and Rwankeli from 10% to 57%. The limited data do not allow us to quantify determinants of efficiency improvement, but the consistency of increase of CBHI use rates in the three health centers with the most improvement suggests the potential association of CBHI and efficiency of HIV/AIDS service delivery.
The most convincing finding from this study is that CBHI had a significant and highly favorable impact on delivering HIV/AIDS services. Although ART, PMTCT, and VCT were provided free of charge in Rwanda, use of PMTCT and VCT, which was highly influenced by recommendations of providers, was improved by CBHI. Two factors are probably responsible. First, CBHI increases the health service use rate14 and to have more contact with the health care system with all training and sensitization messages. For PMTCT, as all pregnant women receiving care are expected to be tested for HIV/AIDS, and CBHI indirectly improved use of PMTCT through increased use of health services (i.e., antenatal care) among pregnant women. Second, the increase of CBHI enrollment has helped increase the population's concern about their health and awareness of HIV/AIDS, encouraging persons to be tested for HIV/AIDS.34 Because ART services are heavily dependent on the number of persons identified through VCT and PMTCT services, we expect that CBHI will increase ART in later years after it improves rates of PMTCT and VCT. The likely lag may explain why the effect of CBHI on ART in this study was not statistically significant.
The finding that CBHI increases the use of PMTCT and VCT suggests an alternative way to finance HIV/AIDS services efficiently and sustainably. Most funding from international donors, such as the Global Fund to Fight AIDS, Tuberculosis and Malaria and the U.S. President's Emergency Plan for AIDS Relief, are used to subsidize providers directly. Thus, most HIV/AIDS services such as ART, PMTCT, and VCT are provided for free of charge to the patient in many recipient countries. However, few programs have explored alternative ways to promote the use of AIDS services by subsidizing consumers rather than providers. Policy makers have debated whether subsidy on the demand or provider side is more efficient in increasing use of essential health services. This debate is more critical in settings where the high cost of HIV/AIDS services is still a barrier to seeking AIDS services, and when recipient countries can no longer rely heavily on international donors for funding HIV/AIDS services. The results of this study complement the growing evidence of the beneficial effect of demand side subsidy for health service delivery.35 Given the finding that CBHI improves the efficiency of HIV/AIDS services, investing resources for health insurance would yield multifaceted benefits, not only relieving the resource constraints on HIV/AIDS with improved efficiency,36 but also facilitating the services delivery with increased demand.
This study confirms that even in resource-limited countries, providing health insurance for the rural population is feasible and can greatly improve the accessibility of important HIV/AIDS services such as PMTCT and VCT. The CBHI is advocated by the World Bank to finance health in many developing countries, but has failed in some countries.37 The major reasons for the failure are weak management, poor quality health services, and the small pool of resources to be mobilized.35,37 Strategically, Rwanda circumvents those barriers and has implemented CBHI with great success38 by empowering communities through elected leaders in the management of CBHI at the health center level and taking local innovative strategies to reach agreed goals within a context of government decentralization, building capacity of the government and local communities in managing the insurance schemes and associating donors to support CBHI in a sustainable vision. The strong government and community involvement in CBHI in Rwanda have overcome the weaknesses often found in many other countries.
This study was not able to provide empirical evidence of the positive impact of PBF on use of VCT and ART. Although PBF has 10 indicators on HIV/AIDS in its evaluation list, the only substantial (and borderline significant) effect of PBF was on PMTCT, but not on ART or VCT. There are two possible reasons. First, the Rwanda government places high priority on maternal health and PMTCT, and a stronger incentive is given to PMTCT services than VCT services. For example, health centers receive $4.50 per visit for providing PMTCT services but only $0.90 for each VCT test.39 Second, PMTCT, compared with ART, is naturally more responsive to incentives. The PMTCT used in this study represented the number of couples tested for HIV/AIDS. It involves every pregnant woman and probably several instances in a woman's lifetime. However, delivering ART has to be largely dependent on the number of diagnosed patients. Third, the most prominent effect of PBF might lie in the improvement of the quality rather than the quantity of services.11 However, our available data did not contain any measures of quality of services. Fourth, despite favorable results reported elsewhere,6 PBF may not be a sufficiently powerful intervention to be measurable in a small study10 including ART and VCT.
This study had several limitations. First, this study has a relative small sample size of 26 health centers, and possible low statistical power, given a wide variability among health centers in providing HIV/AIDS services and the number of covariates in the regression models. Examination of the effect of PBF on VCT and ART would have benefited from further examination with a larger sample size. Second, there are several measurement issues. The measurement of non-personnel expenditure on HIV/AIDS may have been imprecise. Because of data constraints, we could not directly collect the information on non-personnel expenditures specific for HIV/AIDS services. Instead, this expenditure was derived from the total revenue of the health center. For health centers that generated an operating surplus, we may have overestimated their expenses on HIV/AIDS, and thus underestimated their performance. Conversely, we might have underestimated the HIV/AIDS expenses for health centers with operating deficits and thus inflated their performance. Similarly, the imputed population for each catchment area was not precise to estimate the CBHI use rate. However, because most health centers were randomly selected, those measurements (e.g., non-personnel spending and population size) are less likely to present a systematic pattern. Therefore, our results are still valid. Third, some variables are omitted from the regression models. We included CBHI and PBF in modeling the service delivery and the efficiency. Additional factors, such as improved management, increased experience of staff, greater economies of scale, and better supply of drugs and test kits, probably affected the delivery of HIV/AIDS services in health centers. Because these factors may be correlated with CBHI and PBF, our results may overestimate the effectiveness of these two policies. However, for the efficiency estimation, we used a DEA model with variable returns to scale; thus, the scale effect has been incorporated. In addition, because our analysis examined the change in efficiency over time, rather than the level of efficiency, and many characteristics of health centers (e.g., ownership, economic situation where heath centers are located) did not change over time, possible biases from omitted variables were minimized.
ACKNOWLEDGMENTS
We thank Clare Hurley for editing the manuscript, Sabine Furere for assistance in organizing data collection, and Godelive Kashama and other interviewers for assistance in data collection.
Disclaimer: Views expressed in this article are those of the authors and do not necessarily reflect the views of the authors' institutions or the sponsors.
Footnotes
Financial support: This study was supported by a joint research agreement between the Joint United Nations Programme on HIV/AIDS and Brandeis University.
Disclosure: None of the authors has any financial interest in any product discussed in this article.
Authors' addresses: Wu Zeng, Schneider Institutes for Health Policy, Heller School, Brandeis University, Waltham, MA, and Futures Group International, Washington, DC, E-mail: wuzengcn@brandeis.edu. Donald S. Shepard, Schneider Institutes for Health Policy, Heller School, Brandeis University, Waltham, MA, E-mail: shepard@brandeis.edu. Angelique K. Rwiyereka, Global Health Issues and Solutions, Kigali, Rwanda, E-mail: Gasugi@gmail.com. Peter R. Amico, RTI International, Waltham, MA, E-mail: pamico@rti.org. Carlos Ávila-Figueroa, Abt Associates, Bethesda, MD, E-mail: carlos_avila@abtassoc.com.
References
- 1.UNAIDS, WHO . AIDS Epidemic Update. Geneva: UNAIDS, World Health Organization; 2009. [Google Scholar]
- 2.PlusNews . GLOBAL: Value for Money Central to Achieving Universal Access (11/3/10) 2010. http://www.plusnews.org/Report.aspx?ReportId=90612 Available at. Accessed November 3, 2012. [Google Scholar]
- 3.Rwanda Ministry of Health . Ministry of Health Annual Report 2008. 2009. http://www.moh.gov.rw/index.php?option=com_docman&task=doc_download&gid=116&Itemid=13 Available at. Accessed December 10, 2009. [Google Scholar]
- 4.Logie DE, Rowson M, Ndagije F. Innovations in Rwanda's health system: looking to the future. Lancet. 2008;372:256–261. doi: 10.1016/S0140-6736(08)60962-9. [DOI] [PubMed] [Google Scholar]
- 5.Soeters R, Habineza C, Peerenboom PB. Performance-based financing and changing the district health system: experience from Rwanda. Bull World Health Organ. 2006;84:884–889. [PMC free article] [PubMed] [Google Scholar]
- 6.Basinga P, Gertler PJ, Binagwaho A, Soucat AL, Sturdy J, Vermeersch CM. Effect on maternal and child health services in Rwanda of payment to primary health-care providers for performance: an impact evaluation. Lancet. 2011;377:1421–1428. doi: 10.1016/S0140-6736(11)60177-3. [DOI] [PubMed] [Google Scholar]
- 7.Shepard DS, Rwiyereka AK, Beaston-Blaakman A. Community-based health insurance in Rwanda: from case studies to national policy. Rwanda Med J. 2009;14:245–250. [Google Scholar]
- 8.Rwanda Ministry of Health . Community Based Health Insurance in Rwanda. 2008. http://www.cbhirwanda.org.rw/index.php?option=com_content&view=article&id=46:community-based-health-insurance-in-rwanda Available at. Accessed December 20, 2009. [Google Scholar]
- 9.Rwanda Ministry of Health . Integrated Household Living Conditions Survey 3 (EICV 3) Kigali: Ministry of Health, Rwanda; 2011. [Google Scholar]
- 10.Kalk A, Groos N, Karasi JC, Girrbach E. Health systems strengthening through insurance subsidies: the GFATM experience in Rwanda. Trop Med Int Health. 2010;15:94–97. doi: 10.1111/j.1365-3156.2009.02424.x. [DOI] [PubMed] [Google Scholar]
- 11.Rusa L, Ngirabega Jde D, Janssen W, Van Bastelaere S, Porignon D, Vandenbulcke W. Performance-based financing for better quality of services in Rwandan health centres: 3-year experience. Trop Med Int Health. 2009;14:830–837. doi: 10.1111/j.1365-3156.2009.02292.x. [DOI] [PubMed] [Google Scholar]
- 12.Nakakeeto ON, Umaranayake L. The global strategy to eliminate HIV infection in infants and young children: a seven-country assessment of costs and feasibility. AIDS. 2009;23:987–995. doi: 10.1097/qad.0b013e32832a17e9. [DOI] [PubMed] [Google Scholar]
- 13.Quentin W, Konig HH, Schmidt JO, Kalk A. Recurrent costs of HIV/AIDS-related health services in Rwanda: implications for financing. Trop Med Int Health. 2008;13:1245–1256. doi: 10.1111/j.1365-3156.2008.02142.x. [DOI] [PubMed] [Google Scholar]
- 14.Shepard DS, Zeng W, Amico P, Rwiyereka AK, Avila-Figueroa C. A controlled study of funding for human immunodeficiency virus/acquired immunodeficiency syndrome as resource capacity building in the health system in Rwanda. Am J Trop Med Hyg. 2012;86:902–907. doi: 10.4269/ajtmh.2012.11-0202. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Murray CJL, Evans DB. Health Systems Performance Assessment: Debates, Methods and Empiricism. Geneva: World Health Organization; 2003. [Google Scholar]
- 16.Eichler R, Levine R. Performance Incentives for Global Health: Potential and Pitfalls. East Peoria, IL: Versa Press; 2009. [Google Scholar]
- 17.Elliott P, Hawthorne G. Imputing missing repeated measures data: how should we proceed? Aust N Z J Psychiatry. 2005;39:575–582. doi: 10.1080/j.1440-1614.2005.01629.x. [DOI] [PubMed] [Google Scholar]
- 18.Charnes A, Cooper WW, Rhodes E. Measuring the efficiency of decision making units. Eur J Oper Res. 1978;2:429–444. [Google Scholar]
- 19.Banker RD, Natarajan R. Evaluating contextual variables affecting productivity using data envelopment analysis. Oper Res. 2008;56:48–58. [Google Scholar]
- 20.Hao S, Pegels CC. Evaluating relative efficiencies of veterans affairs medical centers using data envelopment, ratio, and multiple regression analysis. J Med Syst. 1994;18:55–67. doi: 10.1007/BF00999452. [DOI] [PubMed] [Google Scholar]
- 21.Zavras AI, Tsakos G, Economou C, Kyriopoulos J. Using DEA to evaluate efficiency and formulate policy within a Greek national primary health care network. Data envelopment analysis. J Med Syst. 2002;26:285–292. doi: 10.1023/a:1015860318972. [DOI] [PubMed] [Google Scholar]
- 22.Hollingsworth B, Dawson PJ, Maniadakis N. Efficiency measurement of health care: a review of non-parametric methods and applications. Health Care Manage Sci. 1999;2:161–172. doi: 10.1023/a:1019087828488. [DOI] [PubMed] [Google Scholar]
- 23.Chilingerian JA. Evaluating physician efficiency in hospitals: a multivariate analysis of best practices. Eur J Oper Res. 1995;80:548–574. [Google Scholar]
- 24.Coelli TJ, Rao DS, O'Donnell CJ, Battese GE. An Introduction to Efficiency and Production Analysis. New York: Springer Science and Business Medica, Inc; 2005. [Google Scholar]
- 25.Cooper WW, Seiford LM, Tone K. Data Envelopment Analysis: A Comprehensive Text with Models, Applications, References and DEA-Solver Software. New York: Springer Science + Business Media, LLC; 2006. [Google Scholar]
- 26.National Institute of Statistics of Rwanda . Demographic and Social Statistics. 2008. http://statistics.gov.rw/index.php?option=com_content&task=view&id=152&Itemid=201 Available at. Accessed December 10, 2009. [Google Scholar]
- 27.Marseille E, Dandona L, Marshall N, Gaist P, Bautista-Arredondo S, Rollins B, Bertozzi SM, Coovadia J, Saba J, Lioznov D, Du Plessis JA, Krupitsky E, Stanley N, Over M, Peryshkina A, Kumar SG, Muyingo S, Pitter C, Lundberg M, Kahn JG. HIV prevention costs and program scale: data from the PANCEA project in five low and middle-income countries. BMC Health Serv Res. 2007;7:108. doi: 10.1186/1472-6963-7-108. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Menzies NA, Berruti AA, Blandford JM. The determinants of HIV treatment costs in resource limited settings. PLoS ONE. 2012;7:e48726. doi: 10.1371/journal.pone.0048726. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Rosen S, Long L, Sanne I. The outcomes and outpatient costs of different models of antiretroviral treatment delivery in South Africa. Trop Med Int Health. 2008;13:1005–1015. doi: 10.1111/j.1365-3156.2008.02114.x. [DOI] [PubMed] [Google Scholar]
- 30.Renner A, Kirigia JM, Zere EA, Barry SP, Kirigia DG, Kamara C, Muthuri LH. Technical efficiency of peripheral health units in Pujehun district of Sierra Leone: a DEA application. BMC Health Serv Res. 2005;5:77. doi: 10.1186/1472-6963-5-77. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Marschall P, Flessa S. Efficiency of primary care in rural Burkina Faso. A two-stage DEA analysis. Health Econ Rev. 2011;1:5. doi: 10.1186/2191-1991-1-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Masiye F. Investigating health system performance: an application of data envelopment analysis to Zambian hospitals. BMC Health Serv Res. 2007;7:58. doi: 10.1186/1472-6963-7-58. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Akazili J, Adjuik M, Jehu-Appiah C, Zere E. Using data envelopment analysis to measure the extent of technical efficiency of public health centres in Ghana. BMC Int Health Hum Rights. 2008;8:11. doi: 10.1186/1472-698X-8-11. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Rwiyereka AK. Making Money Work for Mothers: A Quantitative and Qualitative Assessment of the Impact of Novel Health Financing Policies on Maternal Health Services in Rwanda. Waltham, MA: The Heller School for Social Policy and Management, Brandeis University; 2013. [Google Scholar]
- 35.Tabor SR. Community-Based Health Insurance and Social Protection Policy. 2005. http://siteresources.worldbank.org/SOCIALPROTECTION/Resources/0503.pdf Available at. Accessed December 10, 2009. [Google Scholar]
- 36.Zeng W, Shepard DS, Chilingerian J, Avila-Figueroa C. How much could we gain from improved efficiency? An examination of performance of national HIV/AIDS programs and its determinants in developing countries. BMC Health Serv Res. 2012;12:74. doi: 10.1186/1472-6963-12-74. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Preker AS, Carrin G. Health Financing for Poor People: Resource Mobilization and Risk Sharing. Washington, DC: World Bank; 2004. [Google Scholar]
- 38.Palmer N, Mueller DH, Gilson L, Mills A, Haines A. Health financing to promote access in low income settings: how much do we know? Lancet. 2004;364:1365–1370. doi: 10.1016/S0140-6736(04)17195-X. [DOI] [PubMed] [Google Scholar]
- 39.Rwanda Ministry of Health . Performance-Based Financing Guile for District Hospitals. 2009. http://www.rbfhealth.org/rbfhealth/system/files/080911_DHPBFGUIDE_draft_EN.pdf Available at. Accessed December 10, 2009. [Google Scholar]