Using quality improvement to accelerate highly active antiretroviral treatment coverage in South Africa

Abstract

Introduction

The authors report on a health systems strengthening intervention using quality improvement (QI) methods at the subdistrict level to accelerate highly active antiretroviral treatment (HAART) initiation in South Africa.

Methods

Using a phased scale-up design between August 2006 and November 2009, 14 primary healthcare clinics, one community health centre, one district hospital and one tertiary hospital in a subdistrict were recruited into a ‘learning network’ using QI methods to facilitate cross-facility learning/mentorship/support. Clinic teams consisting of nurses, counsellors, clerks and/or doctors set collective and individual performance targets, analysed their care systems using ‘real-time’ data feedback, and designed/implemented a set of simple changes to improve HIV testing and HAART initiation rates across the region.

Data analysis

Primary clinic data were used to measure HAART initiation rates (primary outcome) and HIV testing (secondary outcome). We analysed data variation/trends using an interrupted time series design. Logistic regression analysis was applied to examine trends in HAART initiation during the intervention phases.

Results

Clinics in the learning network increased HIV testing by 301.8% from 891/month (SD=94.2) to 3580/month (SD=327.7) (p<0.0001). Monthly HAART initiations increased by 185.5% from 179/month (SD=17.22) to 511/month (SD=44.93) (p<0.0001). During the pilot (phase I), the monthly rate of HAART initiations increased by 3.6 patients. In the prototype collaborative (phase II), there was no acceleration in the rate of increase (3.3/month, p=0.92). Significant acceleration was observed in the rate of increase during the QI scale up (phase III) (10.1/month, p<0.001). The proportion of estimated need for HAART met in the region increased from 35.8% to 72.4% at a time of rapid population growth.

Conclusion

A QI approach, using learning networks to teach simple data-driven methods for addressing system failures, with increased training and resource inputs, can assist districts to quickly reach universal coverage targets.

Introduction

South Africa (SA) is at the centre of the HIV/AIDS crisis, with more people with HIV infection than any other nation.¹ With over a million patients initiated on highly active antiretroviral treatment (HAART) between 2004 and the end of 2009, the South African government (SAG) has started more patients on HAART, at a faster rate, than any other country in the world.^2–4 Despite this achievement, less than 50% of patients who need treatment are currently accessing HAART in SA,² and in 2010 the SAG proposed that an additional 1.2 million people be initiated on HAART in the next 1–2 years.⁵ In addition, the SAG recently launched a major campaign to expand testing of its population and access to HIV care through ‘nurse initiation and management of patients on ART’ (NIMART), adding new pressures to the health system.⁶

In other low-income and middle-income nations heavily affected by the HIV epidemic, health systems need to provide for escalating numbers of patients who are HIV positive and require pre-antiretroviral treatment (ART) and chronic HAART services for the foreseeable future.^{4 7} While demand is expanding, resources, both financial and human, to meet these demands are not expanding at commensurate rates.⁸ Governments around the world will need to meet service demands primarily through improvements in system efficiency and more effective utilisation of existing resources.

We report on a health systems strengthening intervention that used quality improvement (QI) methods at the subdistrict level to accelerate HAART initiation for those requiring treatment.

Methods

Setting

The project was situated in the city of Johannesburg in Region F—one of seven city subdistricts. A 2001 census estimated 433 054 people lived in Region F⁹; in October 2008, unofficial population estimates were revised to over 800 000 people living in the region. The primary explanation for this major change was the large influx of immigrants into the region spread over several months prior to October, following political and economic instability in Zimbabwe.¹⁰ With high levels of population migration, Region F is particularly vulnerable to HIV/AIDS given the existence of single sex hostels, marginalised communities living in informal settlements, overcrowding, poverty and income inequality. In recent years, the region has become the centre of government attention for urban renewal and the national response to the HIV/AIDS crisis.^{11 12} To address this crisis, Region F relies upon 14 primary healthcare clinics (PHCs) (managed and staffed by the City of Johannesburg Health Department), one community health centre (CHC), one district hospital, and one tertiary hospital (managed by the provincial health department).

In 2004, when the SAG initially called for the rapid expansion of HAART for patients who are HIV positive throughout the country, Region F healthcare facilities and managers were challenged by the population's limited understanding of the disease and widespread misconceptions, leading to stigma, limited HIV testing, lack of triage and effective referral systems, and long waiting lists to start HAART due to overcrowding. At the beginning of the intervention only half of the regions' clinics were providing HIV testing. Scale up of additional testing sites started in February 2007 and by August 2007, all 14 PHCs were providing this service. The tertiary hospital, which was difficult to reach by some, was the only site initiating patients on HAART in 2004. The CHC started initiation in April 2005 and the district hospital started in August 2006.

Partnership

Given the high prevalence of HIV, health system leadership in Region F recognised that the current state of the healthcare system would not be sufficient to serve a rapid expansion of patients. A partnership was formed among the City of Johannesburg Health Department, Gauteng Department of Health, the Wits Reproductive Health and HIV Institute (WRHI) and the Institute for Healthcare Improvement (IHI) to improve the system to provide reliable, integrated HIV services to increasing numbers of patients in the region. The portion of the work that was led by WRHI was funded by USAID.

Study design

We designed a prospective, cross-sectional study analysing data variation and trends using an interrupted time series design. Logistic regression analysis was applied to analyse trends in HAART initiations.

Intervention

Timeline and resources

The HAART initiation programme was launched in April 2004, 5 months before the introduction of the QI initiatives. During phase I of the study, starting in October 2004, a small pilot was used to test QI approaches to improve patient flow in the main hospital where HAART initiation was first introduced to the region. In April 2005, the CHC started HAART initiation and joined the pilot. During phase II of the study, starting in August 2006, we initiated a regional health systems strengthening prototype intervention in seven PHCs plus the two initiation sites using a Breakthrough Series Collaborative design as described below.¹³ In phase III, starting in August 2007, a QI scale-up phase brought the remaining seven PHCs and third HAART initiation site into the ‘collaborative’, scaling up a tested list of changes developed in phase II. A post-intervention period started in June 2009 when IHI inputs were withdrawn altogether and WRHI's QI support to the clinics (facilitation of QI activities) was scaled back.

During phase II, the WRHI, an academic affiliate of Wits University Medical School, introduced a QI programme that used continuous QI methodologies (described below) to improve HIV care systems across all clinics in the region. IHI provided part-time QI design and training support throughout. In addition, from August 2006 onwards, a half-time improvement specialist from IHI provided ongoing coaching and mentoring to two WRHI project managers and six nurse quality mentors (QMs). QMs were responsible for facilitating QI activities in each clinic and transferring QI skills to local health system area managers/supervisors, facility managers and nurses through side-by-side mentoring and coaching. Four QMs were each responsible for three to four PHCs. One QM was responsible for mentoring the CHC and one was responsible for the tertiary hospital initiation site. These QMs were meant to provide QI support on a bi-weekly basis and to provide some supplemental clinical care support if needed. QMs' time on QI fluctuated between 0.1 and 0.5 full-time equivalent (FTE) throughout the project period (total range 0.6–3.0 FTE).

QI methodology

QMs taught clinic staff QI methods that were used to develop changes to improve identification, treatment and care of patients living with HIV. QMs introduced clinics to a set of HIV standards to help them assess performance according to those standards and prioritise areas for improvement. Each clinic formed a small multidisciplinary, facility-based QI team consisting of representatives from each type of staff in the clinic: nurses, data clerks, administrative staff, housekeepers, security, counsellors and a doctor where available. All facility-based QI teams joined together into a collaborative learning network modelled on the Breakthrough Series Collaborative design to enhance knowledge sharing.¹³ Every 4 months, QI teams from all participating facilities convened with regional and district managers at a Learning Session (workshop) to learn QI methods, set collective regional targets, review progress towards facility targets, and learn from each other about best practices to accelerate HIV testing and HAART initiation (figure 1). IHI and WRHI project staff facilitated sessions and included patient participants to share their perspectives for enhanced learning. Between Learning Sessions, each facility QI team was tasked to meet at least biweekly to identify ways to improve identification, testing and referral of patients who were HIV positive and needed to start HAART. QMs facilitated 1 h meetings when teams set clear targets for what they wanted to achieve, used root cause analysis to understand system barriers, used process mapping to understand patient flow and clinical pathways, and used data review and Pareto-chart analysis to identify opportunities for improvement that would yield the greatest impact with addition of the fewest outside resources. These QI teams used a systems change strategy based on the Model for Improvement and Plan-Do-Study-Act (PDSA) cycles to test process changes in multiple settings.¹⁴ As part of this process, QMs routinely collected monthly data from clinic staff and fed these data back to facility-based QI teams to track progress on targets and guide activities to improve care. A visual ‘dashboard’ of this system performance was created and reviewed monthly (figure 2).

Figure 1.

Breakthrough Series Collaborative design—timeline and sequence of activities and support for the Inner City of Johannesburg, Region F. CHC, community health centre; PHC, primary healthcare clinic.

Figure 2.

Sequential HIV processes of care and average data performance per month over last 6 months (June–November 2009); used by clinics to identify process steps in need of focused improvements. ARV, antiretroviral.

As system failures varied by clinic, QI teams generated multiple ideas for change across each step in the clinical care pathway to improve patient outcomes. QI teams adapted ideas from the clinical literature, and used the HIV standards/self-assessment tool to identify opportunities for improvement (see online supplementary file). Measures were developed to assess the impact of individual changes and to identify changes associated over time with documented improvement in processes and system performance.

Specific process improvement changes

A list of tested changes was developed to address common problems that clinics identified. Table 1 lists the changes for early identification and referral of patients from PHC sites, and rapid initiation of patients on HAART at treatment sites. Figure 3 lists the sequence of activities/changes made at the PHCs.

Table 1.

Changes made to improve the sequence of care steps for highly active antiretroviral treatment (HAART) initiation

System failures identified	Changes
Early identification and referral for HAART initiation
Limited number of patients testing, stigma and lack of information or misinformation about HIV/testing, lack of time for individual counselling, loss of patients after testing not returning for CD4 count results or wellness care, patient bumping around system leading to patient loss, initiation sites turning patients away for not having correct tests/results at appointment	Case finding and HIV testing All PHC sites provide HIV testing Triage system introduced to increase patient flow and opportunities for testing* HIV educational health talks provided in general waiting areas* Group counselling, couples counselling* Engagement with traditional healers to increase referrals of those with signs/symptoms of HIV to the clinics* Cross-referrals of patients from TB, family planning, STI clinics for HIV testing* New longitudinal patient registers added to track TB/HIV integration HIV testing campaigns outside clinic in local community; integrated with community health days and wellness checks HIV counsellors placed at each service site Integration of HIV services into all services and rooms—all staff effort Provider initiated testing and counselling* CD4 count testing Bundling HIV test with CD4 count test on same day, same location in rapid sequence* Start of wellness care at PHC sites for patients who are HIV positive not yet eligible for HAART to keep them in care Patient files marked with barcode signalling need for repeat CD4 testing* Patient collection of CD4 results Fast track queue for those collecting results* Results from lab faxed to clinic; decreased results turnaround time from lab to clinic* Client counselling on importance of knowing CD4 results* Improved documentation of patient contact information for follow-up* Communication to ensure increased patient confidentiality* Patient follow-up reminder calls to collect results Referral for HAART Standardized up-referral† forms provide clarity for patient and staff, include all test results and necessary next steps* Relationship building between PHC and initiation site staff to increase communication and minimise patient loss* Patient follow-up workers call patients to remind them of appointments Improved documentation of patients received from PHC site to identify missing patients*
HAART initiation and chronic care
Misinformation about treatment and adherence, long waiting times leading to patient loss, missed appointments and poor retention, overcrowding, long time from diagnosis to treatment, loss of patients with low CD4 counts	Health Information talks in waiting area on initiation, medication, side effects and adherence* Creation of referral forms and standard procedures* Patient flow analysis leading to changes in booking and better patient flow (eliminating unnecessary steps)* Fast tracking initiation of patients with low CD4 counts* 2 months' supply of medication is provided to patients with stable disease to decrease congestion, unnecessary wait time and free up staff time to initiate new patients* System of triage to improve patient flow* Patient tracing phone calls after missed appointments Pharmacy automated dispensing system Down-referral‡ of patients with stable disease on HAART back to their local PHCs to be managed by nurses

^*Denotes changes that require no additional resources (equipment, supplies, staff, etc.); can be implemented through better use of existing resources.

^†Up-referral defined as referring a patient to a higher level of care (ie, from PHC to secondary/tertiary initiation site for more specialised treatment).

^‡Down-referral defined as referring a patient to a lower level of care requiring less specialisation (ie, from initiation site to PHC for nurse management).

CHC, community health centre; HAART, highly active antiretroviral treatment; PHC, primary healthcare clinic; STI, sexually transmitted infection; TB, tuberculosis.

Figure 3.

Sequence of activities and changes leading to increased HIV testing from May 2006 through November 2009 in Inner City Johannesburg, Region F. Green line represents regional HIV testing target, recalculated in October 2008 with increase in estimated population. PITC, provider-initiated testing and counselling.

Concurrent with the QI intervention, WRHI provided supplemental staffing to provide clinical training/support. Doctors, pharmacists and patient follow-up workers rotated across the HAART initiation sites. Nurses, counsellors and data clerks rotated across a few high-volume PHC sites when high staff turnover rates and absenteeism were affecting patient care. The number fluctuated across the study time period. Lack of HIV clinical skills was identified as a barrier to improvements during causal analysis, therefore WRHI provided HIV clinical management training.

Measures

The number of patients who were HIV positive and needed HAART and who were initiated on HAART was the principal outcome measure for the project. Clinics also measured the key steps (processes) in the system leading to appropriate referral for treatment: the number testing for HIV; the number of CD4 counts completed per total number of patients who were HIV positive; the number of patients collecting CD4 count results per total counts drawn; and the number of patients eligible for HAART who were referred for HAART.

To accelerate the pace of reaching those in need of HAART, the SAG established the HIV National Strategic Plan (NSP) in 2007.¹⁵ The NSP set a target to provide access to treatment for 80% of those in need by 2011. Targets for HAART initiation were based on available census data and published antenatal clinic HIV prevalence rates for the region, and provided a useful aim for the overall project.^{9 16} At the start of the regional initiative (August 2006–September 2008), the estimated need for HAART in Region F was 6000 patients per year, or 500 per month, based on a population estimate of 433 054. In October 2008, this estimate was increased to 9146 patients per year, or 762 per month, based on revised population estimates (the revised population of Region F in 2008 was 845 888).¹⁷

Data collection and analysis

All data analysed were obtained from clinic-based HIV/tuberculosis registers collated monthly from weekly clinic summary sheets by clinic staff. WRHI project staff verified data validity using a sampling process that checked for accuracy and completeness of these data elements. Data for the monthly number of HAART initiations in the region were available between April 2004 and November 2009. This time period can be split into baseline/pre-intervention (April–September 2004), three QI intervention phases (phase I, pilot: October 2004–July 2006; phase II, prototype collaborative: August 2006–July 2007; and phase III, scale up: August 2007–May 2009), and post-intervention period (June–November 2009). Run chart and Shewhart control chart analysis was used to determine the effect of system changes on the processes and outcomes of HAART care as changes were made over time.^18–21 To determine whether the improvements in performance were significant, we analysed data variation and trends, using a time series analysis.^{19 22} All data were analysed in Stata (V.10, Stata Corp). Two-tailed, unequal variance t-test analysis was used to compare change in performance before and after the intervention if baseline data were available. To determine the relative effects of QI interventions and additional resources being introduced during the study period, we applied a logistic regression analysis to the monthly rate of HAART initiations over three intervention periods (phases I, II, III). We were unable to include baseline and post-intervention periods in this analysis due to lack of data points for these periods. We performed a regression analysis for the entire observation period. We included separate terms for each of the three intervention phases and then determined if the rate of increase in HAART initiation in phase III and II was different from the rate in phase I.

Sustainability

Monthly partnership meetings between Department of Health, City of Johannesburg, WRHI and IHI were established at the start of the project to build local leadership capacity to review progress, provide accountability and understand and support clinic QI activities. Local leaders included clinic supervisors and facility operations managers from all sites. Partners met to present study findings, identify barriers, and propose solutions to support the clinic staff driving improvements. In the second half of the project, a systematic effort was undertaken to build capacity for local leaders to sustain QI activities independently through handing over the facilitation of these progress reviews. HIV programme coordinators were mentored by WRHI QMs and managers during cooperative clinic visits on the use of health system data and QI methods described above (model for improvement, PDSA cycles, root cause analysis, etc.) to facilitate system improvements. Starting in February 2008, facility staff were charged with assuming control over the QI activities.

Results

Building regional QI capacity

During the intervention, more than 200 healthcare workers and regional health managers attended nine, 1-day QI learning sessions. At the beginning of the regional initiative, QMs facilitated biweekly meetings with each QI team at each clinic site, typically lasting 1–2 h. As conflicts in scheduling arose, meetings changed to a monthly schedule. In February 2008, City of Johannesburg leaders requested that QI meetings take place during monthly staff meetings. Frequency of meetings held varied greatly across the region, as staff meetings were often cancelled. Some QI teams continued to meet monthly and, as they gained experience, facility managers took over facilitation. Others did not meet unless prompted by WRHI project staff.

Case finding and voluntary HIV counselling and testing

The mean number of patients tested for HIV increased from 891/month (SD=94.2) in the 4 months prior to the regional initiative (May–August 2006), to 3580/month (SD=327.7) during the 6 months post-intervention (June–November 2009). This represents a significant change (p<0.0001). Changes introduced during the initial learning network meeting and soon thereafter were associated with an increase in patients tested for HIV (figure 3). A further increase was observed in the last phase of the project after the introduction of two key changes: making HIV testing available at every service point, and provider-initiated testing and counselling.

CD4 count testing

CD4 counts were routinely drawn on the same day for most patients newly diagnosed with HIV. After January 2008, healthy patients who were HIV positive and had a CD4>200 were routinely offered CD4 testing at every general clinic encounter, maximising opportunities for chronic disease management and early detection of failing immunity. Changes developed to minimise loss of patients after CD4 testing did not yield a statistically significant increase in the mean percentage of patients returning for test results per month (41.9% (n=397 of 946) (SD=0.08) 4 months prior to the start of targeted changes, to 51.7% (n=890 of 1719) (SD=0.109) during the 6 months post intervention (p=0.1363)).

Expansion of HAART initiation capacity through decentralisation

From the outset of the project, increased patient referrals overwhelmed the sole HAART initiation site in the district. In addition, the location of the only initiation point presented an access barrier for many patients who lived some distance away from the tertiary care hospital. In response, the Region F health authorities opened an initiation site at the CHC to initiate patients on HAART in April 2005, the district hospital started HAART services in July 2007, and one of the PHCs followed in November 2009. This decreased the proportion of average monthly HAART initiations in the region at the tertiary care hospital from 54.8% in the 6 months prior to the regional initiative to 17.2% during the 6 months post intervention.

Reaching HAART initiation targets

Six months prior to the beginning of the regional initiative, the region was initiating 35.8% of the estimated monthly need for HAART initiations (n=179 HAART initiations per month of 500 patients per month who were in need of HAART based on original population estimates). The SAG target (80% of need or 400 HAART initiations per month of 500 patients in need) was surpassed 20 months into the regional initiative (April 2008). As described earlier, population estimates in the region were revised in October 2008, and the denominator (number in need of HAART) was revised from 500 patients per month to 762 patients per month. HAART coverage calculated based on this figure fell from 89.6% (n=448 per month of 500 patients in need) to 58.8% (n=448 per month of 762 patients in need). Further efforts to find and refer eligible patients in the region resulted in a further increase in coverage to 72.4% (n=552 HAART initiations of 762 patients in need) at the project's conclusion in November 2009. Over the 3-year regional initiative period, average monthly initiations increased from 179 per month (SD=17.22) to 511 per month (SD=44.93). This represents a significant change (p<0.0001).

Control chart analysis (figure 4) shows significant increase in baseline/early intervention rates of HAART initiation between February 2006 and August 2007 (from 125/month to 201/month). The changes occurring during this time include the launch of the collaborative of seven PHCs (August 2006) and scale up of HIV testing (February–August 2007). After August 2007, there is a rapid increase in initiation rates, which stabilises at a much higher rate of 499/month after October 2008.

Figure 4.

Control chart on monthly highly active antiretroviral treatment (HAART) initiations for August 2004–November 2009 demonstrating the impact of additional clinics providing testing. Quality improvement (QI) intervention both contributed to the increased average initiation and improved performance across the system. The scale up of HIV testing led to a moderate limited increase in initiation, and was a prerequisite for the large increase seen in 2008 and 2009, accelerated by the expansion of QI efforts; 1=start of phase I: pilot to improve flow at sole antiretroviral (ARV) site; 2=community health centre (CHC) joins pilot; 3=seven primary healthcare clinics (PHCs) start HIV testing; 4=start of phase II: prototype regional health systems strengthening intervention, first learning session (seven PHC HIV testing sites, two HAART initiation sites); third site starts HAART initiation; 5=scale up of HIV testing sites begins (three additional sites); 6=start up of phase III: scale up of regional collaborative, Learning Session; scale up of HIV testing sites ends, all 14 PHCs testing and part of intervention; down referral starts; 7=multiple, constant changes to processes leading to sustained new level of performance. LCL, lower confidence limit; UCL, upper confidence limit.

Regression analysis of the three QI intervention phases of the project showed that the rate of increase of HAART initiation during phase II, the prototype PHC learning collaborative was no different from that in the preceding phase (rate of additional patients initiated on HAART during phase I=3.6/month, phase II=3.3/month). We did observe a major change in the rate of increase of HAART initiations during phase III, the scale up of the QI intervention (an additional 10.1/month, p<0.001). The change in the post-intervention period was not significant, but trended towards an ongoing increase in rate over the short period of observation. This indicates a major acceleration of HAART initiations that occurred after the launch of the scale up of the QI intervention, building on the groundwork of previous QI innovations and scale up of HIV testing in clinics completed by August 2007.

Referral of HAART patients back to PHC sites from the HAART initiation sites

By June 2009, HAART initiation sites in Region F were returning 58.6% (n=280 of 478) of all new HAART initiations to the local clinics each month, opening up capacity at initiation sites. Progress to increase this number of patients referred back to the local clinics for chronic HAART care has been slow. Barriers (eg, reliance on external staff with minimal initial integration of chronic management into local structures, restrictive down-referral guidelines, and staff reluctance to down refer established patients to unknown local clinic staff) are being addressed. Two months after referral back to primary care clinics, 91% (n=2754 of 3025) of patients who had been down referred remained on regular HAART treatment.

Discussion

We conclude that it is possible to significantly increase access to HIV testing and HAART within a defined geographical area through the use of QI methods to identify and spread successful, locally tested changes. We also conclude that networking all the clinics of a health system referral unit into a natural improvement ‘unit’ rapidly increases integrated care for that population.

QI methods delivering locally tested changes can augment the capacity of a regional multi-clinic system to refer, initiate and care for patients needing HAART. Such methods also support current efforts to further decentralise care to the primary health centre level through nurse-driven management of patients receiving chronic HAART care.

Organisations have successfully used QI methods to deliver healthcare interventions in developed country settings,^{23 24} but until recently, use of these methods has been limited in resource-constrained settings. Recent literature demonstrates improvements across a broad array of clinical areas.^25–29 The most commonly reported improvements have been in maternal and child health,^29–31 with several focused on improvements in HIV care.^32–35 The Healthcare Improvement Project, funded by the US Agency for International Development, recently published an analysis that demonstrated value in using the breakthrough series collaborative design in 12 countries to address a wide variety of clinical and health systems problems.³⁶

This initiative succeeded at delivering several key changes across the HIV care spectrum. While it is difficult to know exactly what aspect of the QI intervention led to this success, we believe that improvements are likely attributable to increased staff involvement in solving problems and testing solutions, associated with increased use of local data to set targets and monitor progress, improved teamwork, and enhanced staff accountability for care of the HIV-positive population. The formation of a network of clinics that gathered together across several workshops provided an opportunity to link all clinics together, and accelerated the spread of awareness of best practices across clinic sites, including ideas about referring patients with stable disease from tertiary and secondary sites back to the primary health clinic level for chronic HIV care. This resulted in a major increase in care capacity and accelerated efforts to achieve universal coverage in the region. The initiative focused on a data-driven process that identified clear targets, based on population need, for each step of the HAART programme. Calculating targets based on available population data created a sense of urgency and local accountability for the patients under their care. An increase in the initiation targets that followed newly available population statistics led to renewed efforts and improved performance.

The main challenges to successful execution of the QI programme and long-term sustainability are staff turnover, resulting in irregularity of clinic improvement meetings; fluctuations in leadership support for the initiative and local accountability for improvement activities; reliance on intermittent external clinical and QI resources; lack of stipends for counsellors, leading to staff shortages; and, at times, inconsistent monitoring and feedback of progress to local staff.

A major limitation of this project is the lack of a comparison group. Despite numerous attempts to acquire consistent, high-quality data from comparable regions within the City of Johannesburg, these data were not available during the study period in sufficient quantity and reliability to make adequate comparisons.

A further limitation was the quality of locally collected facility data, which was the principal source of information for the primary outcome. To identify issues of data quality, data audit teams sampled facility register data to verify the completeness and accuracy of reported data. No significant deviations from protocol were noted in the reported data. In addition, as baseline data for some process measures were not available prior to the intervention, early intervention data from the beginning of the change period when improvement activities were already in progress were used to compare with post-intervention data; therefore, the amount of true improvement may actually be under-represented by the data we have presented here. While these data limitations are often intrinsic to large-scale improvement activities,³⁷ the size of the effects observed in HIV testing and HAART initiation, the timing of these effects, and the strength of the regression analysis can be used to support the existence of a plausible cause–effect relationship between the interventions and HAART referral and initiation.³⁸

Several potential confounders exist that could affect interpretation of the effects the quality intervention had on the primary outcome. First, during the intervention the region was simultaneously increasing the number of sites doing HIV testing and HAART initiation. However, we believe we showed that additional sites alone without efficient systems in place led to only modest increases in initiation rates. A major acceleration of initiation rates was only observed after the QI interventions were scaled up. Strong evidence for the contribution of scaled-up QI interventions comes from the time series and regression analyses. The control chart analysis showed a small increase in initiation rate during the start of the QI work in the PHCs and the HIV testing scale up. The regression analysis showed no significant change in rate during initial deployment of QI in the PHCs but confirmed a significant acceleration of the rate of HAART initiation when this intervention was scaled up to all clinics. Second, this study took place during a dynamic time of increased attention to and shifts in HIV/AIDS policy nationally which were likely to have positively influenced testing and initiation. However, the major identified barriers to HIV initiation at the time in this region were specific failures in care processes at all sites that preceded initiation of HAART. We believe that the increase in the number of clinics providing HIV testing before August 2007 was a crucial prerequisite for the improvement in initiation rates seen in 2008 and 2009, which was accelerated by the expansion of QI efforts. The addition of HIV testing services to clinics and QI interventions contributed to the increased average initiation and improved performance across the system.

While initial results are promising, it is unknown whether these results can be sustained by the health service, and can indeed be built upon, and will depend on the success of an intense effort (currently underway) to engage health managers to lead this process and to transfer QI skills and personnel to local health department structures. Further study will be needed to further separate the effects of the various components of the quality intervention (standards, training and improvement). Additionally, further investigation will be needed to evaluate the cost effectiveness of quality interventions. Such efforts are already underway in South Africa and elsewhere, to increase access to HAART and to prevent mother-to-child HIV transmission.^{35 39} Our results support the notion that QI methods can be used as a mechanism to improve reliable delivery of healthcare interventions in resource-limited countries.