A review of HIV/AIDS system-level interventions

Abstract

The escalating HIV/AIDS epidemic worldwide demands that on-going prevention efforts be strengthened, disseminated, and scaled-up. System-level interventions refer to programs aiming to improve the functioning of an agency as well as the delivery of its services to the community. System-level interventions are a promising approach to HIV/AIDS prevention because they focus on (a) improving the agency’s ability to adopt evidence-based HIV prevention and care programs; (b) develop and establish policies and procedures that maximize the sustainability of on-going prevention and care efforts; and (c) improve decision-making processes such as incorporating the needs of communities into their tailored services. We reviewed studies focusing on system-level interventions by searching multiple electronic abstracting indices, including PsycInfo, PubMed, and ProQuest. Twenty-three studies out of 624 peer-reviewed studies (published from January 1985 to February 2007) met study criteria. Most of the studies focused on strengthening agency infrastructure, while other studies included collaborative partnerships and technical assistance programs. Our findings suggest that system-level interventions are promising in strengthening HIV/AIDS prevention and treatment efforts. Based on our findings, we propose recommendations for future work in developing and evaluating system-level interventions.

Introduction

Researchers’ understanding of how HIV/AIDS is transmitted, prevented, and managed has increased substantially over the past twenty years. Researchers and practitioners have developed clinical and behavioral strategies that seek to decrease HIV transmission and enhance HIV-related quality of life. Multiple literature reviews have found solid evidence examplifying effective prevention strategies, including community needle exchange programs (Bastos & Strathdee, 2000), community education and advocacy (Janz, Zimmerman, Wren, Israel, Freudenberg, & Carter, 1996), linkage to antiretroviral therapy to decrease mother-to-child transmission (Volmink, Siegfried, van der Merwe, & Brocklehurt, 2007), distribution and use of the female condom (Hoffman, Exner, Leu, Ehrhardt, & Stein, 2003), and behavioral interventions to modify sex risk behaviors (Johnson, Hedges & Díaz, 2007; Lyles, Kay, Crepaz, Herbst, Passin, Kim et al., 2007). These comprehensive literature reviews have evidenced that prevention is effective, yet these efforts are sometimes labor intensive and slow. To strengthen on-going prevention efforts through dissemination and scale-up, we must ensure that the capacity and resources of the agencies carrying out these efforts are maximized.

Multiple intervention levels for HIV prevention and care

A social system refers to the network of institutional structures providing standardized and quality-assured services to large numbers of people and communities. These institutional networks may include federal agencies (e.g., CDC), local government agencies (e.g., city health departments), non-government organizations (NGOs) (e.g., AIDS service organizations [ASOs] and community-based organizations [CBOs]), and private health organizations (e.g., hospitals and healthcare providers). Delivery of HIV prevention through these institutions has been an efficient rollout mechanism because they may (a) reach and affect large numbers of people efficiently; (b) create and establish policies and procedures that maximize the diffusion of interventions; (c) increase program sustainability; and (d) incorporate the needs of specific communities into their tailored services. Nonetheless, we know very little in the HIV literature of how strengthening these social structures, individually and collectively, may ameliorate the provision of HIV prevention programs. System-level interventions are a promising approach to HIV/AIDS prevention because they focus on (a) improving the agency’s ability to adopt evidence-based HIV prevention and care programs; (b) develop and establish policies and procedures that maximize the sustainability of on-going prevention and care efforts; and (c) improve decision-making processes such as incorporating the needs of communities into their tailored services.

While most psychosocial interventions have focused on delivering individual and community programs, structural interventions have received little attention until recently (Blankenship, Bray, & Merson, 2000). We define system-level interventions as a set of strategies used to modify an agency’s culture, its internal process to preserve and maximize resources, and the efforts placed in strengthening external ties with other institutions for HIV prevention and care. An underlying assumption of this approach is that an agency’s capacity to deliver strong HIV prevention messages across different levels is tied intricately to the stability and effectiveness of its internal processes. Consequently, a system-level intervention focuses on strategies that promote institutional growth, capacity, and sustainability rather than on the mechanisms used to disseminate a program into the community.

The distinction between system-level strategies and dissemination mechanisms is vital in understanding the impact of system-level interventions. A system-level intervention may have as one of its outcomes the successful scale-up and dissemination of a program, yet it may not be the only outcomes of interest. For example, a CBO conducts a system-level intervention when it seeks to strengthen its staff skills through a technical assistance program or when it restructures its organizational hierarchy by including different community stakeholders in its Executive Board to increase diversity in its decision-making process.

A variety of theoretical frameworks and outcomes may inform system-level interventions. To this end, our goal is to review HIV/AIDS system-level interventions published in scientific journals from January 1, 1985 to February 1, 2007. In summarizing the findings by intervention strategy (i.e., technical assistance, infrastructure development, and inter-agency partnerships), we hope to illustrate different approaches and identify their strengths and weaknesses to inform future studies. We conclude with a summary discussion of intervention efforts demonstrating potential success or needing further replication and propose recommendations for future work

Theoretical Frameworks Informing System-Level Interventions

Theoretical frameworks should inform system-level interventions to facilitate program rationale and increase transferability of lessons learned when successful. Various theoretical approaches have been proposed to guide system-level intervention development, but their use has been limited in the HIV literature. We add to the basic assumption that, at a minimum, three components take place in creating and delivering services (Alter & Hage, 1993). These three components are: (1) inputs (internal and external resources that an agency can access), (2) process (an agency’s culture and its decision-making procedures), and (3) outputs (an agency’s aptitude to deliver services). We present an overview of common theoretical frameworks used to intervene in these three steps.

Stage Theory of Organizational Change (STOC)

STOC proposes a staged framework through which organizations incorporate innovations. Similar to the Transtheoretical Model (Prochaska et al., 2002), an organization’s ability to move through the stages towards adoption, implementation, and maintenance of an innovation depends on the institution’s ability to overcome external and internal barriers within each stage. Beyer and Trice (1978) propose seven stages: (1) sensing of unsatisfied demands on the system, (2) search for possible responses, (3) evaluation of alternatives, (4) decision to adopt a course of action, (5) initiation of action within the system, (6) implementation of the change, and (7) institutionalization of the change. Goodman and Steckler (1989) have argued for an eighth stage, renewal, to account for the program’s ability to evolve as demands change.

Organizational Development Theory (ODT)

Rooted in management theory and influenced by social ecology models, ODT posits that environmental influences (e.g., an organization’s norms and values) can shape organizational performance by influencing the individual and collective attitudes and behaviors of an organization’s staff (Steckler, Goodman, & Kegler, 2002). Three inter-related constructs inform interventions based on this theory: (a) organizational climate, (b) organizational culture, and (c) organizational capacity. Organizational climate can be defined as the institution’s personality and is caused and continuously shaped by the interaction of members’ collective beliefs, perceptions, and attitudes (Hoy & Miskell, 1987). Organizational culture refers to the underlying assumptions, values and norms that grow and persist as an institution evolves (Bolman & Deal, 2003). Finally, organizational capacity is characterized by an institution’s ability to create deliverables, to maintain the organization in operation, to respond to and receive support from the social and political demands in the environment, to adapt to changes within and outside of the institution, and to synchronize these competing demands appropriately (Katz & Kahn, 1978; Dill, 1994).

Interorganizational Relations Theory (IOR)

IOR focuses on how multiple organizations work together to maximize deliverables within their communities (Steckler et al., 2002). IOR posits that partnerships across organizations occur in order to tackle comprehensively the social, political, and economic demands affecting their communities. Interorganizational linkages facilitate access to new information, skills and resources, while sharing costs and responsibilities. Conversely, interorganizational linkages may delay action steps due to slow consensus development, may cause conflict if a policy or position taken by the whole is incongruent with the goals or mission of one or more of the organizations, and may divert much needed organizational resources to the collective mission of the coalition.

System-Level Intervention Strategies

Based on these frameworks, several strategies may be proposed as system-level intervention activities. We catalogue these strategies into three domains: (a) technical assistance, (b) infrastructure development, and (c) inter-agency partnerships and coalition building.

Technical assistance

System-level interventions may use technical assistance strategies to improve the delivery of services by intervening on an agency’s input component. Technical assistance may include agency staff training in the program planning process, consultation with experts in program planning and implementation, or external support in monitoring and evaluation procedures. Within HIV/AIDS, researchers and practitioners have identified evidence-based prevention strategies and programs (National Institute of Mental Health Multisite HIV Prevention Trial Group, 1998), efforts to prepare system-wide dissemination of these efforts have relied on STOC and ODT strategies. Examples of STOC and ODT strategies include the training in conducting a problem diagnosis like the one carried out in HIV Prevention Community Planning Groups (CPGs; Johnson-Masotti et al., 2000). Similarly, another example is the provision of technical assistance to agencies as they select, translate, and implement a Diffusion of Effective Behavioral Intervention (DEBI) to address community needs.

Infrastructure development

System-level interventions may seek to build an institution’s capacity by increasing an agency’s resource availability and improving its inputs. However, infrastructure development may also focus on an agency’s process component by reorganizing its internal functioning. Agency reorganization may include a shift in the agency’s mission and goals, an increase in fund-raising activities to increase personnel retention as well as resources for new and ongoing services, and/or the reorganization of an agency’s leadership and decision-making process. Within HIV/AIDS, an example of a system-level strategy that may be informed by STOC and ODT would be the development and refinement of a clinic’s goals and objectives for a high-risk group, and the monitoring of quality of care standards to ensure optimal treatment of HIV-positive patients. Another strategy might include the rearranging of an ASO’s organizational structure to diversify staff tasks, increasing volunteer manpower and ability to secure funding, or the training of staff to strengthen their ability to translate evidence-based interventions into community-relevant HIV prevention programs.

External Partnerships

System-level HIV/AIDS strategies informed by IOR rely on the development of new or existing collaborations across a social system as a way of strengthening an agency’s outputs. These strategies can include partnerships of a particular kind of institution (e.g., a coalition formed exclusively by ASOs in a defined region) or a variety of stakeholders from various kinds of institutions seeking to minimize duplicate efforts in service provision across agencies (e.g., representatives from ASOs, CBOs, religious leaders, policy-makers, hospital managers). Potential activities include the development of coalition of ASOs and CBOs to increase outreach and testing efforts in their communities, the formation of a service bundle to provide greater comprehensive care to high-risk populations such as injection drug users, and the expanded access to resources and lessons learned, locally and at a distance, through electronic networks such as listservs and newsgroups.

Markers and Outcomes of System-Level Interventions

Markers of “success” can vary according to the strategies proposed as part of a system-level intervention and can range across different levels of analysis (e.g., individual, agency, and city/region) and units of analysis (e.g., patient, care provider, geospatial unit). We present briefly an overview of potential outcome indicators for system-level interventions and categorize them as (a) institutional-based outcomes and (b) HIV-related markers.

Institutional-based outcomes

System-level evaluation plans include an agency’s internal (“inputs”) markers, external (“outputs”) markers, or both, to monitor change in organizational efficiency and productivity. A capacity building initiative may focus, for example, on improving access to care by optimizing the organizational efficiency of a hospital. Internal agency markers include indicators such as changes in team composition and responsibilities within a medical team; creation and buy-in of group rules and values among staff; ensuring transparency in agency decision-making procedures by recording the number of staff meetings and the quality of feedback from clients and personnel; developing systems to monitor quality of care; and increasing agency funding and revenue.

Similarly, system-level programs may focus on improving organizational productivity. DEBIs, for example, seek to maximize the reach and impact of prevention programs across ASOs, CBOs, and health departments by identifying evidence-based behavioral strategies that are transferable to different communities. External agency outputs may include indicators such as the number of people reached within a catchment area; number of cross-agency collaborations; the quality of services, workshops and trainings offered; the proportion of evidence-based programs and services successfully meeting the needs of the community; and, the number of counseling and outreach services sponsored by the agency, among others.

As part of a system-level intervention, changes can occur at the individual level of analysis. System-level evaluation plans may include provider-specific measures to assess changes in provider attitudes and behaviors. A technical assistance program to facilitate the scale-up of voluntary counseling and testing (VCT) in a health clinic, for example, may include intervention components seeking to increase providers’ knowledge and attitudes regarding HIV counseling and testing, maximize their perceived job satisfaction, their perceived comfort towards service delivery, and enhance their skills to deliver counseling to clients and provide social support, among others. Similarly, individual-level changes can be evaluated among clients receiving services before and after a system-level intervention. Increases in the number of services offered in health clinic, for example, may increase clients’ intention to seek medical care, facilitate centralization of multiple health services within a single setting to optimize adherence, offer a greater number of opportunities for health education, and improve the quality of life of the patient and their social support networks.

HIV-related markers

System-level interventions can modify the epidemiologic characteristics of a disease at the regional and population levels. An agency strengthened by a systems-level initiative such as its capacity building to disseminate a DEBI may implement a primary prevention intervention that may decrease the incidence of HIV infection across a region. Similarly, restructuring a hospital’s initiative to increase the time spent in quality patient-provider interactions may increase the access and adherence to medical care that may result in the decrease of a county’s AIDS-related mortality rate and improve the quality of life of people living with HIV/AIDS.

Measurement of population health markers such as HIV/AIDS incidence and prevalence may be important to test the reach of a system-level intervention at a population-level. Nonetheless, these estimates may change slowly, additional epidemiologic markers such as quality of life indicators (QALYs/DALYs) and adherence rates should be included into system-level evaluations.

Method

Literature Search

We identified an inclusive list of search terms for studies that would be added to an electronic database for cataloguing: community, network, social, system, structural, intervention, program(me), and HIV/AIDS. To maximize the search, each term was assessed individually (i.e., structural & HIV) or in conjunction with other terms (i.e., structural intervention & AIDS) using Boolean expressions.

We approached the review process from two directions: (1) an inductive systematic collection of all potentially relevant articles found through computerized searches and (2) a deductive approach by reviewing the references of identified publications. The literature search included iterative searches of MEDLINE, Pub Med, PsycInfo, and The Cochrane Library. In addition, retrieved articles’ bibliographies were examined for any additional relevant intervention studies. To maximize the search, the research team placed no initial restrictions on publication year, population, or explicit mention of psychosocial factors associated with HIV prevention. Any publication written in English and related to HIV prevention, related HIV-risk behaviors, or potential predictors of these behaviors in the social sciences, public health, medicine, social work and human organization fields were eligible for inclusion.

Eligibility Criteria for Literature Search

We had a three-step filtering of the identified citations. First, we identified 4,129 citations published in over 100 journals between January 1, 1985 and February 1, 2007 using the generated list of keywords. From the citation abstracts, we excluded 3,339 studies because they were not written in English, solely detailed medical procedures, focused solely on HIV/AIDS prevalence or incidence surveillance statistics, and/or presented the results of predictor studies. An additional 166 entries were excluded because they were duplicates.

Second, we refined the search by reading each citation’s abstract and kept citations that offered any description of a potential system-level effect (n= 624 citations). Where relevant, we also included studies describing the implementation of an intervention. To be conservative, we included a citation if the abstract did not include sufficient information to determine if there a systems-level effect (i.e., did not include an agency-level or provider-level outcome). After filtering all non-relevant citations, we identified 104 potential citations for review. We were unable to find the full-text of sixteen of these studies.

Finally, we developed additional criteria for inclusion in the review. We included in this review interventions focusing on reducing HIV risk through technical assistance, scale-up and roll-out programs, or through optimizing agency capacity of hospitals, community-based organizations (CBOs) and AIDS service organizations (ASOs), and government institutions. To be considered for review, studies had to have been published in the scientific literature between January 1985 and February 2007. Only studies between 1995 and 2007 met review criteria. This period also reflects the onset of ARV scale-up and rollout. Following Exner et al. (1997), we focused on studies designed specifically to address HIV/AIDS and with available process and outcome evaluation data. We did not restrict our review to a subset of intervention outcomes to capture the scope and breadth of eligible systems-level intervention outcomes.

We acknowledge that many systems-level interventions may be missing from our review because they are implemented by agencies (i.e., UNAIDS, FHI, WHO, World Bank, Global Fund) that disseminate their program findings through non-peer-reviewed journals (for a recent review of these initiatives, see Druce et al., 2006). Furthermore, while the reach and importance of health policies as system-level interventions should not be undermined, we chose to exclude policy interventions from our review because they require different analytic considerations (Cason et al., 2002; Lazzarini & Klitzman, 2002; Weimer & Vining, 1999). Policy analysis requires an analytic approach that takes into account the political process, the social action movements lobbying for changes in government regulations. While influencing system-level initiatives, policy interventions are beyond (more “upstream”) the scope of system-level interventions.

Results

Out of 624 citations, we reviewed 26 reports meeting study criteria (i.e., interventions focusing on reducing HIV risk through technical assistance , infrastructure development, scale-up and roll-out programs, or through optimizing partnerships between hospitals, community-based organizations (CBOs) and AIDS service organizations (ASOs), and government institutions). While most studies reported more than one intervention strategy, the most common strategies were infrastructure development (17 studies; 74%), followed by collaborative partnerships across agencies to provide bundled services (12 studies; 53%), and technical assistance for agencies trying to develop and implement new interventions (9 studies; 43%). We found, however, that the choice of a particular strategy varied by location (see Table 1). Among studies based in the United States, infrastructure development was the strategy cited most often, followed by technical assistance programs and external partnerships. Among international studies, infrastructure development was the strategy cited most often, followed by external partnerships and technical assistance programs.

Table 1.

Overview of strategies used across the reviewed studies by institutional context and location

	Non-Government (Organization (n=8)	Health Care Institution (n=10)	Government Agency (n=12)	Total (n=23)
Technical assistance				9 (40%)
U.S.	4	0	3	7 (78%)
International	1	1	0	2 (22%)
Infrastructure development				17 (74%)
U.S.	4	3	5	12 (71%)
International	1	3	1	5 (29%)
External Partnerships				12 (53%)
U.S.	1	3	5	9 (75%)
International	0	1	2	3 (25%)

Note. Strategies are not mutually exclusive: (a) Technical Assistance + Infrastructure Development were found in 6 occasions; (b) Technical Assistance + External Partnerships combinations were not found; (c) Infrastructure Development + External Partnerships combinations were found in 5 occasions; and (d) All three strategies were found on 2 occasions.

Four research designs repeatedly emerged in the studies reviewed: a pretest-posttest design with one group (11 studies; 48%) or two groups (2 studies; 9%), a one-shot case study (5 studies; 22%), a randomized control trial (4 studies; 17%), and a one-group time series design (1 study; 4%). Below, we summarize our findings by cataloguing studies across intervention strategies: (a) technical assistance, (b) infrastructure development, or (c) external partnerships. Within each strategy, we identify common intervention components and comment on selection of outcomes to test intervention effects.

Technical assistance

Nine studies reported technical assistance as a component of their intervention (see Table 2). Over 75% of all technical assistance strategies included a component aiming to improve skills in program planning (7 studies) and intervention implementation (7 studies), respectively. Two studies included technical assistance in community outreach and service delivery. Technical assistance components were found more often in the U.S. than in international settings, particularly within NGO and government institutions.

Table 2.

Summary Review Table of System-Level Interventions

Author (Year)	Location (Agency Type)	Design	Strategy	Intervention Components	Primary Outcome(s)	Results
Ball (2006)	Zambia (GOV)	One group pre/post test	Infrastructure Development External Partnerships	Mainstreaming of AIDS education by training employees of the Rural Water Supply Project (RWS). RWS employees participated in the following activities: (a) Peer educator-led health education programs; (b) social marketing of free condoms; (c) drafting of a workplace policy regarding protections if HIV-positive; and (d) referral of employees to VCT centers. Community members from rural villages participated in (a) Systematic inclusion of AIDS prevention and education in hygiene, sanitation, and community meetings; (b) dramatizations on STIs, stigma and VCT; (c) AIDS education presentations.	HIV infection. Number of days lost to illness. Increased water coverage rate. Decrease water-borne illnesses.	Effects of the intervention included: (a) placing water pumps closer to villages to avoid the threat of assault and rape of women who go to the water pump; (b) quicker access to water for sick individuals; (c) training of women as pump menders. No data on epidemiologic markers such as prevalence of HIV and other illnesses were reported.
Blank et al. (2005)	New York, USA (HCO)	One-group pre/post test	External Partnerships	Provision of service-bundles for MSM in non-traditional health venues such as bars and clubs. Services included screening of general services (hypertension, diabetes, cholesterol, depression, and substance abuse), diagnostic tests (HIV, syphilis, gonorrhea), vaccines (Hepatitis A & B), and referral services (smoking cessation, and methamphetamine counseling).	Use of any service. Participation rate in venues.	73% of all MSM attending a sponsored event did not access any services. Among those who received one or more services, participation rates ranged from 12% to 44%.
Bluespruce et al. (2001)	Washington, USA (HCO)	One group pre/post test	Infrastructure Development	Primary care providers (physicians, physician assistants, nurse practitioners, registered nurses, and social workers) working in two primary care clinics received an intervention to improve their assessment and counseling of patients about HIV risk. The intervention was delivered across four trainings to increase knowledge and skills through role-plays and case studies. In addition, questions regarding HIV/STD risk were included into patient questionnaires before physical examinations.	Provider attitudes, knowledge, self-efficacy to assess HIV risk, and perceived support and barriers to HIV assessment and care.	Providers’ self-efficacy to provide advice to gay men and single heterosexuals increased from baseline to follow-up. At follow-up, physicians felt greater confidence in asking patients about specific sexual practices. These effects were not found for non-physician personnel. Providers reported increases in their access to trainings to help talk to patients about HIV risk and prevention, to use education materials, and to link patients with medical social workers.
Harshbarger et al. (2006)	U.S. (GOV)	One-shot case study	Technical Assistance	Through the CDC’s DEBI program, agencies were trained and offered technical assistance in the VOICES/VOICES intervention.	Number of agencies adopting the intervention Modifications to the intervention	Staff from 229 agencies was trained in the intervention. Of the 162 agencies interviewed at 3-months, 62 had not implemented the program. 61 agencies modified the intervention. Modifications included forming groups with a different number of clients than those recommending (64%), expanding the length of the intervention (33%), adding materials to the program (36%), and integrating the full intervention to another program (33%), among others. Most agencies highlighted the need for technical assistance.
Hayes et al. (1995) Grosskurth et al. (1995) Mayaud et al. (1997)	Tanzania (HCO)	RCT	Technical Assistance Infrastructure Development	Three-week training in the treatment of STDs was provided for staff of health centers and dispensaries. Training focused on enabling staff to make careful clinical examinations and applying treatment algorithms. In addition, drug supplies were delivered regularly to health facilities for treatment of STDs. Regular supervisory visits to crosscheck patient charts and availability of medication in site. Staff performed periodic visits to surrounding villages in order to encourage people to visit the clinic when necessary.	Incidence of syphilis. Incidence of urethritis. Incidence of HIV. Incidence of self-reported STD symptoms. Prevalence of STD in antenatal clinic attendees.	While comparable at baseline, the incidence of syphilis in the intervention group was lower (5%) than the control group (7%) at follow-up. The intervention reduced the incidence of HIV by 40%. While no difference in the prevalence of male urethritis was found between treatment and control conditions, the prevalence of symptomatic urethritis was reduced by 50% in the intervention group. No differences in the prevalence of any STD among antenatal clinic attendees were found across conditions.
Kelly et al. (2000)	USA (NGO)	RCT	Technical Assistance	Treatment A: Technical assistance manuals described how to implement research-based HIV interventions. Treatment B: Offered manuals plus a staff-training workshop on program implementation. Treatment C: Manuals, staff training, and follow-up consultation calls.	Adoption of intervention model by ASOs.	Treatment C had the highest adoption and use of the HIV prevention model in one-year follow-up, followed by Treatment B. ASOs used the behavioral intervention least often with clients who were women; suggesting that technical assistance for women-focused services is needed.
Kelly et al. (2004, 2006)	Africa, Central/East Europe, Central Asia, Latin America, Caribbean (NGO)	RCT	Technical Assistance Infrastructure Development	Control (n=42): Orientation of popular opinion leader (POL), delayed treatment, a computer, subsidized Internet, access to a network of other NGOs, and briefing papers on grant writing and evaluation. Experimental (n=44): Control + CD with POL curriculum. Post-curriculum consultation through distance education and technology transfer (i.e., telephone/internet/instant messages)	Percentage of NGOs in each condition developed a new program or modified an existing program. Changes in POL core elements in existing programs.	43% of experimental NGOs developed a new program based on the disseminated model compared to 17% of control NGOs. Nearly twice as many experimental NGOs either developed a new program or modified an existing program based on the disseminated model. Twice as many experimental NGOs than control NGOs incorporated POL components into their programs.
Klein et al. (2002)	New York, USA (GOV)	One-shot case study	External Partnerships	Interagency collaborations were developed to provide a continuum of care for New York State prison inmates. Collaborations between the Department of Correction, Department of Health, and community-based organizations included HIV education and VCT, outreach to inmates, peer training, condom distribution for family visits, HIV primary care, referrals, support groups for HIV-positive inmates, and case management for HIV-infected inmates upon release.	Breadth and extent of services available. Inmate attendance at trainings and education sessions. Number of HIV-positive inmates receiving transitional case management upon release.	54 facilities offer HIV education and anonymous counseling and testing services, 39 have peer education training programs, 57 have face-to-face transitional planning for HIV-positive inmates (with the remainder of agencies offering support through a hotline). In 2000, over 6,500 inmates attended peer trainings and HIV risk reduction sessions. Transitional planning services were offered to 468 HIV-positive inmates, with 185 receiving case management upon release.
Koenig et al. (2004)	Haiti (GOV)	One-shot case study	Infrastructure Development External Partnerships	To improve the scaling-up of HAART treatment, a network of community health workers was created. Additional clinical staff hiring took place throughout the region.	Number of people receiving HAART.	In the first year of the scale-up initiative, over 1,050 patients were treated with HAART.
Mitchell et al (2005)	Chicago, IL, USA (NGO)	One group pre-post test with follow-up	Infrastructure Development	Integrates prevention counseling and social services into the ASO by fostering ties between case managers and HIV-positive clients. The intervention focused on individualized risk assessments, risk reduction planning, and ongoing management with an emphasis on prevention education and supportive services.	Knowledge of HIV risk reduction. Identifying triggers and risks relapses.	Of clients with follow-up data (n=60), 68% reported working towards accomplishing or maintaining their identified goals.
Molitor et al. (2005)	California, USA (GOV)	One group pre-post test	External Partnerships	Hired community members worked with the AIDS Office of the California Department of Health Services to link HIV-positive persons of color and injection drug users into services for HIV care and treatment. Community members accompanied participants to their initial service appointments. Staff could refer clients to other organizations and services if they were more appropriate for participants’ needs (i.e., housing).	Number of linkages (e.g., receipt of a referred service).	Across 21 sites, referrals and assessments were conducted for 1453 persons living with HIV but without routine care. Almost 60% of participants referred were successfully linked to a program. IDUs were less likely than HIV-positive people of color to link into the program. Nonetheless, they were more likely to be referred and linked to other community services.
Morrill et al. (2005)	Massachusetts, USA (GOV)	One group pre-post	Infrastructure Development	The Massachusetts Prevention Planning Group (MPPG) underwent a restructuring to improve the use of behavioral data available. Three core areas were identified and redesigned: (1) group structure (decision-making rules, roles, and leadership); (2) data inputs (data availability and quality, and data presentations); and, (3) MPPG member characteristics (participation and representativeness of communities).	Decision-making structure index Leadership index Satisfaction with data index Member knowledge and skills index Member confidence index	Post-intervention increases in overall decision-making structure index were found. In particular, group members increased their agreement on suitable processes for decision-making, MPPG’s autonomy in a driven concurrence process and ownership of MPPG decisions. Members felt that leaders encouraged members to participate fully in all decision-making and that greater satisfaction of the data quality was presented in decision-making. No changes were found across knowledge and skills. Member confidence increased by the end of the intervention.
Parry et al. (2004)	Connecticut, USA (HCO)	One-group time series	Infrastructure Development External Partnership	Through multisectoral collaboration, the hospital created a multipoint disease management operation. Activities included: (a) creating of HIV service assessment and HIV clinical flowcharts to standardize practice, data collection, and communication across multiple venues); (b) hiring of a HIV nurse practitioner to provide clinical care, improve medication adherence, and systematically collect data; (c) disseminate evidence-based guidelines to reduce practice variation and improve clinical outcomes; (d) development of a multidisciplinary specialty consultation clinic for all newly diagnosed HIV patients and follow-up visits for HIV+ patients; (e) establishment of an electronic database tracking patient clinical information; hiring an adherence nurse counselor to assist patients with ART adherence; and (f) creation of extension teams to tackle specific issues such as MTCT.	Adequacy of prophylaxis (vaccination rates, percentage of patients screening for TB). Success in patient management indicators (ED visits, preventable admissions for opportunistic infections, HIV inpatient cost and length of stay) Adherence indicators (number of patients attending HIV specialty clinic, clinic no-show rate, and self-reported adherence). Quality of life indicators (Karnofsky scoring, patient satisfaction). Overall management indicators (flowchart completion rates).	Inoculation against influenza and pneumococcal infection increased by Year 3. Similarly, screening for HepB, HepC, and TB increased each sequential year. While clinic attendance rate did not change over three years, the assessment of medication adherence increased to 100% by Year 3. While emergency department visits have decreased, the number of hospital admissions due to HIV has increased. Nonetheless, length of stay has decreased. Hospital costs have remained flat. No changes in quality of life were observed and are attributed to measure insensitivity.
Peltzer et al. (2006)	South Africa (HCO)	Two group pre-post test	Infrastructure Development	Traditional healers in the experimental condition received a four-day training on HIV prevention and transmission, a reference manual, 100 razors and gloves, condoms, and follow-up meetings with intervention staff. Traditional healers in the control condition received a pamphlet with information regarding HIV/AIDS.	HIV knowledge. Distribution of condoms. Intention to work with biomedical practitioners. Referrals to biomedical practitioners.	When compared to healers in the control group, healers in the experimental condition reported increased HIV knowledge at follow-up. Increases in healers’ intention to work with biomedical practitioners increased for the experimental group but not for the control group. No differences in actual referrals were observed.
Perez et al. (2004)	Zimbabwe (HCO)	One group pre/post test	Infrastructure Development External Partnerships	VCT services were provided in a rural district hospital’s antenatal clinic. Women who tested HIV-positive were informed of their serostatus and offered a single dose of nevirapine. Nurse and social worker staff were trained in HIV testing and rapid testing. Community mobilization to enhance adoption of rapid testing and antenatal care was implemented.	Acceptance of HIV testing. Proportion of women returning to obtain their results. Percentage of women agreeing to nevirapine.	Across the 18-month intervention, 2,298 pregnant women were offered an HIV test. 93% of these women agreed to be tested for HIV and 75% returned to pick up their results. HIV prevalence was 20% (n=437). One-hundred and four of the HIV+ women received MTC prophylaxis through nevirapine.
Peterson & Randall (2006)	Michigan, USA (NGO)	One group pre-post test	Technical Assistance Infrastructure Development	Several strategies were used over a two year period: (a) building knowledge and skills to understand and appropriately use EBIs; (b) securing provider buy-in to implement EBIs; (c) selecting appropriate and feasible EBIs; (d) training health department staff and service providers on EBIs; (e) awarding contracts to implement EBIs; (f) utilizing capacity development partners at the national, state, and local level; (g) providing on-going, individualized technical assistance; and, (h) instituting commitment to intervention fidelity.	Proportion of CBOs implementing effective behavioral interventions (EBIs).	The proportion of state health department-funded CBOs implementing EBIs with fidelity increased from 5% in 2002 to 78% in 2004.
Poindexter et al. (2002)	Massachusetts, USA (HCO)	One-shot case study	Infrastructure Development External Partnerships	Using empowerment and participatory principles, a statewide training program was developed by the Department of Health and Boston University’s School of Social Work. Community-based HIV educators served as cofacilitators in the trainings and played an active role in the program’s planning, implementation, and evaluation. Rather than educating health workers on how to educate people about HIV transmission, the intervention focused on helping skills (i.e., importance of professional helping relationships, interaction techniques, advocacy, and self-care) and socializing them in the workplace (i.e., managing conflict, minimizing burnout, management skills).	Relevance of the training content to their work. Changes toward self-selected goals.	Most participants reported that the training was highly relevant for their work. Nonetheless, response rate for follow-up interviews was low (30%). Most participants reported progress on their chosen goals.
Ramos & Ferreira-Pinto (2002)	Texas, USA (NGO)	One group pre-post test with follow-up	Technical Assistance Infrastructure Development External Partnerships	In order to create sustainable, collaborative training networks among CBOs. The capacity building intervention was implemented in three phases: (a) training on building the organizational capacity of individual CBOs; (b) gaining experience in the integration of curricula and participation as cofacilitators; and (c) reinforcing lessons learned by facilitating trainings of other CBOs.	Performance indicators (infrastructure development, program development, networking, and collaboration). Impact on program capacity and changes in level of services.	While only half of the organizations had developed a long-term strategic plan, over two-third of CBOs had created an organizational chart and were formally incorporated as non-profit institutions from baseline to follow-up. Only 58% of organizations had a systematic process for designing programs. No data presented on networking and collaborating outcome markers, or impact on program capacity or changes in level of services.
Richie et al. (2001)	New York, USA (GOV)	Two group pre-post test	Technical Assistance Infrastructure Development External Partnerships	The Health Link intervention sought to assist drug-using jailed women reintegrate into their communities, decrease their STI risks, and avoid rearrest. Intervention services consisted on direct services (case management during arrest and after release), technical assistance for community service providers, staff support for strengthening a network of service providers, and policy analysis and advocacy to reduce barriers to re-entry. Control condition was composed only of jail-based services.	Program retainment Rearrest rate	Results come from a previous evaluation (1994–1996). 12-month retention rate was 35%. Compared to women who volunteered for the jail program but who were ineligible due to their location (59%), women receiving full Health Link services had lower rearrest rates (38%).
Richter et al. (2006)	USA (NGO)	One group pre-post test	Infrastructure Development Technical Assistance	In order to build CBO capacity, providers who are primarily responsible for overseeing and developing HIV programs were invited to participate in the CDC/ASPH Institute for HIV Prevention Leadership. Providers are exposed to a 4-week curriculum on HIV prevention programming, health behavior theories and strategies, program planning and management, and computer competency.	Knowledge of core components. Satisfaction with Institute. Transferability of lessons learned to CBOs prevention efforts.	Increases in knowledge were found from baseline to follow-up. Participants reported high scores in Institute satisfaction and transferability to their CBOs.
Shea et al. (2006)	Maryland & Massachusetts, USA (GOV)	One-shot case study	Technical Assistance Infrastructure Development	In Maryland, DEBIs were promoted by creating multiyear contracts and building provider capacity by training program managers and evaluators on how to create logic models and evaluation plans for the DEBI programs. In addition, a training-the-trainer model was followed to tailor the training to the needs of local agencies and communities. Extensive training and assistance was provided throughout. In Massachusetts, DEBIs were promoted by being required in their request for program funds. Workplans required at least one DEBI program, which could be combined with other non-DEBI components. Capacity building and technical assistance was tailored to the goals and expectations of each agency. In addition, 4-day training series were revised and regional program development meetings were convened twice a year.	Maryland: Number of DEBI programs funded Number of evidence-based programs funded Massachusetts: Number of DEBI programs funded Number of evidence-based programs funded	By 2005, 98% of all Maryland HIV health education programs were evidence-based programs. 34% of the interventions were DEBIs; 64% of the interventions came from the Compendium of Effective Interventions or local best practices. By 2005, 95% of all Massachusetts HIV health education programs were CDC-diffused (note: not necessarily a DEBI – data not shown).
Takahashi et al. (2007)	California, USA (GOV)	One group pre-post test	Infrastructure Development External Partnerships	The intervention provided capacity-building activities for a network of community based social services and LGBT organizations targeting Asian Pacific Islanders. Program activities focused on creating organizational stability and viability by providing $5,000 per year for seed money to develop trainings and workshops. In addition, one-on-one organizational “coaching” and training workshops to increase organizational skills related to HIV-prevention program development	Organizational viability (diversification of funding sources) Organizational stability (similar budgets across program sizes) Agencies’ HIV knowledge and stigma	Increases in organization viability were noted across organizations. Agencies reported greater capacity in financial, human relations, and strategic management issues. Nonetheless, agencies reported decreases in organizational stability. This trend was attributed to an increase in the number of paid staff and board members and a decrease in volunteers. Agency knowledge about HIV and HIV sensitivity increased.
Xiaoming et al. (2000)	China (HCO)	RCT	External Partnerships	The intervention’s delivery sought to merge family planning and HIV prevention services as a bundle. Program activities included: (a) reading materials, (b) weekly 30 minute radio programs, (c) workshop and videos available through village workers and doctors, (d) discussion groups by women leaders and family planning providers, (e) bimonthly 10-minute home visits, (f) individual counseling, and (g) free supply of condoms. Control communities did not receive any program.	AIDS Knowledge. Attitudes toward AIDS prevention. Sexual behavior. Condom Use.	80% of young adults in the intervention group reported correctly to all questions on AIDS knowledge. No differences were noted for the control group. While both conditions had similar attitudes at baseline, the experimental communities reported increased behavioral intentions to prevent HIV and positive regards toward people with HIV at follow-up. At follow-up, the number of sexual intercourse occasions increased in the experimental condition and decreased in the control condition. Condom use in the experimental group increased by 5%, yet no change was observed in the control group. Number of partners remained stable in both conditions.

Interventions using technical assistance components presented internal (e.g., improved in agency performance) and external (e.g., proportion of agencies adopting a new program) agency indicators among their primary outcomes. Only one study using technical assistance reported a positive increase in agency staff behavior change. Three studies reported improved outcomes among community members; two studies reported increased HIV prevention behavior change among clients while another reported decreased HIV/STI prevalence. All studies reported positive effects from baseline to follow-up in their primary outcomes.

Taken together, these studies suggest that technical assistance is a promising system-level strategy. Nonetheless, the impact of these interventions was variable. In particular, technical assistance components were given to a large number of agencies yet a smaller proportion adopted these interventions. Harshbarger et al. (2006), for example, reported that staff members from 229 agencies were training in the VOICES/VOCES intervention. At the three-month follow-up, more than a third of the agencies had decided to forgo implementation. Another third decided to implement the intervention with modifications, with no consideration as to how it would affect the intervention’s success. On the other hand, the studies by Kelly et al. (2000, 2006) deserve particular mention because they identify critical intervention components when delivering technical assistance to NGOs. In their randomized control trial comparing different teaching components within a national technical assistance program in the United States, Kelly et al. (2000) found that technical assistance efforts may have the strongest effect when multiple teaching components (i.e., intervention manuals, staff training, and follow-up consultation calls) are delivered as a bundle. In a multi-site international study, Kelly et al. (2004, 2006) found NGOs in the treatment condition (e.g., receiving distance education and technology transfer through the telephone, emails, or instant messaging) were more likely to implement a program than control NGOs. These findings are promising and highlight that system-level interventions including technical assistance components should carefully consider the inclusion of different types of learning materials as well as the integration of multi-communication modalities.

Infrastructure Development

Seventeen studies included infrastructure development components (see Table 2). The most common intervention strategies included skill building activities (8 studies; 47%) such as increasing patient-provider competency and efficient HIV prevention outreach strategies, followed by organizational restructuring (7 studies; 41%) such as the creation of institutional decision-making groups, and access to material resources (6 studies; 35%) such as computers and condoms. Only one study by Takahashi and colleagues (2007) discussed the access to seed-money for trainings and workshops. Infrastructure development components were found more often in the U.S. than in international settings (see Table 1).

Many of the studies including infrastructure development reported agency-level outcomes. Five studies reported internal agency-level outcomes. Four of these studies reported increased internal agency functioning; one study reported decreases in internal agency functioning due to a decrease in number of volunteers as greater number of paid positions were created in the agency. Three studies reported increased external agency outcomes such as an increase in number of programs implemented by the agency. Infrastructure development programs also included positive increases in agency staff’s skills and knowledge (n=2). Five studies found increases in client health behavior (e.g., integrating HIV care to antenatal care). Two studies found decreases in STI/HIV prevalence and incidence markers.

Most studies reported positive effects across their primary outcome measures. Overall, these studies suggest that system-level initiatives including infrastructure development components can help increase agency functioning and HIV prevention outcomes. The study by Morrill et al. (2005) deserves particular mention as an interesting case study on how the Massachusetts Prevention Planning Group restructured their internal processes to improve its decision-making process. Equally important were the findings on provider-communication skill building activities as an infrastructure development initiative to improve client outputs. Bluespruce et al. (2001), for example, found that training health care providers increased their self-efficacy to counsel patients on risky sexual practices as well as their ability to integrate sexual health questions into routine check-ups. Furthermore, Grosskurth et al. (1995) found that training health care providers in clinical examinations for STD-related symptoms, combined with community mobilization, decreased the incidence of HIV among clinic attendees. Taken together, these studies suggest that system-level interventions using infrastructure development components may benefit by activities that aid refine an institution’s internal processes (e.g., organization stability and community-input in the decision-making process) while also paying attention to the training needs of agency staff.

External Partnerships

Twelve studies included external partnerships (see Table 2). The most common intervention strategies included coalition building across community stakeholders (5 studies; 42%), followed by community mobilization (4 studies; 33%) and service bundling (4 studies; 33%). External partnership programs were based in the United States (see Table 1).

Studies including external partnerships reported increases in agency-level internal (n=4) and external (n=2) outcomes, respectively. Six studies measured changes in clients’ behaviors. Five of these studies found positive increases in clients’ behaviors. One study with an external partnership component informed decreases in HIV/STI prevalence and incidence markers.

The bundling of services, per se, is variable and may not always lead to clients’ use of services. Blank et al. (2005), for example, found that only 27% of all MSM attending an event in a non-traditional health venue accessed one or more services. This suggests that the availability of bundled health services may result in poor client uptake if the program setting is not considered carefully. On the other hand, given the increasing needs in bundling services and maximizing HIV prevention and care efforts, it is encouraging to find most system-level interventions with external partnership components reported positive effects.

Discussion

Overall, our findings suggest that system-level initiatives are a promising modality if we are to maintain or increase institutional capacity to provide HIV prevention services and rollout evidence-based interventions. From our review, we found three overarching themes that may strengthen system-level strategies. Within technical assistance, Kelly et al.’s (2000, 2004, 2006) work suggests that multiple highly-active education components in technical stategy are most effective. Particularly in a global world, using new technologies for distance learning and consultation seems promising and increases iterative consultations. Within infrastructure development, restructuring agency procedures to enhance internal staff communication and feedback seems vital. Furthermore, staff-training initiatives in interpersonal communication and public health skills seem promising as a mechanism to strengthen agency productivity and stability. External partnership strategies linking existing resources into bundles were found to be effective. In an era of scare funds for new services, the impact of service linkage is particularly heartening.

Reported intervention efforts varied by geographic locale, highlighting different epidemiologic priorities in the HIV epidemic. Interventions in the U.S., for example, commonly focused on strengthening the capacity of agencies working with subgroups at greatest risk for HIV infection (e.g., men who have sex with men and racial/ethnic minority women) in the U.S. System-level efforts in the U.S. focused on infrastructure development (e.g., organizational restructuring of community planning groups, or the creation of a database to monitor quality of care in a hospital) and external partnerships (e.g., coalition building among community stakeholders interested in a particular population group). These strategies reflect the need to strengthen the individual and collective resources of U.S.-based agencies in order to deliver, implement and tailor evidence-based HIV prevention education and treatment.

International locations with high HIV prevalence and incidence, on the other hand, sought to deliver HIV prevention and treatment services to the population in a defined catchment area. Most system-level interventions focused on infrastructure development (e.g., access to material resources for day-to-day functioning such as computers and funding) and external partnerships (e.g., the bundling of reproductive and HIV prevention services within a clinic). It is important to note that the selection of infrastructure development and external partnerships may reflect the need for prompt HIV prevention and treatment rollout in regions with high infection rates and mitigating social and economic vulnerabilities. Strikingly, very few of the studies meeting eligibility criteria focused on international settings.

Effective programs, both U.S.-based and international, commonly incorporated multiple intervention strategies. The presence of multiple strategies in an intervention limits our ability to understand the efficacy of the intervention components independently. While this is a common critique of public health programs (commonly referred to as an ‘intervention black box’), our findings suggest that, similar to individual and community prevention programs, system-level interventions require multiple strategies in order to support an agency’s ‘behavior change’ during service delivery, optimization and maintenance. Below, we propose several recommendations for future work in HIV/AIDS system-level programs.

Efforts to curtail the spread of HIV/AIDS, locally and globally, have highlighted the importance of identifying contexts and populations and tailoring theoretical constructs and strategies to improve program effectiveness. Unfortunately, although perhaps implicit, none of the reports reviewed mentioned explicitly the theoretical framework informing their system-level program. Moreover, the ecological nature of system-level interventions requires that these be matched with evidence-based community-level and individual-level health education programs. Consequently, peer-reviewed journals should allocate publication space so authors may include richer descriptions of the system-level constructs informing their program. These descriptions will facilitate program replication, translation, and adaptation, and, avoid wasting material and human resources, especially in communities where these resources are severely strained. Similarly, formative work is required prior to the implementation of a system-level intervention and the importance of this should not be trivialized (Jenkins & Carey, 2005). Researchers and practitioners’ experiences in developing and implementing system-level interventions are needed in the scientific literature. Without an understanding of the needs and resources required during the formative phase, replication of a system-level intervention may be difficult.

Stronger evaluation components are required in system-level intervention programs. The acknowledgement that system-level programs are context-specific highlights the importance of developing strong causal linkages between the intervention components and the desired outcomes (internal validity) and undermines the usefulness of randomization for generalizability (external validity). While some would argue that a randomized control trial design is the “gold standard” within system-level interventions, these designs may not be feasible because of pragmatic considerations such as increased costs, ethical concerns in randomizing systems, identifying and selecting characteristics for randomization and comparison of trials arms, among others (Rapkin & Trickett, 2005). Borrowing from lessons learned from community-level interventions (Shinn, 1990), system-level interventions should include elements that, when possible, strengthen the system-level intervention’s internal validity (i.e., having multiple groups, having different operationalizations of the outcome, alternating or reverting treatment conditions across groups, offering delayed treatment to a comparison group, and including multiple observations before and after treatment).

Quantitative and qualititative system-level process and outcome indicators deserve greater attention in the literature (Lipsey & Cordray, 2000). Careful programmatic and data monitoring procedures should be in place to detect potential biases such as measurement corruptibility (Rossi, 1997). When possible, success indicators should encompass multiple levels of analysis (i.e., agency outcomes, staff outcomes, and HIV markers).

Data linking system, community, and individual-level outcomes should be explored further in intervention evaluation. Nested data allow the testing of cross-level effects that, until recently, were disheartening because they provided biased estimates and violated the assumption of independence in procedures such as ANCOVA and linear regression (Raudenbush & Bryk, 2002). New statistical advances in this area, however, have overcome these issues and allowed for computation of nested models through packages such as Mplus and HLM. While a detailed review of this analytic technique is beyond the scope of this paper (see Bingenheimer & Raudenbush (2004) for an introduction to multilevel modeling in health), we provide several examples to illustrate the potential power of this technique. Examples of multi-level models within system-level programs may include: (1) exploring the effects of an agency capacity-building program (i.e., increases in funding) on the characteristics of the communities (i.e., community capacity) these intend to serve; (2) assessing the effects of a training for physicians (i.e., provision of social support toward HIV+ clients) on patients’ behavior (i.e., antiretroviral adherence); or (3) appraising how external partnerships increase the diffusion of an evidence-based intervention across multiple agencies and impact a region’s HIV/AIDS prevalence and incidence.

We acknowledge that our review has several limitations. First, our findings may be subject to publication bias given that unpublished studies offering weak or null effects are not reported as often in the scientific literature. Second, our criteria for peer-reviewed journal publications excluded system-level interventions implemented by international agencies (i.e., UNAIDS, FHI, WHO, World Bank) or health policies that are system-level interventions (i.e., needle exchange programs). While the bulk of system-level interventions tend to be carried out by these international agencies, we chose to exclude these interventions from our analysis as a recent paper by Druce et al. (2006) addressed these iniatives. Nonetheless, our review does include multiple system-level interventions within non-government agency settings. Furthermore, as the demand for peer-reviewed evidence-based findings continues to increase, we present the evidence in peer-reviewed scholarly journals in hopes that their evidence informs on-going programmatic efforts. Nonetheless, future studies reviewing system-level programs implemented by international agencies is useful. Finally, the variety in outcome operationalization and intervention strategies did not allow us to perform a meta-analysis. As more HIV system-level interventions are published with similar outcomes, however, we hope that measuring effect sizes across different system-level intervention strategies will be possible.

These limitations notwithstanding, our literature review of the peer-reviewed evidence on system-level interventions suggests that they are effective and varied. Future research focusing on expanding/refining system-level theoretical frameworks and sharing lessons learned from implementing these initiatives are needed. On-going prevention and treatment efforts can incorporate system-level interventions to strengthen their efforts from a top-down approach that may lead to strengthening evidence-based community, behavioral, and clinical HIV prevention and care efforts.