HIV Prevention Technology Transfer: Challenges and Strategies in the Real World

Abstract

Objectives. We examined implementation of evidence-based interventions for HIV prevention at community-based organizations in Los Angeles County, CA.

Methods. We conducted 2 waves of interviews with 34 organization staff members. We analyzed activities reported by staff in the phases (preimplementation, implementation, and maintenance and evolution) and activities defined by the technology transfer model for evidence-based HIV prevention interventions.

Results. Staff members were able to select, adapt, and implement evidence-based HIV prevention interventions despite challenges in each phase of technology transfer. Preimplementation challenges included lack of information and poor fit between the interventions and organizations' clients. Implementation challenges included retention of participants across intervention sessions and staff turnover. A challenge in the maintenance and evolution phase was enhancing staff skills in outcome monitoring and cost analyses.

Conclusions. Technical assistance must be matched to the specific challenges found in each phase of technology transfer. Successful transfer of evidence-based HIV prevention interventions will depend on their continued uptake and use by organization staff. This study highlights directions for improving communications regarding appropriate modifications to these interventions and for organizational planning to continue adapted interventions.

Community-based HIV prevention programs in the United States are strongly encouraged by funding agencies and policy-making bodies to implement interventions with evidence of effectiveness.^1–3 Substantial investments promote the dissemination, adaptation, and diffusion of evidence-based interventions (EBIs) from research settings into the work of community-based organizations (CBOs), a process called technology transfer.^4,5 Since 2003, 141 CBOs in the United States have received funding to implement interventions promoted through the US Centers for Disease Control and Prevention (CDC) Diffusion of Effective Behavioral Interventions project.⁴

In 2005, 10 such interventions were implemented in Los Angeles County, California, including Healthy Relationships; Mpowerment; Many Men, Many Voices; Popular Opinion Leader; Real AIDS Prevention Project; RESPECT; Safety Counts; SISTA (Sisters Informing Sisters on Topics about AIDS); Street Smart; and VOICES/VOCES. Of the 49 agencies and CBOs funded for HIV prevention by the health department, 17 implemented interventions from the diffusion project. The remaining agencies used locally evaluated or other evidence-based HIV prevention interventions. In 2005, 5 organizations received direct funding from the CDC to adapt and implement interventions from their diffusion project.

Providers of capacity-building assistance have reported challenges in technology transfer, such as inadequate funding to conduct the EBI, limited access to training or technical assistance, and limited guidance on appropriate local modifications.^6–8 Researchers have observed that characteristics of the intervention may serve as barriers: for example, lack of teaching materials and materials that are easy to use, limited access to training, and restrictions on modifying the intervention.^9,10 Barriers reported by public health departments include the limited availability of EBIs and cost-effective technical assistance and failure to cross-train CBO staff.^11,12 Scant research has been conducted on how common these challenges are across CBOs or how they are resolved as EBIs are scaled for implementation.^13,14 Information about how CBOs enhance the external validity of EBIs and minimize challenges in technology transfer could facilitate future efforts to diffuse evidence-based prevention practices.¹⁴

The Los Angeles County HIV Prevention Plan for 2004 to 2008 listed as key priorities delivering evidence-based HIV prevention services, conducting multisession interventions, providing HIV prevention services to HIV-positive persons, and conducting program evaluations.¹⁵ Culturally specific programming was also needed to address the disproportionate effect of HIV/AIDS among African Americans and Latinos in Los Angeles County.¹⁶ The plan fostered tremendous interest in EBIs among local CBOs throughout 2005, providing a unique window of opportunity to study HIV prevention technology transfer. We assessed the activities of CBOs as they implemented HIV prevention EBIs and identified challenges encountered and strategies used in technology transfer, in partnership with the City of Los Angeles AIDS Coordinator's Office.

METHODS

Participants and Procedures

We identified participants via publicly available lists of staff at HIV/AIDS organizations that were implementing HIV prevention EBIs. Recruitment letters and e-mail messages were sent to individuals and to e-mail discussion lists of HIV/AIDS organizations in Los Angeles County. Eligibility criteria were employment in a CBO that provided HIV prevention services in Los Angeles County, involvement in technology transfer activities (e.g., review, selection, implementation, or evaluation of EBIs), and willingness to participate in 2 recorded interviews and a brief background survey. This brief survey included questions about staff background and the organizational history of conducting HIV prevention. We interviewed 1 to 3 staff members from each organization implementing an intervention. Of the 41 individuals who were contacted to participate in the study, 34 agreed to participate, 3 declined participation, and the remaining 4 were ineligible (the persons contacted were not involved in technology transfer).

We conducted all interviews in person and coded them according to the 3 phases of HIV prevention technology transfer—preimplementation, implementation, and maintenance and evolution—delineated by the technology transfer model.¹⁷ This model was developed by CDC scientists to improve the dissemination of EBIs and to build the capacity of HIV prevention service providers to use them. To ensure successful transfer, the model underscored the importance of communications among HIV prevention service providers, researchers, and other stakeholders in HIV technology transfer during each phase as well as planning to implement and evaluate adapted EBIs. Specific programmatic CBO activities were associated with each phase (Table 1). We conducted the first wave of interviews between December 2005 and May 2006, when most of the CBOs had completed the preimplementation phase and had begun implementing the EBIs. We carried out the second wave of interviews between August and October 2006, when many of the organizations had completed 1 year or more of implementing the interventions and had begun planning for sustainability.

TABLE 1.

Phases and Activities Defined by the Technology Transfer Model

Phase	Activity	Examples
Preimplementation	Identify need for new intervention	Review epidemiological data, client data, or community assessments
	Acquire information	Acquire intervention packages, talk with staff from other agencies or with behavioral scientists
	Assess fit	Consider feasibility, fit with organization, or linkages to other services
	Prepare organization and staff	Build organizational support, tailor the intervention, provide staff training
Implementation	Secure technical assistance for implementation	Seek assistance for intervention adaptation from scientists, funders, or technical assistance providers
	Conduct process evaluation	Monitor whether intervention was delivered as planned, track services delivered to clients
Maintenance and evolution	Support staff for continued implementation	Provide booster training, seek technical assistance to identify needed improvements
	Support organization change	Integrate the intervention into organizational operations, seek continuation funding
	Conduct process through outcome evaluation	Review costs to the organization, outcomes of the intervention, benefits of using the intervention

Note. This table was adapted from Kraft et al.¹⁷

We derived a semistructured interview from the technology transfer model and research on the adoption of evidence-based HIV prevention programs.¹⁸ (The text of the interviews is available as a supplement to the online version of this article at http://www.ajph.org.) We reviewed and edited the study questions after receiving feedback from a community advisory board that included HIV prevention policymakers, HIV prevention program directors, and community advocates. Sample questions included, “Who is involved in making decisions about adding new HIV prevention programs or making major changes to existing programs?” (preimplementation), “What are some of the challenges you've seen in your agency's implementation of these interventions?” (implementation), and “How has the experience of using EBIs affected how the agency responded to new or emerging client HIV prevention needs?” (maintenance and evolution).

Analyses

Interviews were transcribed and entered into Atlas.ti version 5 (Scientific Software Development, Berlin, Germany). We derived primary codes from the 9 activities corresponding to the 3 phases of the technology transfer model. We also created 1 additional activity code, concerning training or technical assistance, for the preimplementation phase, consistent with recent CDC emphasis on the selection of EBIs, for a total of 10 technology transfer activity codes.⁸ An example of this type of activity was consulting with a funder or technical assistance provider about which EBIs to use in an organization. We also created 2 new codes representing strategies and challenges. Coding reliability for the 12 codes among 3 coders was established with a random sample of 3 interviews from each wave. Kappa ranged from 0.82 to 1.00, well above the recommended 0.70 level for similar research.¹⁹

Descriptive statistics were obtained from the background survey. We analyzed 2764 coded transcript segments. We used the 10 technology transfer model activity codes to group segments into the preimplementation (1004 segments), implementation (896 segments), and maintenance and evolution (864 segments) phases. We used the strategies and challenges codes within each activity to identify primary themes within each phase. The number of staff members mentioning specific challenges indicated the salience of these challenges within each phase and activity.

RESULTS

Thirty-four participants completed the first wave of interviews, and 33 completed the second wave. We made repeated, but unsuccessful, attempts to schedule the second interview with the last person. The typical participant was Latina; was a program director, manager, or coordinator; had 10 years or more of HIV prevention experience; and had received training in specific evidence-based HIV prevention interventions, group facilitation, and behavior change theories (Table 2). Twenty-nine percent of participants worked for CBOs that were implementing more than 1 EBI. Their organizations had provided services for an average of 12.4 years (SD = 5.9) and currently conducted an average of 4.5 HIV prevention programs (range: 1–15). An average of 7.5 full-time staff (range: 0–45) and 2.1 part-time staff (range: 0–8) worked on EBIs at an organization. By October 2006, 18% of participants (n = 6) had either changed positions or had left the CBO. Interviews with these participants were completed before October 2006, and their data were included in the analyses.

TABLE 2.

Characteristics of Community-Based Organization (CBO) Staff (n = 34) Conducting Evidence-Based Interventions for HIV Prevention: Los Angeles County, California, 2005–2006

	CBO Staff, No. (%)
Gender
Female	21 (62)
Male	10 (29)
Transgender male-to-female	3 (9)
Ethnicity/race
Latino/Hispanic	18 (53)
American Indian/Alaska Native	0 (0)
Asian	1 (3)
Native Hawaiian/Pacific Islander	0 (0)
Black/African American	4 (12)
White	4 (12)
> 1 Race	7 (21)
Primary role
Executive director	3 (9)
Program director/manager/coordinator	23 (68)
Evaluation staff	2 (6)
Facilitator/health educator	5 (15)
Did not report	1 (3)
Tenure with CBO
6–11 mo	6 (18)
1–2.9 y	8 (24)
3–4.9 y	6 (18)
5–9.9 y	10 (29)
≥ 10 y	4 (12)
Experience with HIV prevention
< 6 mo	0 (0)
6–11 mo	6 (18)
1–2.9 y	1 (3)
3–4.9 y	4 (12)
5–9.9 y	9 (26)
≥ 10 y	14 (41)
Intervention-related training received
Adaptation	18 (53)
Behavior change theories	28 (82)
Budget development	14 (41)
Curriculum design	23 (68)
Specific evidence-based interventionsa	27 (79)
Group facilitation	24 (71)
Health education	23 (68)
HIV counseling and testing	17 (50)
Program coordination	13 (38)
Program evaluation	21 (62)
Statistics	9 (26)
Substance abuse prevention	15 (44)
Survey/questionnaire development	17 (50)
Other	4 (12)
CBO clients receiving HIV prevention services/mo, no.
1–50	11 (32)
51–250	18 (53)
251–500	4 (12)
500–1000	1 (3)

^aFor example, Mpowerment, SISTA (Sisters Informing Sisters on Topics about AIDS).

Preimplementation Phase

Participants were asked to describe their experiences with learning about, seeking out, or selecting EBIs. For some CBO staff members, the process of selecting their EBI coincided with the preparation of funding applications for the intervention, leaving little or no opportunity for training or necessary technical assistance. Prominent challenges to technology transfer identified in this phase were limited accessibility of information on the EBIs, poor fit of interventions with the organization's ethnically and socially diverse client populations, and a lack of knowledgeable providers from whom CBOs could obtain technical assistance for selection or adaptation (Table 3).

TABLE 3.

Strategies Used and Challenges Encountered by Community-Based Organizations (CBOs) in Technology Transfer in Evidence-Based Interventions (EBI) for HIV Prevention: Los Angeles County, California, 2005–2006

Activity	Strategy	Challenge
Preimplementation phase
Identify need for new intervention	Examine agency data	Adaptation of intervention to new population
	Solicit stakeholder feedback	Outreach
		Structural barriers to data collection
		Interagency competition for clients
		Insufficient funding to target population
		Limited intervention information available
Acquire information	Seek information via colleagues, conferences, the Internet, and meetings	Unavailability of intervention training
	Obtain information via secondhand sources	Inaccessibility of intervention manual
Assess fit	Review existing client and program data with intervention requirements	Poor fit between the intervention and the population CBO was funded to serve
Prepare organization and staff	Modify key characteristics of EBI	Funder demand for major modifications to EBI
Secure TA for selectiona	Seek TA from external consultant	Unavailability of TA resources
	Seek TA from provider of capacity-building assistance
Implementation phase
Secure TA for implementation	Obtain TA from funders	Funder demand for major modifications to EBI
	Receive TA via conference calls	Funder rejection of modifications to EBI
Conduct process evaluation	Review staff preparedness to conduct EBI	Cycles of staff turnover during EBI
	Review client rapport and satisfaction	Retention of clients across multiple intervention sessions
	Review collaborations to promote EBI recruitment
	Increase local relevance by targeting specific populations
	Conduct quality-assurance monitoring
Maintenance and evolution phase
Support staff for continued implementation	Provide booster training or additional training	Need for staff trained in EBI-related skills
	Select/hire staff with EBI-related skills
	Allocate agency resources to EBI
Support organizational change	Integrate EBI into agency and programs	Limited agency resources for EBI
		Inability to make necessary adaptations to EBI
		Difficulty serving clients with multiple health issues
Conduct process through outcome evaluation	Obtain additional funding to conduct outcome evaluation	None identified

Note. TA = technical assistance.

^aThis code was added to reflect recent Centers for Disease Control and Prevention emphasis on the selection of EBIs in the preimplementation phase.

Identifying a need for a new intervention.

Participants described using data from existing client services, informal and formal community assessments, and direct client feedback to identify possible new interventions needed. Only 1 of the participants mentioned reviewing epidemiological data to identify the need for new interventions. Only 7 participants listed any challenges with this activity. The issues they listed were diverse, and none was mentioned repeatedly across participants.

Acquiring information.

Participants reported searching for information after hearing about EBIs at conferences and meetings or from colleagues. Sixteen participants mentioned that it was challenging to acquire EBI manuals or training. A participant with 5 to 10 years HIV prevention experience said,

I didn't realize how strict the people are to give out the information… . You can't just buy it, you have to go through the training… . We had missed like already two trainings that were happening … so the next time around was going to be too late.

Assessing fit.

Participants mentioned reviewing clients' and agency information in conjunction with the EBI requirements to examine fit and feasibility. Seventeen participants observed inadequate fit between the selected interventions and the populations for which organizations were funded. A participant with 10 years or more of experience in HIV prevention said,

Evidence-based interventions don't fit with the agency because of the complexity of the target population … homeless, Latino and African American, men and women … because of the issues that they deal with, such as language barriers and culturally related issues, some of these interventions really don't happen to apply specifically to this population.

Preparing organization and staff.

To prepare for implementation, participants modified the key characteristics of their EBIs by combining or shortening sessions, physically relocating the intervention, and editing EBI curricula for cultural and linguistic appropriateness. Notably, 12 participants mentioned being required by their funders to make what they viewed as significant modifications to the interventions. A participant with 5 to 10 years HIV prevention experience reported being required to adapt program components and to include persons with HIV infections: “We know that prevention is different for [HIV] positives and [HIV] negatives. Now you are asking us to not only adapt and tailor but to develop another curriculum to complement this.”

Securing technical assistance for intervention selection.

Only 3 participants reported seeking technical assistance to select their interventions when asked what EBI-related resources were available to them. A participant with 5 to 10 years of experience reported that providers of capacity-building assistance “didn't know how to do it either. There were no other resources available.”

Implementation

At the time that participants were asked about implementation activities, many had been implementing the program for 6 months or more. For some CBOs, this phase focused on balancing contractual obligations regarding program components against their ability to implement the EBI. In this phase, challenges described by participants included receiving technical assistance that conflicted with the interventions, continuing the intervention during cycles of staff turnover, and retaining participants across program sessions.

Securing technical assistance for implementation.

Participants reported obtaining technical assistance primarily from their funders rather than from other potential sources of technical assistance such as researchers or CDC-funded technical assistance providers. However, 7 participants described difficulties in applying the technical assistance received from funders, such as modifications that CBO staff viewed as inconsistent with the EBI or that limited their ability to enhance implementation. A participant with 3 to 5 years of experience described being required to expand an intervention from 1 to 3 sessions “without having any evidence that that's gonna make it any more or any less effective.” A participant with 5 to 10 years of experience said, “We've tried to combine sessions because of retention, and we've gone through a couple of series where we've combined a couple of sessions together, and the clients have made it through it”; however, the CBO was reprimanded and required to revert to 5 independent sessions.

Conducting process evaluation.

Participants described various means by which they monitored EBI delivery and clients' responsiveness. These included assessing and improving staff preparedness to conduct the interventions, gauging client rapport and satisfaction, increasing recruitment of participants via interagency collaborations, making the intervention more locally relevant by targeting specific risk populations, and conducting quality assurance by reviewing the completeness of EBI data forms. Fifteen participants reported finding it difficult to continue the programs as staff turnover occurred. A participant who had less than 1 year of HIV prevention experience said, “We're spending all our time trying to learn what to do, then adjust what we're gonna do, and by the time we're actually doing it, then our staff member leaves.” Fifteen participants reported difficulties with retaining participants in EBIs that required follow-up contacts or multiple sessions. A participant with 1 to 3 years of experience commented,

The big barrier with retention involving adults 24, 25 and up is people work and have lives. People go on vacation. They'll show up to one session, maybe two, but to show up for all three is really hard unless we're providing … a big incentive.

Maintenance and Evolution Phase

In 2006, health department funding for some of the CBOs carrying out HIV prevention EBIs was extended or alternate funding sources were obtained. Only 1 of the participants reported plans to examine program outcomes and overall costs of delivering EBIs. The key challenge observed in this phase was a shortage of staff with necessary skills.

Supporting staff for continued implementation.

Participants reported providing additional training to their staff, selecting or hiring staff who were already trained in intervention-related skills, and ensuring that organizational resources were made available to strengthen the EBI. Sixteen participants reported lacking staff with the skills necessary for EBI delivery. A participant with over 10 years of experience described having staff members who needed training not only in administering a questionnaire but in understanding the theory behind it: “I don't think there has been training out there that gives the staff ability to do that.”

Supporting organization change and institutionalization.

Participants whose agencies had begun making changes related to the EBIs described steps to integrate the EBIs into their other programs. A participant with 5 to 10 years of experience said, “We are continuously looking to expand the services that we offer.” An EBI was particularly successful because it fit “with all the care services that we have, case management, education, mental health, transportation, housing, food bank.” Participants also mentioned such challenges as having insufficient organizational resources to implement the intervention, make necessary adaptations to it, or serve clients with multiple health issues in addition to HIV.

Conducting process, outcome, and cost evaluations.

Only 1 participant reported having a plan to conduct outcome monitoring or evaluation for an EBI. This participant had over 10 years of experience and had been awarded funding specifically to conduct process and outcome evaluation of a CDC intervention from the diffusion project.

DISCUSSION

Our goal was to gain insight into how CBOs strategically selected, implemented, and sustained HIV prevention EBIs while they addressed challenges in technology transfer. The developers of the technology transfer model stressed the significance of clear communications among CBOs, researchers, and other stakeholders in technology transfer as well as planned implementation and evaluation of adapted HIV prevention EBIs. Our findings suggested a need for improvement in both communications and planning. The model's phases were useful for characterizing specific gaps in technology transfer.

Current EBI dissemination was not effectively reaching all of the CBOs seeking to implement these programs and the clients they served. The immediate preimplementation communication barrier of lacking access to EBI information was a planning barrier: CBOs selected interventions that they were underprepared to carry out. At minimum, publicly available materials and consultation were needed to assist organizations in determining the staffing, agency resources, and staff skills development required to conduct the intervention. The diffusion of EBIs was prematurely truncated by the restriction that EBI information was only released to organizations that could afford to send their staff to training sessions.

Accelerated research and development of interventions that included vulnerable and diverse populations were needed to respond to the poor match between available EBIs and clients served by CBOs. The populations disproportionately affected by HIV in Los Angeles County were African Americans and Latinos, yet there was only 1 CDC diffusion intervention, SISTA, that was appropriate for African American women.⁸ Until greater diversity is achieved in the pool of available EBIs, the content of technical assistance in this phase must emphasize adaptations that promote cultural and linguistic fit with local target populations.

The implementation of EBIs was not well integrated with the contractual and compliance contexts in which CBOs operated, making reinvention more likely. Reinvention occurs when components believed to be responsible for an intervention's effectiveness are deleted or when competing or contradictory components are added. Staff members who were well grounded in behavioral theories for HIV prevention readily recognized when proposed innovations to EBIs limited implementation effectiveness but were unable to reject them. Although communication among technology transfer stakeholders occurred, this communication presented planning difficulties for the CBOs. The local prevention priority to deliver multisession interventions led some funders to require that organizations append new content to the interventions as part of contract compliance.

A similar compliance issue arose when organizations were required to serve individuals with and without HIV infections with the same interventions. To minimize unintended reinvention of these EBIs, recommended modifications must be justified by their potential to enhance the external validity and client relevance of the intervention. To further facilitate implementation, early technical assistance in this phase should be provided to CBOs on planning for recruitment and retention of participants and staff in the EBIs.

The maintenance and evolution of EBIs depend on planning to develop and retain a pool of qualified HIV prevention staff, as well as to ensure the fiscal and operational viability of the interventions. Participants described their needs to strengthen and enhance staff capacity to deliver EBIs and their focused efforts to meet these needs. Improvement was needed in planning to conduct outcome monitoring and cost evaluation, activities that were recommended for adapted EBIs.¹ Evaluation activities mentioned in the interviews largely reflected process monitoring and contract compliance practices. Technical assistance on outcome monitoring and cost analyses must be given higher priority for adapted EBIs in particular as the next cycle of prevention services is funded.

Failure to address these gaps in technology transfer may lead to implementation of programs that are incomplete or are inconsistent with the intended goals of the intervention or the needs of the target audience. CBOs that were early adopters of EBIs and found them problematic to use because of lack of information or guidance might discontinue their use and discourage others from adopting EBIs. Future diffusion of EBIs might be met with skepticism or distrust. Sustaining these interventions in real-world settings requires addressing the identified shortcomings of existing dissemination and implementation efforts.²⁰

Our study had several limitations. Not all of the staff who were implementing EBIs at their CBOs were interviewed. Selection bias in the staff who were interviewed may have led to a skewed picture of EBI implementation. Social desirability may also have been operating, and the results may not accurately represent how EBIs were conducted at these agencies. Future studies that include observational methods, document review, and multiple interviews might lead to a more complete understanding of the use of EBIs.

The study design did not include a comparison group, nor did it include any pre- or posttest measures to assess organizational capacity to select, implement, or evaluate HIV prevention EBIs. Thus, no causal inferences can be drawn from the data. Nevertheless, our findings offer insights into the progress and pitfalls of HIV prevention EBIs conducted in the real world. Anticipating the needs of CBOs that are adopting these interventions could help to optimize future diffusion efforts.