Impact of Human Resources on Implementing an Evidence-based HIV Prevention Intervention

Abstract

Evidence-based interventions (EBIs) often require competent staff, or human resources (HR), for implementation. The empirical evidence characterizing the influence of HR fluctuations on EBI delivery is limited and conflicting. Using the Interactive Systems Framework, we explored staff fluctuation and the subsequent influence on RESPECT, an HIV prevention EBI.

Methods

We conducted interviews with staff in two waves (n=53, Wave I; n=37, Wave II) in a national sample of organizations delivering RESPECT (N=29). We analyzed interviews qualitatively to describe changes among RESPECT staff and explore the subsequent influences on RESPECT implementation.

Results

Organizations reported downsizing, turnover, and expansion of staff positions. Staff changes had multiple influences on RESPECT implementation including clients reached, fidelity to specific RESPECT protocols, and overall sustainability of RESPECT over time.

Discussion

HR fluctuations are common, and our analyses provide an initial characterization of the relationship between HR fluctuation and EBI implementation. Given the prominent influence of HR on EBI implementation, the Interactive Systems Framework is a useful guiding tool for future examinations.

The HIV incidence rate in the US has remained static (1) despite a variety of available interventions designed to reduce the risk of HIV transmission. The use of evidence-based interventions (EBIs) has become integral to national HIV prevention strategies to reduce the incidence of HIV (2, 3). EBIs are valuable because they demonstrate efficacious outcomes in research settings. However, community-based organizations (CBOs) and public health departments (DPHs) frequently face challenges implementing EBIs, which may undermine program effectiveness (3). Among the array of influences, human resources are hypothesized to have a substantial impact on EBI implementation (4). Recent efforts to assess general trends in the public health workforce suggest its characteristics are changing (e.g., decreases in the number of state public health workers and overall changes in the composition of the public health workforce), raising concern about the relationship between the workforce and the quality of public health service delivery (5, 6). Despite theoretical support that human resources influence EBI delivery and concern that changes in the public health workforce are influencing service delivery, the empirical evidence examining the relationship between human resources and EBI delivery is limited and conflicting.

Human Resources

Although a focus on human resources in public health is relatively new, other disciplines, particularly business management, provide a wealth of research on the topic. Human resources is an organizational-level concept that describes the staff needed to provide a service (7, 8). An organization must have sufficient numbers of employees with appropriate skill to meet client demand for a service and provide the service with quality (7).

Changes and fluctuation in human resources are common and are conceptualized in three primary ways: turnover, downsizing, and expansion (7). Staff turnover is when a staff member leaves an organization, either voluntary or involuntarily, and the organization is required to hire a new staff member to fill the vacant position (7). Reviews show the rate of turnover and the jobroles of lost staff influence organization-level performance (8). Within organizations where the rate of turnover is low, the effects of turnover are greater, suggesting that in these organizations staff members develop specialized skills and knowledge that may be difficult to replace (7). High rates of staff turnover are also considered an undesirable outcome because of the substantial investment required to replace key personnel (9). Turnover in supervisory positions has a greater influence on organizational performance than turnover among service positions (10). This may be due to the fact that supervisors directly influence the oversight and quality assurance that can be conducted, and indirectly impact the quality of working conditions for the remaining staff (7).

Downsizing reflects instances where a number of employees are dismissed because the service they provide is discontinued. This may be the most detrimental form of change in human resources (8). Staff are dismissed because there is less demand for a service; workers leave, taking their expertise with them; then the quality of services diminish, further hurting client demand. Thus, a cycle of loss and decreased demand is created. In contrast, expansion reflects instances where a number of personnel are hired to fill new positions within an organization to provide new services or expand existing services. While this may be generally beneficial for organizations, there may be a lag before the benefits are realized, because newly hired staff members require time to develop proficiency and efficiency in their job roles (7).

The Interactive Systems Framework

We conceptualize the relationship between human resources and EBI implementation in public health using the Interactive Systems Framework for Dissemination and Implementation (ISF) as a guide. The ISF describes the public health system in which EBIs are disseminated and implemented, including the Prevention Delivery System, the Prevention Support System, and the Prevention Synthesis and Translation System (4, 11, 12). In the present study, we focus on the Prevention Delivery System, the public health system in which EBIs are implemented and provided directly to communities (See References 11 and 12 for more detailed descriptions of other systems described in the ISF). The Prevention Delivery System includes multiple influences on EBI implementation at individual, organizational, and community levels. The ISF offers several advantages for examining human resources and EBI delivery. Several researchers have noted the lack of theory to guide examinations of human resources and EBI implementation (13). Unlike many other frameworks within dissemination and implementation science, the ISF describes the relationship between human resources and EBI delivery, making it a useful tool for guiding the current examination.

Individual and Organizational Capacity

According to the ISF, the staffing (i.e., human resources) needed to adequately deliver EBIs with quality, is conceptualized as one aspect of organizational capacity. Organizational capacity is the human, financial and physical resources necessary for an organization to deliver a service (4). At the individual staff level, capacity may be delineated along two dimensions: innovation-specific capacities and general capacities. Innovation-specific capacities include specific knowledge and skills regarding a new EBI and the underlying health problem, the perceived ability to implement the EBI, and attitudes toward the EBI. General capacities are general skills and attitudes that support a staff member’s success in a position (4). These dimensions of individual staff capacity are considered the most proximal influences on the quality of the program delivery to clients. The accumulation of individual capacities gives rise to capacity at the organization level. Organizations with greater numbers of staff or higher quality staff have a greater capacity to deliver services. However, there are also distinct organizational general capacities (e.g., organizational structure and climate), and innovation-specific capacities (e.g., organizational support for an EBI, ability to evaluate EBI implementation/efficacy).

Human Resources and EBI implementation in Public Health

Although some fluctuation in human resources is normal within public health organizations (14), understanding the influence of downsizing and expansion may be especially important in light of concerns that the size and composition of the public health workforce are changing (5, 6). For example, a recent study designed to enumerate and characterize the public health workforce indicates that the size of the public health workforce declined an estimated 4% from 2010 to 2012, with the greatest declines among public information specialists (33% decline), public health informatics specialists (29% decline), and behavioral health professionals (20% decline). However, some specialties saw increases in the number of staff, including public health managers (11% increase), nutritionists (7% increase), and environmental health workers (3% increase; 6)

The empirical evidence characterizing the relationship between human resources (i.e., the workforce) and EBI delivery is limited and conflicting. Implementation studies often focus on staff turnover (i.e., the loss of staff that must be replaced). Similar to the evidence in the business management literature, several studies suggest that turnover, especially unexpected or high turnover, negatively effects EBI implementation. Turnover especially effects the quality of EBI implementation in aspects of fidelity, or the extent to which EBIs are delivered as designed (15–18); and reach, or the volume of clients who receive the EBI (19); and EBI maintenance/sustainability, or the extent which EBIs are provided consistently over time (19). However, a handful of studies found that turnover had either no effect or a positive effect on EBI fidelity, and subsequently, client outcomes (20, 21). Very few studies have focused on downsizing or expansion among human resources and the subsequent influence for EBI implementation, although some have noted the need to explore these issues in the context of EBI delivery (16, 8). As such, continued research on staff changes in organizations delivering EBIs is needed, and this work should move beyond the focus on staff turnover to examine downsizing and expansion of staff and the impact of these changes on EBI delivery.

The Current Research

The current qualitative study draws on the concepts of individual and organizational capacity, as described in the ISF, to examine the influence of human resources fluctuations on the implementation of RESPECT, an EBI widely disseminated and implemented through the Diffusion of Evidence-Based Intervention program (DEBI). The Centers for Disease Control and Prevention (CDC) established the DEBI program in 2002 to promote adoption of designated evidence-based behavioral HIV prevention programs by public health organizations (22, 23). The RESPECT program was one of the most widely adopted DEBI programs (24). Many DEBI interventions have been subsumed under the High Impact Prevention Initiative, which promotes multiple biomedical, social, and behavioral interventions (2).

RESPECT was included as a DEBI intervention because it demonstrates efficacy in reducing risk behaviors and sexually transmitted infection (STI) rates (25, 26). The program is a clinic-based counseling and testing intervention targeting at-risk individuals seeking testing for HIV or other STIs. Counselors work through a series of several key components with clients during counseling sessions where clients actively identify risk behaviors and possible solutions for risk reduction, which are among the mechanisms thought to make the program efficacious (25, 26). If clients indicate only a minimal risk behaviors for HIV/STIs, counselors are encouraged to provide an abbreviated version of the program, where the clients do not engage in steps to reduce risk behaviors. If clients indicate high-risk behaviors, counselors are encouraged to provide additional steps in the program, where clients develop a risk-reduction plan and attempt to carry out steps in this plan. RESPECT was originally conducted in two counseling sessions, but has been modified to also be conducted in a one-session format to accommodate the use of rapid HIV testing (27).

RESPECT relies on specific staff training before implementation and requires additional staff for supervision, monitoring and evaluation (26). Moreover, RESPECT is implemented with greater success if staff members possess basic counseling skills (28, 29). As such, staff members delivering RESPECT require both innovation-specific and general skills. Supervisors are intended to help contribute to the fidelity of the program across providers by helping individual providers deliver the program appropriately. Several studies suggest that staff fluctuations, especially turnover, have a significant impact on organizations’ ability to implement RESPECT (30–32). Consequently, this program provides an appropriate context for examining the influence of organizational capacity, especially human resources, on EBI implementation.

First, we identified patterns of human resource fluctuation in both the number of staff (i.e., turnover, expansion and/or downsizing) and positions (i.e., providers and/or supervisors positions) within CBOs and DPHs delivering RESPECT nationally. Second, we investigated how these human resource fluctuations influenced implementation of RESPECT and provided rich descriptions of how these influences unfolded in the context of an organizational setting.

Method

We collected data for the current qualitative paper as part of a longitudinal mixed-methods investigation, the Translation into Practice (TIP) study, a national survey of CBOs and DPHs delivering RESPECT (29, 33, 34). Institutional review boards at Westat Corporation and Oregon State University approved all protocols.

Recruitment & Sampling

Organizations

A more detailed description of sampling and procedures is presented elsewhere (see references 33 & 34). Briefly, we collected data in two waves, approximately one year apart beginning in the fall of 2010. Thirty organizations were enrolled for participation using a quota sampling frame to fill 4 cells: urban DPH (N=7), urban CBO (N=10), nonurban DPH (N=7), and nonurban CBO (N=6). The present analyses are limited to 29 organizations with at least two staff members dedicated to RESPECT. Agency characteristics are provided in Table 1.

Table 1.

Agency Characteristics

	Wave I n (%)	Wave II n (%)
Agency Type
DPH	14 (48)	11 (48)
CBO	15 (52)	12 (52)
Catchment Area
Urban	13 (45)	11 (48)
Non Urban	11 (38)	7 (30)
Both Urban and Non Urban	5 (17)	5 (22)
RESPECT sites
One Site	13 (45)	12 (52)
Multiple Sites	16 (55)	11 (48)
Total	29 (100)	23 (100)

Organization Staff

At the time of organization recruitment, we recruited executive directors for participation. Executive directors enumerated staff, including staff responsible for providing RESPECT directly to clients (i.e., RESPECT providers) and staff responsible for supervising RESPECT providers (i.e., RESPECT supervisors). A maximum of two supervisors were interviewed at each organization. In organizations with more than two supervisors, two supervisors were randomly selected for interview. A maximum of four providers were interviewed at each organization. In organizations with more than four providers, four providers were randomly selected for interview. We made efforts to recruit the same participants across waves, and one new supervisor was recruited in Wave II. Staff characteristics are provided in Table 2.

Table 2.

Participant Characteristics

	Wave I	Wave II
Years at Agency -(M)	9.3	10.6
Age -(M)	43	43
Gender -n (%)
Male	23 (43)	17 (46)
Female	30 (57)	20 (54)
Ethnicity -n (%)
White	32 (60)	21 (57)
Black	6 (11)	5 (14)
Hispanic	9 (17)	7 (43)
Mixed	5 (9)	4 (11)
American Indian	1 (2)	1 (3)
Education -n (%)
High School	5 (9)	2 (5)
Associates Degree	7 (13)	4 (11)
Bachelor's Degree	21 (40)	16 (43)
Master's Degree	20 (38)	15 (41)
Total -n (%)	53 (100)	37 (100)

Data Collection

Interviews

We conducted semi-structured telephone interviews with participants at each wave of data collection. Interviews lasted 45–60 minutes. Participants provided consent at the time of the interview, and received a $25 gift card for participation for each interview. Interviews were audio-recorded, transcribed, and checked for accuracy.

Interviews focused broadly on adoption and implementation of RESPECT. We tailored interviews to the position of the staff member. For the present paper, we focused on specific questions related to general changes and specific staff changes within the organization (e.g. “Since you first began implementing the RESPECT program, have you experienced changes in the number of individuals providing program delivery, or conducting supervision?”). We used follow-up probes to elicit information about the kinds of staff changes that may have occurred among RESPECT providers and supervisors.

Client surveys

We collected client exit surveys in both waves, asking clients to complete surveys after the conclusion of their first RESPECT counseling session. We asked about general demographic information, current risk behavior, and events occurring within the RESPECT counseling session. Surveys elicited information from the client about receiving the key elements of RESPECT (See reference #33 for a more detailed description of data collection with clients and measures of RESPECT fidelity). No incentives were provided.

Data Management & Analysis

We used NVivo 8.0 to code interview data. We carried out the analysis plan in several stages. Initially, the first author conducted a directed content analysis to extract discussions of staff changes and the context under which staff changes influenced program implementation. We then analyzed the data using three different coding techniques: categorical coding, thematic coding, and case study analyses. We used each of these techniques to provide complementary insight into the kinds of staff changes that occur within organizations and how these changes influenced RESPECT implementation. Categorical coding helped to provide insight into the kinds of human resource changes that organizations experienced related to RESEPECT. We then analyzed data thematically to explore potential relationships between human resource changes and RESPECT implementation. Finally, we selected two exemplary cases demonstrating many of the different phenomena that emerged in these analyses. Multi-case comparisons are ideal for providing rich descriptions that illuminate the complexity of phenomena, especially to answer “how” and “why” certain phenomena occur (35). These cases provided a rich description of the staffing events that occurred over the course of RESPECT implementation, the impact that these changes had on the organization environment, and the implication of the changes for program outcomes. A more detailed description of each analysis technique is provided below.

We used a subset of participants for the categorical and thematic coding: executive directors and RESPECT supervisors. Although RESPECT providers were interviewed for the larger TIP study, for theoretical and practical reasons, we used data from administrative staff (executive directors and supervisors) for the main analyses. First, the ISF suggests that organizations experience multiple effects due to staff changes and these effects may be most evident to administrative staff. Several of the organizations enrolled in the study had many staff at multiple sites, but provider staff only knew about conditions at their own sites. Further, many RESPECT providers were not able to adequately articulate how staff may have changed over time because staff changes often occurred before the participant was hired (i.e., the program had been in place for several years before the staff member had been hired). Therefore, we did not use providers’ responses for the categorical or thematic analysis. However, we did utilize responses from RESPECT providers in the case study analyses (see below).

Categorical Coding

Individual interviews with executive directors and supervisors from both Wave I and Wave II were coded categorically to identify the kinds of staff changes that occurred in the study organizations. We developed and coded interview data for human resource changes using two schemes: a) staff change type and b) position change type. Categorical codes of staff changes were defined a priori (See Table 3 for categories and definitions), and were limited to RESPECT providers and supervisors. Staff change type captured how numbers of staff members changed in the organization (i.e., downsizing, turnover, expansion, or no changes). Position change type captured the role that the staff member served (i.e., supervisor or provider).

Table 3.

Categorical Coding Schemes

Scheme	Definition
Staff Change Type *
No Change	No staff positions or hours changed.
Expansion	Staff positions or hours added working with RESPECT.
Downsizing	Staff positions or hours reduced working with RESPECT.
Turnover	Loss of a staff member which the organization subsequently replaced, or a position was currently vacant that the organization intended to refill.
Position Change Type†
Provider	A staff member involved in providing RESPECT directly to clients.
Supervisor	A staff member involved in providing supervision for providers.

^*Agencies may be double coded.

^†In cases were there were a change occurred among a staff member that acted both as a supervisor and provider, the change was coded as a superivosr position.

We used reports from individual participants within each organization to create a categorization at the organization level. All types of changes that were reported are included in the organization-level coding. We coded organizations to multiple categories if reports differed among individuals within an organization or if individuals reported multiple types of change. Given that participants operated at multiple levels within organizations and at different organization sites, multiple coding allowed the authors to capture more complex staff changes that occurred within organizations.

Thematic Coding

All authors reviewed data from content analysis and categorical coding to develop a thematic coding scheme. Various themes and interpretations were discussed until there was consensus among all team members. Detailed descriptions of each theme are included in the results. Data could be coded to multiple themes if participants reported multiple outcomes as a result of reported staff changes in organizations. Ten cases (n=5 from each wave) were independently coded by an independent second coder to check for reliability in application of categorical and thematic codes and interpretation of data. A consensus approach was used to resolve any disagreements between coders.

Case Study Narratives

Because loss was commonly reported among organizations (See Results section), we selected a case that exhibited clear downsizing between Wave I and Wave II. We then selected a contrasting case where an organization exhibited clear expansion. In developing these cases, we used all available data for the organization including executive director, RESPECT supervisor, and RESPECT provider interviews. Perspectives of multiple individuals from each case provided additional insight on their self-assessment of skill in delivering RESPECT, the influence of human resources changes on RESPECT delivery, and other valuable contextual aspects of implementing RESPECT within organizations.

We also report an organization-level index of fidelity to RESPECT (i.e., the extent to which the program was delivered as intended). We used client exit surveys (for case I: n=59 at Wave I and n=31 at Wave II; for case II: n=38 at Wave I and n=32 at Wave II) to generate an index of fidelity to the RESPECT program at the organization level (0%-100%). Briefly, we scored surveys on a scale reflecting whether or not the client received the appropriate RESPECT program components given the risk characteristics of the client. High scores reflect good fidelity. We aggregated individual scores to obtain an organization-level measure of fidelity (see reference #34 for a more thorough discussion). Together, these data provide a picture of what transpired within the organization over the course of the study. Pseudonyms have been given to participants to ensure confidentiality.

Results

Most organizations experienced changes among staff dedicated to RESPECT following the initial implementation the program, and reports were similar among CBOs and DPHs. As shown in Table 4, downsizing and turnover were the most commonly reported changes. Many organizations also reported multiple types of staff change, often experiencing staff turnover in conjunction with either downsizing or expansion. Changes among provider positions were more common than changes among supervisor positions.

Table 4.

Agencies Reporting Staff Changes by Position and Type

	Wave I	Wave II
	n (%)	n (%)
Any Change	20 (69)	17 (74)
Multiple Changes	10 (34)	8 (33)
Provider
Downsize	10 (34)	9 (39)
Expansion	7 (24)	5 (22)
Turnover	10 (34)	7 (30)
Supervisor
Downsize	3 (10)	6 (26)
Expansion	5 (17)	0 (0)
Turnover	3 (10)	4 (17)

The Impact of Staff Changes on Implementation of RESPECT

When discussing the impact that fluctuations among staff had on RESPECT delivery, executive directors and supervisors reported influences on RESPECT implementation, on the remaining staff at the organizations, and on the organization overall. Four primary themes were identified regarding the impact of human resource fluctuations on RESPECT implementation: 1) change in the number of clients served and program maintenance, 2) change in skills and knowledge among remaining employees, 3) change in workload for remaining employees, and 4) innovative approaches to downsizing. Detailed descriptions of each theme are provided below, and supporting quotes from executive directors and RESPECT supervisors are presented in Table 5.

Table 5.

Human Resource Change Influence on RESPECT Implementation

Number	Theme	Supporting Quote
1.1	Changes in Clients Served and	I would say that it’s a reduction in how many clients we can reach with this service… (Agency A; Provider Downsize).
1.2	Program Maintenance	RESPECT is part of our counseling and testing program. Total numbers have gone from about 2,300 to about 1,700 tests every year. So you can see that we’ve lost some actual interface unfortunately, since I’ve lost about half of my staff… (Agency B; Supervisor Downsize and Turnover; Provider Downsize).
1.3		We hired specifically because we needed an increase in staff, and we also got the funding. So I think the client load was already there. We were just really struggling to meet [the client load]… (Agency C; Provider Expansion).
1.4		The old full-time clinician relocated, so we had to hire a new staff member, and it was slow because the training…she was here four months before she was actually trained for RESPECT. Then another staff member came on, and we decided to identify her as a backup person…so that we wouldn’t be stuck with nobody being able to provide the service (Agency M; Provider Turnover and Expansion).
2.1	Changes in Skill and Knowledge	Some of those who left had a whole bunch of experience, and then you have to train somebody new. So that’s been a challenge (Agency D; Provider Downsize and Turnover).
2.2		And losing him…he was one of the first test counselors [we] had and he’s got a lot of years experience, so him leaving definitely put us in a different place as far as problem solving. But I don’t think it really created a problem because we still have [another counselor] and myself here who are long-time test counselors, and many of our original test counselors are still with us (Agency E; Supervisor Turnover).
2.3		We have a very low turnover [such] that all those people that started with us [at] implementation, they’re still with us, so they require less observation and less supervision (Agency F; Supervisor Expansion; Provider Expansion and Downsize).
2.4		We did increase the [supervision] to monthly within the last year, just because of the complexity of the addition of new staff… and the complexity of record keeping and requirements. We wanted more frequent review (Agency D; Provider Downsize and Turnover).
3.1	Changes in Workload for Remaining	A staff decrease means that the ones of us left are having to carry what they did, and so it just adds on to our workload… (Agency H; Supervisor Downsize; PD Downsize).
3.2	Employees	Well, with the one supervisor leaving, that put the load on to the other. We had a co-manager scenario, and so now it’s just one manager for the program. And our [RESPECT] counselors have a tremendous workload because the number of cases have gone up, and we’re not able to hire new staff…with the staffing changes the health educator and myself are spending more time in the clinic doing low-risk testing as much as we can…before we were hardly in the clinic at all (Agency J; Supervisor Downsize).
3.3		My workload is changing a lot just because…we recently hired some other folks who took HIV training, so that will help eliminate some of the burden (Agency I, Provider Expansion and Turnover).
4.1	Innovative Approaches to Downsizing	Since we’ve implemented RESPECT, we actually lost a person and a half. That’s how we got a volunteer though (Agency A; Provider Turnover).
4.2		We’ve got fewer staff, but we’re forming more collaborative partnerships, so we’re going to be able to reach a broader spectrum of the higher-risk population than we have in the last couple of years (Agency A; Supervisor Downsize; Provider Downsize).

Changes in the Clients Served and Program Maintenance

The most commonly reported impact was on the number of clients who received RESPECT. As noted above, it was more common among organizations to experience either turnover or downsizing among staff either as a reduction of the number of positions or staff hours. As a result of turnover and downsizing, staff members from the majority of organizations reported conducting RESPECT with fewer clients (Table 5, 1.1). Although downsizing typically resulted in fewer RESPECT providers, sometimes organizations experienced dramatic downsizing such that both supervisor and provider positions were eliminated (Table 5, 1.2). In the example of Agency B, downsizing resulted in several hundred fewer clients receiving RESPECT. Among organizations that had sufficient resources, several hired additional staff in order to accommodate client demand for testing and counseling (Table 5, 1.3). In the example of Agency C, adding staff extended delivery of RESPECT to additional clients.

Organizations had to temporarily suspend RESPECT delivery on occasion due to downsizing or turnover, especially when just a few staff members were dedicated to the program. Even when organizations were able to replace staff, resuming RESPECT implementation was sometimes delayed because of time lags between hiring new staff and provision of appropriate training (Table 5, 1.4).

Changes in Skill and Knowledge

Respondents reported change in skill and knowledge within the organization as an outcome of staff fluctuations. Downsizing or turnover among staff translated into loss of skill, knowledge, and/or experience (Table 5, 2.1). Many participants described retention of longtime staff as beneficial for RESPECT because of the continuity in skill and knowledge and their ability to problem-solve (Table 5, 2.2). Furthermore, organizations that experienced continuity in their staffing felt that longtime staff members required less supervision and monitoring (Table 5, 2.3). In contrast, new employees required initial training and more intensive supervision to ensure that they understood the appropriate RESPECT procedures (Table 5, 2.4).

Changes in Workload for Remaining Employees

Staff members who remained at organizations experienced changes to their workload and job roles as a function of broader changes at organizations. When organizations experienced downsizing or turnover, remaining staff often experienced increased workloads (Table 5, 3.1). This appeared to be particularly acute for supervisors, who often took on additional supervising responsibilities or were required to add delivery of RESPECT to their supervisory workload (Table 5, 3.2). When organizations expanded RESPECT, existing staff experience a reduction in their workload (Table 5, 3.3).

Innovative Approaches to Downsizing

While most organizations attempted to cope with downsizing loss by either increasing workloads for staff or reducing the number of clients served, one organization was able to expand RESPECT delivery in the face of loss. At Wave I, this organization initiated a volunteer component to the program when their funds were insufficient to support paid staff (Table 5, 4.1). No other organization in our study actively made the decision to replace paid staff with voluntary staff, although some other organizations used volunteers in their programs. At Wave II, the same organization, which continued to experience reductions in paid staff, expanded their strategy by building partnerships with existing organizations to recruit at-risk participants with greater ease (Table 5, 4.1). The ability of this organization to reach more clients with RESPECT in the face of downsizing by collaborating with other organizations was highly divergent from other organizations.

Case Studies

Staff fluctuations in organizations led to changes in organizational capacity; some changes were positive and some negative. Often, organizations experienced changes in the numbers of skilled staff dedicated to RESPECT over the course of a single year, experiencing both positive and negative influences in relatively short periods of time and in dynamic ways. The two organizations described in the case studies that follow offer a more nuanced view of the impact of staff fluctuations on organization staff, on ability to meet the needs of clients, and on program fidelity.

Case I

Case I was a CBO dedicated entirely to providing HIV-related services, focusing primarily on African Americans, men who have sex with men, and youth. Three staff members, two full-time and one part-time, provided general HIV prevention and education, HIV testing and RESPECT. The organization had been conducting RESPECT for approximately one year at the time of the Wave I interviews and had stable staffing at that point.

The organization exhibited good fidelity to the program, conducting RESPECT correctly with 69% of clients. At the time of the Wave I interviews, the organization had not experienced any staff changes since initiating RESPECT. The executive director identified Doug, the prevention/education coordinator, as the key person for solving RESPECT-related problems in the organization. He received training in RESPECT from a regional training center in the spring of 2010. Regional training centers were designated by the CDC as the official training centers for RESPECT. Thus, Doug received the highest quality training available. Doug subsequently provided in-house training and supervision for all other staff members. He delivered RESPECT skillfully to a variety of clients, consciously making an effort to fully deliver the primary elements of RESPECT with at-risk clients (i.e., with fidelity). When Doug was interviewed at Wave I, he alluded to the increasing demand on staff because of tightening budgets and downsizing in other programs:

Right now the organization is in a little bit of a fiscal crunch as most nonprofit, HIV-service organizations in the country are, and our education and testing staffing has been cut quite considerably. One of the reasons [I’m feeling pinched] is that I’m now also a case manager. So we’re relying on me, the executive director and floating staff to provide HIV/STD education and testing.

Downsizing in other programs required Doug to take on additional roles within the organization and increased his workload.

A year later (Wave II), there had been significant loss of staff. Only one staff member remained, Stacey, who was actively conducting RESPECT. The other two staff members had left the organization, including Doug, who resigned about halfway through the year. The fidelity score for the organization at Wave II dropped to 29%, a 40% decrease from the previous year. Stacey, who was also interviewed for the study, maintained her RESPECT client load at approximately 6 clients a week. When asked about her workload, Stacey replied, “(It) just added responsibilities because of the staff changes.… I was working a 50-hour week, and now it’s up to 60 hours....” Stacey discussed how these factors impacted her delivery of RESPECT:

[Interviewer]: And [your workload] changed because of the loss of [Doug]?

[Stacey]: Correct.

[Interviewer]: And how do you feel about those changes?

[Stacey]: You know…. I love the interaction, I love doing testing, but it’s certainly not what I should be spending the majority of my time on, so I usually pick the easy stuff rather than the difficult stuff—like most people.

Stacey implied that she would rather not conduct RESPECT when given a choice. She also struggled to adapt the program to certain clients, even though some adaptations would have saved time and volume of work. For example, Stacey reported that she delivered the full RESPECT program to low-risk clients to “keep them low risk,” although maintaining fidelity to the RESPECT protocol would entail providing an abbreviated version of RESPECT to low-risk clients. In another example, Stacey reported that if she was pressed for time, she would not provide the program at all, even if the client was considered at-risk and an appropriate candidate for the full program. Again, RESPECT protocol would entail providing all steps in the RESPECT program for at-risk clients in order to maintain fidelity to the program.

Unfortunately, Stacey lacked the support of co-workers that would have made her workload more reasonable and the supervision that potentially would have helped her deliver RESPECT with higher fidelity. In follow up questions, Stacey discussed that there was not adequate staff dedicated to education and prevention to provide RESPECT for the number of clients coming to the organization; however, Stacey was optimistic because the organization had just enrolled in new programs that would bring in funds for new staff. Regardless, downsizing at this organization was dramatic within a single year, and likewise had dramatic consequences for RESPECT in terms of the quality with which it was implemented.

Case II

Case II was also a CBO contracted by the state to conduct HIV testing, and their clients were primarily homeless and transient populations. The executive director and two staff members had been conducting RESPECT for approximately 12 months. Two RESPECT providers had been hired specifically to implement the program and provide more HIV testing services, representing an expansion for RESPECT and the organization. According to the executive director, Becky, the two new providers were hired because of a significant increase in the number of clients accessing services, and the organization needed additional staff to meet the client demand.

Becky, who also supervised the other staff members, participated in a more lengthy training program through the state than did the RESPECT providers, Steve and Amy. Rather, these two received an abridged, 3-day version of the training, also provided by the state. Based on Steve’s discussion of RESPECT, he demonstrated a thorough understanding of the program and the steps necessary to implement the program with fidelity, often encouraging clients to identify their own risk behaviors and build on personal strengths. Amy mentioned that she struggled to implement RESPECT appropriately when discussing how she provided the program. Similar to Stacey in Case 1, Amy often missed opportunities to abbreviate RESPECT with low-risk clients, save time, and minimize workload.

At the time of the Wave I interviews conducted in the late spring of 2010, the organization exhibited excellent fidelity, conducting RESPECT with fidelity to 88% of clients. RESPECT providers identified Becky as a key individual to the program. Becky, who was also interviewed for the study, expressed confidence in their organization’s ability to conduct RESPECT successfully:

… because of the records that we keep, and I always review them every month before they go to billing, I’m able to track and kind of follow whether [RESPECT is being provided with quality] or not. I also get to see the risk assessments that are done, I get to see the risk reduction plans that are made, and any case notes that are made along the way. So I get to look at those and see that they’re filled out, and what they told the client or what needs to be told to a client.

By routinely reviewing the records, Becky was able to identify successes and challenges in delivering the program.

At the time of the Wave II interviews, conducted in the fall of 2011, Becky had left the organization. Steve had assumed Becky’s supervisory role.

[Interviewer]: Any loss of a key person who was important in problem solving?

[Steve]: We lost Becky…she understood well about all this. She had a lot of experience on HIV prevention.

Steve ultimately assumed the role of supervisor, and noted the heavier workload in this new role. “When I’m extremely busy and somebody walks in…of course I try to do it, [but] I don’t sit with them and converse as much as I do when I’m not doing another program.” Again, the workload for Steve made conducting RESPECT a challenge, simply because of the limited time he had with clients and the numerous responsibilities he had to fulfill.

The organization fidelity score dropped to 52%, a 36% decrease from the previous year. Several new staff members had been hired within the organization, but only one was trained to conduct RESPECT. However, this new staff member was responsible for conducting case management and not delivering RESPECT. Similar to the first case, Steve was hopeful regarding the newly hired staff members, the skills that they brought to the organization, and potentially a more manageable workload:

[Steve]: My workload is changing a lot because we recently hired some other folks…

[Interviewer]: So that will help eliminate some of the burden.

[Steve]: And the person who we hired speaks Spanish and English. So we were [lucky] that person was in [another office] where we have a very large Hispanic population…so we’re excited for that.

[Interviewer]: You’re looking forward to a slightly more manageable workload in the near future?

[Steve]: Oh yeah, because as I may go for vacation they will be able to offer [RESPECT], and I’m sure that will be very good.

Discussion

The provision of EBIs is integral to ensuring communities receive high quality services. Successful implementation of EBIs, especially those focused on behavior change, is frequently contingent on adequate human resources. However, fluctuation in human resources within an organization is a normal occurrence. Consequently, it is important to understand the kinds of human resource fluctuation occurring within organizations and the extent to which changes in human resources influence the implementation of EBIs. Using the ISF, we identified patterns of human resource fluctuation (i.e., turnover, downsizing, and expansion) in numbers of staff providing services directly to clients and among staff providing supervision in public health organizations implementing an EBI. These exploratory analyses also point to a variety of ways these fluctuations influence EBI implementation and delivery.

ISF: Individual and Organizational Capacity

Our findings suggest that human resource changes play a dynamic role in EBI implementation, influencing EBI fidelity, reach, and/or maintenance over time. As the ISF suggests, organizations most successful at implementing RESPECT maintained individual staff with innovation-specific capacities (i.e., specific training in RESPECT) and general capacities (e.g., general experience or the ability to troubleshoot problems). Organizations with appropriate numbers of staff with the requisite skills (i.e., individual capacity) had greater organizational-level capacity to facilitate delivery of RESPECT (3,4).

In alignment with the existing literature and the ISF, public health organizations reported turnover, expansion, and downsizing among staff dedicated to RESPECT, and multiple ways these fluctuations impacted organizational capacity, which in turn influenced RESPECT implementation. Similar to other studies, the present thematic and case study analyses demonstrated how turnover often had multiple negative influences on organizational capacity and EBI implementation (e.g., reducing the number of clients that were delivered RESPECT or a loss of programmatic skills and knowledge; 4–9). Our focus on downsizing and expansion in addition to turnover extends the current literature and draws attention to the frequency and extent to which these fluctuations, especially downsizing, occur. Downsizing had similar negative, albeit more permanent influences similar to those of turnover. In contrast, expansion often had positive influences on EBI implementation (e.g., increased numbers of clients served, maintaining skills and knowledge), although the benefits were delayed when training for new employees was not immediately available.

Furthermore, the present analyses suggest that the influence of fluctuations on EBI implementation may differ depending on the kinds of human resource fluctuations that take place (i.e., turnover, expansion, or downsizing), the individual capacities that staff may bring or take with them, and how these changes contribute to the overall capacity within an organization to provide EBI. For example, the case studies demonstrate the impact of downsizing on fidelity is dependent on whether the organization retains staff with requisite skills (i.e., individual capacity). If downsizing leads to an absence of essential skills required for the program, there may be reductions in fidelity, but there may be no impact on fidelity if the remaining individual staff members possess the necessary skills and continue to provide the EBI with fidelity. Downsizing also may lead to a reduction in EBI reach since fewer clients can be served with fewer overall numbers of delivery staff members (i.e., organizational capacity), regardless of their skill level. In this way, our study sheds light on the contradictory findings in the extant literature regarding the influence of human resource changes on EBI implementation.

Implications for Policy and Practice

The present study may inform policy designed to promote EBIs. Similar to most behavioral interventions, RESPECT was dependent on individual staff for successful delivery to clients, highlighting the importance of human resources in EBI implementation. Implementation models, including the ISF, (3,4) emphasize the fundamental importance of having the requisite number of staff with appropriate skills. Supervisors are especially valuable to conduct quality assurance and ensure high quality EBI implementation. However, capacity building efforts have focused largely on developing skills among existing staff by providing training and technical assistance (36). It may be beneficial for supporting organizations (e.g., state-level agencies, CDC) to focus on both training and staff retention in their work with agencies adopting and implementing EBIs. Our findings also suggest that downsizing may create a poor working environment (i.e., organizational capacity) that can lead to burnout and additional turnover among remaining staff, further reducing capacity and the quality of EBI implementation. As others have noted, capacity building efforts would benefit from a focus on creating a more stable and consistent workforce, such as organization-level interventions designed to improve retention and the working environment (37, 38).

Future Directions

Because of the complex role work force changes may play with regard to EBI implementation, it is critical to obtain a better understanding of the relationships demonstrated in these exploratory analyses. In this study we characterized the relationship of human resources to EBI implementation to help clarify inconsistencies in the literature, especially in regard to the influence of turnover on EBI implementation. Additionally, little attention has been paid to the impact of organizational growth and downsizing on program implementation, despite good evidence that the public health work force is changing (6), and funding for public health programs also varies over time. We provide some initial insight into the relationships between expansion or downsizing and EBI implementation. However, additional research guided by theory is necessary to replicate and extend our findings.

The ISF provides a useful guide for understanding how human resources may influence EBI implementation differently depending on both independent variables (e.g., expansion, downsizing, or turnover) and dependent variables (EBI fidelity, reach, or maintenance). In this study, we were limited to an examination of the human resources aspect of the Prevention Delivery System at the individual and organizational levels, with a primary focus on how human resources influence EBI implementation. However, the ISF suggests complex and bi-directional effects between the organization, the individual worker, and EBI implementation. The case studies described earlier exemplify some of the potential bi-directional effects that may be taking place; particularly how staff loss can derail quality assurance. Without good quality assurance, staff members may not maintain essential skills (i.e., individual capacity) or deliver the EBI with fidelity (39), undermining the value of EBIs for clients that receive them. In the context of organizational capacity, this is an instance where reduction in individual-level capacities feeds back to negatively impact organization-level capacities (e.g., supervisor time), and, consequently, organization programs and the capacity of other individual staff. Additional research further exploring these and other components of the ISF may illuminate larger programmatic and system consequences.

Lastly, we were not able to examine factors that led to many of the human resource fluctuations observed. While incorporating factors that contribute to human resource fluctuations is beyond the scope of this particular analysis, there is a small but growing body of literature beginning to address this important topic. These studies suggest that human resources changes occur for a variety of reasons. For example, shifts in policies and financial support for services (40), overall social and economic trends (33), or retirement within an aging workforce (6) may all contribute to the human resource fluctuations observed here. Continued research addressing the impact of larger systemic factors on human resources will further our understanding of, and ability to address these issues in the context of evidence-based public health programs.

Limitations

The present study has several limitations. Our findings suggest complex relationships between human resources and EBI implementation using the ISF, but we were not able to examine all possible relationships. However, we offer a number of hypotheses that may be tested in other samples using different research designs. Methodological limitations should also be considered. Participants may have been subject to reporting bias. For example, participants may not have been able to accurately recall all instances of staff changes or the influences staff changes had on RESPECT delivery. Finally, because of the quota sampling research design used in this study, organizations may not be representative of all public health organizations delivering RESPECT.

Conclusions

Adequate human resources are a central element in the delivery of EBIs and likely have varied influence on the quality of EBI implementation, depending on the kinds of changes that are occurring and the outcome of interest. The present study extends the current literature by focusing on the influence of downsizing and expansion in addition to turnover, and on the influence of these changes on EBI implementation. The ISF provides a useful guide for examining these relationships and guiding future studies in this area. Organizations that have capacity to train, support, and retain staff will likely have greater success offering high quality EBIs, especially behaviorally-based EBIs, over time.