Implementing a brief evidence-based HIV intervention: a mixed methods examination of compliance fidelity

ABSTRACT

Dissemination of HIV behavioral prevention programs has increased the reach of evidence-based interventions, but there is a paucity of data on implementation and diffusion. The present mixed methods study focused on RESPECT, a brief counseling and testing intervention, examining compliance fidelity and the extent to which Centers for Disease Control and Prevention (CDC) policies and training have diffused to practice settings. Using client exit surveys (N = 830) and counselor interviews (N = 64), we examined implementation in 26 community-based agencies (CBOs) and public health departments (DPHs) in the USA. Multivariate analyses showed that at-risk clients, ethnic minority clients, and those who were primarily seeking services other than HIV/STI testing, were more likely to receive the program with fidelity. Counselor data suggested that multiple factors (e.g., client characteristics, agency structure) impact program adaptations. RESPECT is being delivered with good fidelity and reaching at-risk clients. The data provide support for CDC diffusion efforts. Future studies should continue to examine compliance fidelity and program sustainability.

Through the Centers for Disease Control and Prevention (CDC) Diffusion of Evidence-Based Interventions (DEBI) effort, hundreds of departments of public health (DPHs) and community-based organizations (CBOs) in the USA are implementing behavioral HIV prevention programs [1]. RESPECT, an individual-level intervention, has been widely adopted. This program has broad appeal because of its brevity [2], its adaptability to the HIV/STI testing context, and because it can be tailored to individual clients. However, it is not known how well the program has been implemented in practice settings. The present mixed methods study examines compliance fidelity in a national sample of CBOs and DPHs implementing RESPECT.

RESPECT was originally tested in a randomized, controlled trial conducted in public STI clinics and was designed to reach at-risk populations [3, 4]. RESPECT [5, 6] has been delivered in a range of settings [3, 4, 7–9], and the two-session program has been adapted to a single-session format to accommodate rapid HIV testing technology. Given the widespread adoption of RESPECT, it is critical that studies address translation of this program, including issues of fidelity and diffusion. We turn first to the question of program delivery by focusing on compliance fidelity. We then address the broader issue of diffusion.

DELIVERING PROGRAMS IN PRACTICE SETTINGS: COMPLIANCE FIDELITY

Translation research has been guided by systems-based approaches that consider how factors at multiple levels impact implementation, including national-, agency-, and client-level factors [e.g., 10, 11]. These approaches recognize that some program adaptations may be necessary to achieve community and agency fit and to sustain program fidelity [10, 12–16]. However, eliminating or substantially changing core program components will diminish program efficacy [12, 15, 17]. Compliance fidelity reflects the extent to which the core program components are delivered [16] and is fundamental to implementation. The core program components for RESPECT include assessment of actual and perceived risk, recognition of the barriers that inhibit risk reduction, negotiation of an achievable plan to reduce risk, and support of client-initiated behavior change [4].

Program efficacy is also dependent on the extent to which programs reach the appropriate populations [12, 15, 17, 18]. In this regard, both risk factors, those factors which are part of the causal chain leading to an elevated frequency of the disease (e.g., sexual behavior), and risk markers, which serve as proxies for risk factors (e.g., sexual orientation), are important to consider [19]. The CDC provides policy and training that direct agencies to deliver RESPECT primarily to at-risk clients; epidemiological markers (e.g., men who have sex with men (MSM), injection drug users (IDU)) and behavioral indicators (e.g., repeat testing) are common guides for identifying at-risk individuals. Agencies, in turn, expect counselors to decide whether a given client should receive the full RESPECT program. However, counselors often make decisions about which clients are at risk using relatively limited information. Ideally, screening should strive to achieve maximum coverage of the at-risk population, although some low-risk individuals may receive the program because screening is not perfect.

Thus, in the case of RESPECT, compliance fidelity must consider both (a) the risk factors and risk markers of clients and (b) counselor actions (e.g., delivery of core program components). However, investigators do not always approach fidelity with this in mind. In a prior implementation study, RESPECT was widely delivered to low-risk clients [7], but fidelity assessments did not take client behavioral risk into account, nor did this study address whether clients were given the program on the basis of risk markers (e.g., MSM, IDU). As noted earlier, some low-risk clients may be included even when screening occurs. However, inclusion of substantial numbers of low-risk clients may inflate costs unnecessarily and also lead to negative perceptions of a program’s utility.

IMPLEMENTATION OF RESPECT

RESPECT has features that may make it easier to implement in practice, relative to many other HIV interventions. Overall, the simplicity of RESPECT may lead to higher program fidelity. This is important because prior translation studies suggest that it is difficult to achieve and sustain good fidelity in more complex HIV prevention programs (e.g., longer sessions, more sessions, multiple activities, and facilitators) [e.g. 20–24].

There is evidence that RESPECT can be implemented with fidelity. Hitt and colleagues [7] examined the impact of adding RESPECT in four agencies (DPHs and CBOs). Agencies received a 5-day training on RESPECT and the introduction of program quality assurance (QA) procedures to their organizations. Both observation of sessions and anonymous client exit surveys showed strong adherence to the RESPECT protocol in counseling sessions occurring within 3–6 months of the training. Thus, with reasonable training and oversight, counselors were able to sustain full program delivery of RESPECT during the early stages of implementation.

RESPECT is now being implemented on a large scale in the USA. In this broader context, a systems perspective provides a framework for considering how multilevel factors may impact fidelity. For example, the CDC offers policies about target populations and provides training through a national network of Prevention Training Centers (PTCs), but it is not known if these have diffused evenly across CBOs and DPHs delivering RESPECT. Thus, although training on RESPECT should be widespread, the availability of highly trained staff at a given agency will depend on a variety of factors such as training funds, staff turnover, refresher trainings, and supervision. Likewise, awareness of and adherence to CDC policies about target populations for program delivery may vary across agencies. Differences in compliance fidelity may be observed if policies and training have not diffused to particular types of agencies or geographic locales (e.g., urban CBOs, nonurban DPHs).

THE CURRENT STUDY

The current mixed methods study assessed program delivery in 26 agencies implementing RESPECT in order to address three primary research questions. The first aim was to identify client characteristics associated with receiving the program (i.e., who gets all core components). In this context, we also assessed compliance fidelity, the extent to which the full program was delivered to at-risk clients. The second aim was to examine compliance fidelity across agency type and geographic locale. This aim provides data on whether CDC policies and training efforts have diffused across participating agencies. The third aim of the study was to use the qualitative data from counselors to verify, extend, or demonstrate exceptions to our quantitative findings. Qualitative data provide insight into how and why counselors adapted the program to various clients or under certain circumstances. This research provides an initial test of the extent to which RESPECT is being delivered with fidelity in practice settings.

METHODS

Overview

Data for the current paper were collected as part of a larger mixed methods investigation, the Translation into Practice (TIP) study. The present report focused on client characteristics and agency factors associated with full program delivery. We used data from client exit surveys for quantitative analyses and semi-structured qualitative interviews with counselors for triangulation.

Sampling and samples

Agencies

The agency sample frame (N = 30) was constructed from listings of the Academy of Educational Development, CDC Behavioral Training Centers, and Kaiser Family Fund’s National HIV Prevention Inventory [25]. Agencies were eligible if they had been conducting RESPECT for a period of ≥6 months and the executive director agreed to participate. Quota sampling was used to fill four cells: urban DPH (n = 7) and CBO (n = 10) and nonurban DPH (n = 7) and CBO (n = 6). Agencies were from 11 states, representing all regions of the USA.

Clients

We obtained anonymous exit surveys from an opportunistic sample of clients. Clients were eligible to participate if they were 18 years of age or older and had received the first session of RESPECT. Twenty-six agencies provided client exit interviews (N = 830; M = 32 cases/agency), and our analyses are limited to these agencies.

Counselors

Counselors were enumerated during executive director interviews. Priority was given to selecting counselors identified as core program providers (i.e., provided the program on a regular basis). In agencies with four or fewer core counselors, we recruited and interviewed all counselors (≥18 years of age). In agencies with more than four core counselors, we randomly selected four for recruitment and interview. If agencies had insufficient numbers of core counselors, we sampled from the pool of counselors who were providing RESPECT on an ad hoc basis. No counselor declined to participate. Counselors from the 26 agencies that provided client data are included here (N = 64). Most counselors were paid employees (94 %) and were longstanding staff members (paid employees M = 7 years; volunteers M = 4 years). The majority of counselors were female (63 %) and represented a variety of racial/ethnic backgrounds: White (45 %), Black (20 %), Hispanic (16 %), and mixed or other (19 %).

Data collection procedures

Brief, anonymous client surveys were obtained at participating agencies where staff had been trained in study procedures. The exit survey was completed after the participant’s first RESPECT session, which was chosen (a) because the activity that takes place in this session is fundamental to the program; (b) to standardize the assessment, since all clients get the first session, but not all get or return for a second session; and (c) because clients are able to report on essential components that constitute this session. This approach provided reasonable coverage of the program (see “Measures”). No written consent was obtained, in order to ensure anonymity. Following informed consent, a staff member other than the counselor provided clients with the survey and an envelope in which to place the completed survey. Clients returned sealed envelopes to a secure drop box, agency staff, or mailed them postage-paid; surveys were received by Westat Corporation.

We conducted semi-structured, telephone, and in-person interviews (45–60 min) with counselors. Interviews were conducted under private conditions and were digitally recorded, transcribed, and checked for accuracy. Small incentives were provided.

Measures

The survey instruments were developed in a multistage process including consultation with experts in the field of translation research, development and refinement of instruments, cognitive pretesting of the client survey, and full instrument pretesting in the field with an agency conducting RESPECT.

Client survey

The self-administered client survey was brief (5 min) in order to accommodate clinic flow and client schedules. The survey asked about the following client characteristics and experiences: (a) demographics (gender, age, race/ethnicity), (b) sexual orientation, (c) sexual and drug use risk, (d) reason for attending the clinic, and (e) number of times tested for HIV. The survey also asked clients to report the gender of their counselor. Finally, the exit survey assessed exposure to RESPECT counseling (see below) and was used to determine program delivery. Client exit indices have been found to be reliable reports of what transpires in related settings and are widely used in health services research [23, 26–29]. The full measure is available from the first author.

Determining full program delivery

Based on the client survey, a program delivery index was designed to assess three fundamental program components reflecting the primary objectives of RESPECT [6] and determine if these components were conducted during the counseling session. Table 1 includes items and scoring criteria. Rapport building and risk assessment included (a) items assessing counseling, with affirmative responses to all three items considered reflective of a high standard of counseling; (b) discussion of health goals and risk behaviors; and (c) discussion of sexual and/or drug use behaviors. We weighted items that reflected one-on-one counseling more heavily than other items because the counseling is fundamental to the delivery of the program. Risk prevention analysis comprised (a) items assessing discussions of reasons to engage in risky behavior and (b) an item about discussion of barriers to reducing risk. Negotiated risk reduction was assessed by items reflecting discussion of behavior change plans around sex and/or drug use. As an index, the total score is a simple summation, and there is no assumption that these items are correlated [see 30]. Scores indicate if the elements of each program component occurred but do not assess quality of counseling.

Table 1.

Fidelity index: survey items

Component	Items	Points
Rapport building and risk assessmenta,b		3
	Did you feel that the counselor listened to you?
	Did the counselor give you a chance to talk as much as you needed about your concerns and questions?
	Did the counselor understand what you had to say?
	Did you talk about your health goals? For example, health goals might include eating better, drinking less alcohol, or not catching diseases in the next month.
	In your session today, did you discuss your sexual behavior?
	Did you discuss the sexual things you have done recently that may have put you at risk for getting the AIDS virus or other sexual diseases?
	Did you discuss your drug use?
	Did you discuss how your drug use might put you at risk for getting the AIDS virus?
Risk prevention analysisc		2
	Did you discuss the reasons why you sometimes have sex without a condom?
	Did you discuss how bigger things in life might influence your sexual behavior?d
	Did you discuss how bigger things in life might influence your drug use?d
	Did you talk about how to change things that might make it difficult for you to reach your health goals?
Negotiated risk reductionc		1
	Did you agree on something about your sexual behavior that you can do in the next few weeks that would help lower your risk for getting the AIDS virus or other sexual diseases?
	Did you and the counselor agree on something that you can do in the next few weeks that would help lower your risk for getting the AIDS virus from drugs?

We took as positive evidence any discussions of sex or drugs in the context of the various components

^aScoring: To reflect a high standard for counselor skills, a “yes” response to all three items was required to receive 1 point

^bScoring: One point for discussing either sex or drug risk behaviors or both

^cScoring: Risk prevention—1 point if any one of first three items received a “yes” response and 1 point if “yes” to health goals item; negotiated risk—1 point if either item received a “yes” response

^dDefinition provided: “Bigger things might include losing a job, family problems, your friends, your lifestyle, or what you do for fun”

Counselor interviews: program adaptations

Counselor interviews provided an opportunity to validate and extend quantitative findings. In the present paper, we focused on circumstances that lead to adaptations in RESPECT. The interview included questions on adaptations to core program elements (e.g., What kinds of circumstances cause you to change how you…conduct one-on-one counseling; use a teachable moment to motivate clients to change risk-taking behavior; explore the circumstances and context of a recent risk behavior to increase client’s perceptions of susceptibility; or negotiate an achievable step which addresses the larger risk reduction goal of the client?) and included specific queries about program adaptations owing to time constraints, with repeat testers, and with low-risk clients.

ANALYTIC PLAN

Quantitative analyses: model building

In order to address aims 1 and 2, we constructed logistic regression models examining client, agency type, and counselor characteristics with regard to program delivery scores of 6 (complete program delivery) versus 0–5 (less-than-complete program delivery). We anticipated that counselors would be able to deliver the program with a high degree of accuracy because (a) RESPECT is a relatively straightforward behavioral intervention and (b) the CDC/Prevention Training Centers have made a substantial effort to train agency personnel. We dichotomized the measures because the operational definition of fidelity is delivery of all core components (i.e., perfect score). Further, logistic regression requires fewer distributional assumptions about independent variables than does ordinary least squares regression. The goal was to develop a parsimonious model that allowed for examination of correlates of compliance fidelity, testing for variation across types of agencies and counselor gender. All univariate and multivariate analysis results are based on logistic regression models run in Stata Release 12, with standard errors adjusted for clustering by agency.

Using our knowledge of the program, HIV/STI epidemiological research, and relevant policies, we identified risk factors (i.e., sex and/or drug use) and risk markers that might be used to make delivery decisions. With regard to risk markers, we included epidemiological markers (i.e., race/ethnicity, sexual orientation, age, gender), a recommended risk marker (i.e., repeat testing), and client’s reasons for attending the clinic. This latter variable was examined because clients whose primary reason for obtaining services was to obtain HIV testing might be expected to have relevant risks, whereas those whose interest in testing is secondary to other issues may be less likely to have HIV-related behavioral risks. We also examined if the types of risk factors or markers used in decisions about delivering the full program differed by agency type (i.e., urban CBO, urban DPH, nonurban CBO, nonurban DPH) by testing interactions between risk factors/markers and agency type.

We initiated model building by entering variables that achieved p < 0.10 at the univariate level: client behavioral risk, race/ethnicity, agency type (four categories), and number of times tested for HIV (dichotomized). We included those variables that evidenced trends toward significance to ensure inclusion of all potentially important correlates. A “basic model” containing significant correlates of full program delivery was used to test for other significant correlates by adding one additional correlate at a time to the basic model. We also examined all possible two-way interactions (e.g., risk factors and risk markers by agency type). Finally, we examined other potential significant main effects (e.g., client gender, services sought) before determining that we had identified the most parsimonious and complete model. We then constructed a final model that included only significant correlates of full program delivery.

Qualitative analysis

Using a team approach [31], we conducted descriptive coding and pattern coding [32] in order to verify and extend quantitative results [33] (aim 3). Multiple team members read transcripts and coded text. Where discrepancies were found, coders discussed specific cases and came to consensus. At various stages, coders’ findings were reviewed by the team and interpretations discussed.

RESULTS

Table 2 presents client demographic and background data and data on the distribution of the client sample by type of agency and counselor gender. Although the majority of clients were young adults (18–29 years), White, heterosexual, and female, over 40 % of clients were ethnic or racial minorities, and over one third identified as gay or bisexual. Most clients reported sex- or drug-related risk factors, had been previously tested for HIV, and were seeking HIV services. The majority (89 %) received testing and counseling from female counselors (63 %) and attended urban agencies (93 %). There was no difference in the number of respondents obtained from CBOs and DPHs.

Table 2.

Sample characteristics and fidelity score distribution

Characteristic	%a	(n)b	Characteristic	%a	(n)b
Gender			Agency type
Male	55	(454)	Urban DPH	34	(278)
Female	43	(354)	Urban CBO	31	(255)
Transgender	1	(11)	Nonurban DPH	18	(146)
			Nonurban CBO	18	(151)
Race
White	54	(431)	Services
African American	22	(178)	HIV/STI testing	93	(767)
Hispanic	20	(159)	Other	7	(54)
Other	5	(37)
			Counselor gender
Age			Male	38	(314)
18–29	51	(414)	Female	62	(508)
30–39	26	(212)
≥40	23	(187)	Program delivery scores
			0	0.3	(2)
Sexual orientation			1	1.5	(12)
Gay or bisexual	32	(264)	2	4.1	(33)
Male	24	(201)	3	4.8	(38)
Female	8	(63)	4	8.8	(70)
Heterosexual	61	(509)	5	12.5	(100)
Male	28	(228)	6	68.0	(542)
Female	34	(281)
Other/unknown	7	(57)
HIV tests in past yearc
None	38	(308)
One	34	(275)
Two or more	29	(237)
Sex/drug risksd
Yes	89	(693)
No	11	(82)

DPH department of public health, CBO community-based organization

^aPercentages ≠ 100 due to rounding

^bN_s ≠ 830 due to missing data

^cAt the agency currently attending (n = 26 agencies)

^dPast 3 months (see “METHODS”)

Program delivery

Program delivery index scores ranged from a low of 0 to a high of 6 (see Table 2), with scores of 6 indicating that clients reported all program elements having been delivered. Our descriptive data provide a measure of how consistent counselors were in providing the full program to clients regardless of risk. Approximately two thirds of all clients received the full program (program delivery score = 6).

Multivariate correlates of program delivery

The final logistic model (see “ANALYTIC PLAN”) included only significant correlates of full program delivery. Full program delivery was associated with client risk factors, client race/ethnicity, type of services sought, and agency type by counselor gender interaction (see Table 3). The main effects for counselor gender and agency type are not interpretable because of the significant interaction. Aims 1 and 2 were addressed using the single multivariate regression model, allowing us to simultaneously control for other correlates.

Table 3.

Correlates of high fidelity (logistic regression model)

Variable (reference)	OR	95 % CI
Risk (none)
Any	2.12**	[1.33, 3.39]
Client race/ethnicity (White)
African American	1.89*	[1.12, 3.19]
Latino	2.04***	[1.39, 3.00]
Other	1.05	[0.44, 2.50]
Services (HIV/STI test)
Other	2.72*	[1.01, 7.32]
Agency type by counselor gender interaction
Urban DPH: male vs. female	3.85**	[1.40, 10.57]
Urban CBO: male vs. female	0.64	[0.34, 1.20]
Nonurban DPH: male vs. female	1.18	[0.67, 2.07]
Nonurban CBO: male vs. female	0.56	[0.17, 1.81]

N = 710, high fidelity = 1 (Kamb score of 6), low fidelity = 0 (Kamb score of 0–5)

OR odds ratio, CI confidence interval

*p = 0.01, **p = 0.001, ***p = 0.0001 (Wald χ ² = 198.19, p < 0.00001). Hosmer-Lemeshow, p = 0.75

Which clients receive the full RESPECT program?

The multivariate logistic model examines client characteristics associated with full program delivery (aim 1) and may provide insight into factors that counselors use to determine program eligibility. The final multivariate logistic model indicates that likelihood of delivering the full RESPECT program was associated with client sex and drug use, being African American or Hispanic (relative to White), and seeking services other than HIV/STI testing. Thus, counselors in our study may have relied on both risk factors and risk markers to determine which clients received the full program. Descriptive analysis of the specific types of “other” services sought indicated that substantial proportions of those seeking drug and alcohol treatment or mental health services received the full program (n = 28/30; 93 %). In contrast, those seeking general health care, pregnancy services, or miscellaneous services were less likely to have received the complete program (n = 29/63; 46 %). A logistic model controlling for clusters shows that this effect is significant (odds ratio (OR) = 16.4, 95 % confidence interval (CI) = 3.78, 71.23). Thus, in making decisions to deliver the full program, counselors appear to be focusing on clients who have comorbidities that may elevate HIV risk.

Due to the importance of risk markers and risk factors in program delivery, we conducted several sets of post hoc analyses to further examine these variables.

Repeat testing as a risk marker

Repeat testing is a recommended risk marker for eligibility for RESPECT but was not significant in the final multivariate model. Preliminary analyses, however, showed a relationship between race/ethnicity and repeat testing (data available from authors), and our final model revealed significant differences in delivery of the full program by race/ethnicity.

A logistic model examining repeat testing and race/ethnicity, while controlling for cluster effects, found that, relative to Whites, African American (OR = 2.34, p = 0.006) and other-ethnicity clients (OR = 2.96, p = 0.05) were significantly more likely to be repeat testers, but Hispanics were not significantly different from Whites (OR = 1.51, p > 0.10) (repeat testing: White 55 %; African American 74 %; Hispanic 65 %; other race/ethnicity 78 %). The results highlight the complexity of factors counselors may face in screening clients to determine eligibility. Counselors may be using either repeat testing or race/ethnicity as proxy indicators of risk.

Risk markers and risk factors

Both risk markers (e.g., repeat testing, Black or Hispanic ethnicity, and behavioral risk factors (i.e., sex and drug use in past 3 months)) were significantly correlated with full program delivery in the multivariate analyses. In order to better understand counselor decisions, we examined the extent to which risk factors and risk markers co-occurred. In these analyses, we used all risk markers included in our data set. Not surprisingly, there was overlap between the two groups: 90 % of clients with risk markers also reported risk factors. However, about one fifth of clients with risk factors did not have a risk marker. Therefore, using risk markers to make decisions about program delivery provided counselors with entrée to a substantial proportion of at-risk clients. However, the use of risk markers alone would have missed a fifth of the clients whose behavior placed them at risk.

We were also interested in whether risk factors and risk markers independently related to full program delivery. To explore this, we divided the sample into four groups: clients with both risk markers and risk factors (69 %; n = 532/765), clients with risk markers only (7 %), clients with risk factors only (20 %), and clients who had neither risk markers nor risk factors (3 %). Using logistic regression, we found that clients with both indicators were more likely to receive the full program than those who had only one indicator or who had no risk markers or risk factors (<0.001). The full program was delivered to 74 % of clients with risk factors and risk markers; 55 % with risk factors but no risk marker; 59 % with no risk factors but who had risk markers; and 44 % with neither risk factors nor risk markers. These findings suggest relatively high compliance fidelity when clients have both indicators; but, at the same time, the program is also delivered to those who have neither.

Have policies and training diffused evenly across agency types?

Aim 2 addressed whether compliance fidelity varied across agency types and geographic locales and was also assessed in the multivariate logistic model. These analyses allow us to examine whether the policies and training related to RESPECT have diffused differentially. Although there is some evidence of differential delivery of the full program by agency type—urban CBO (69 %), nonurban CBO (87 %), urban DPH (61 %), nonurban DPH (60 %)—and by counselor gender (male 74 %; female 65 %), the presence of an interaction between these variables reveals that these differences are modified by counselor characteristics. We found that the likelihood of full program delivery was associated with counselor gender only for counselors at urban DPHs, with female counselors less likely than male counselors to deliver the full program (79 % at urban DPHs got full program delivery with male counselor; 54 % with female counselor). Because the multivariate model controls for risk factors and risk markers, this outcome suggests that policies and training have diffused evenly across agency types with the exception of a segment of female counselors in urban DPHs.

Qualitative findings: validating and extending quantitative findings

Aim 3 of the study was to compare the quantitative client survey findings with qualitative data from counselors (N = 64). This comparison provides external validity for inferences derived from quantitative findings and offers an opportunity to explore issues not identified a priori. Based on counselors’ discussions of how and under what conditions they adapted the program, we highlight three factors that may impact compliance fidelity: client risk, repeat testing, and environmental factors (see Table 4 for representative quotes).

Table 4.

Counselor adaptations to RESPECT: client risk, repeat testing, and environmental factors

Topics	Adapted programs
	Yes	No
Low-risk clients	“To an extent. I’m still going to ask them, you know, for their risk exposures, things of that nature, but when I notice that, you know, they’re not really engaging in much risk exposure, I won’t keep asking them the same thing over and over.” (M, UC)	“I try to educate them more in reference to—because there is a reason why they are here in our office… because a lot of times when they do come in, they might not say that they are at-risk, but there are other things that might be present.” (F, UD)
	“We don’t go over the risk factors or creating a plan or identifying risk if they have no risk, ‘cause a lot of people come in to get HIV tests for immigration.” (M, UD)	“No. No. Because even though they tell you that they don’t have any risk, they wouldn’t be there if they had no risk… and it doesn’t hurt to educate a person more… if they’re to see me, there’s got to be a reason.” (F, ND)
Repeat testers	“When a repeat offender comes over here, he always thinks it’s [test results] going to be positive, and then when it comes negative, it’s actually the highest leverage that you have to get it [into] their mind. Now you [are] negative. How are we going to work this time not to get you back over here?” (M, UD)	I: “Okay. Do you modify your one-on-one counseling sessions for repeat testers?” R: “No. I actually do not, because if they didn’t get it the first time, actually got to be more elaborate the second time.” (M, UC)
	“You know, yes, in that I’m not so much looking for the highest of their risks usually. [I look] for new risks. I’m aware of what they’re, what they are doing. I’m aware of their risks and… looking for different teachable moments based on… certain things that happened prior to their last session.” (M, NC)	“What I do is, each time they come back, it’s like they’re brand new… because one of the things you have to understand [is] that the repeat clients, they’re basically doing the same high risk behaviors, they’re continuing doing. So it’s the step one, and we’re starting over again.” (F, UC)
Environmental factors	“… It’s only for 24 h… so we do have a rush because it’s been, it’s going to be publicized, you know, through TV and radio stations… So in that case, if we have a lot of people, I’d say that we would go without counseling.” (F, UD)
	“I’m in outreach, so I see a lot of low-risk and a lot of high-risk clients en masse, you know, kind of in groups, large groups, so my shtick doesn’t change. I mean, a low-risk client is, to me, is just as at-risk as a high-risk client.” (F, UD)
	“You know, in the field that can be an issue… if they… have a sense of urgency that they just want to be tested, they don’t want to have a whole lot of counseling, then they may not be a RESPECT client. So, sometimes we have to pull back from the RESPECT even though we’ve introduced it and started it, we have to pull back and just say they’re not a RESPECT candidate.” (M, NC)

F female counselor, M male counselor, UD urban DPH, UC urban CBO, ND nonurban DPH, NC nonurban CBO

Client risk

Qualitative data on client risk corroborated quantitative results and provided additional insight into when and how counselors adapted the program for low-risk clients. Some counselors indicated that they modified delivery for low-risk clients, including clients seeking HIV tests for immigration, and those reporting no risk. Adaptations focused on changing or eliminating two elements of the RESPECT protocol: negotiating an achievable step that supports the larger risk reduction goals of the client and exploring the context of recent risk behavior.

In keeping with the quantitative results, we found that not all counselors made distinctions based on client risk. Some counselors did not modify the program because they assumed that clients’ presence at the agency was indicative of some level of undisclosed risk; others did not assume risk but viewed RESPECT as an opportunity to reinforce safer behavior by providing information.

Repeat testing

Counselor interviews suggest that repeat testing was a factor in making adjustments to delivery, including intensification of the delivery or making appropriate changes to counseling to reflect ongoing risk. Other counselors approached repeat testers as if they were “new” at-risk clients. Both approaches likely result in good fidelity.

Environmental factors

Qualitative interviews revealed the importance of contextual or environmental factors in program delivery, thus extending the quantitative findings. According to counselors, contextual factors decreased the practicality of using risk or other client characteristics as a means of vetting clients. Such factors include outreach settings involving group delivery of RESPECT or where client priorities led to diminished interest in counseling and typically involved modifications to the program.

DISCUSSION

We examined the implementation of RESPECT in practice settings. Our findings suggest that the CDC’s structured efforts to diffuse evidence-based HIV behavioral interventions have achieved considerable success with regard to this brief individual-level counseling and testing program.

Counselor decisions: delivering RESPECT to at-risk clients

In order to achieve good program reach and population-level efficacy, it is fundamental that at-risk clients receive RESPECT. Thus, agencies must be able to efficiently sort low-risk clients from those at risk; quantitative findings suggest successes and challenges in this regard. Although the majority of at-risk clients received RESPECT, many low-risk clients did so as well. Because counselors play a significant role in determining who receives the program, their reflections provide insight into decision-making. Qualitative findings show that many counselors adapted RESPECT appropriately for low-risk clients, while providing the full program to those at risk. Nevertheless, a minority of counselors mentioned providing the program to low-risk clients to support continued safe behavior or because they believed that clients are not forthcoming about their risks. These findings suggest that some fine-tuning of screening and program delivery procedures—for example, through training and supervision—would increase the efficiency and cost-effectiveness of RESPECT.

The quantitative findings show that the majority of counselors used risk factors and/or risk markers to make reasonable decisions about program delivery. In order to determine program delivery, counselors may be using their knowledge of epidemiological data linking high HIV prevalence with demographic characteristics and comorbidities (e.g., race/ethnicity) [34] and also relying on risk proxies such as testing history. Moreover, clients with comorbidities, such as those seeking drug treatment or mental health services, may be perceived by counselors as being in high need of intervention. The prioritization of clients found in this study is consistent with national policies that emphasize targeting at-risk groups for HIV [35, 36] and with general trends toward higher rates of STI screening in particular ethnic groups [27].

Race/ethnicity was a significant correlate of full program delivery in our multivariate logistic model, but no counselor mentioned race or ethnicity as a factor in the decision to deliver RESPECT in the qualitative interviews. This discrepancy between the quantitative data and qualitative findings has several potential explanations. First, the qualitative findings may reflect counselors’ desires to avoid the appearance of stereotyping racial/ethnic groups when, in fact, they do use these demographic characteristics in making decisions. Second, if agencies serve primarily ethnic minority clients, race or ethnicity would not be a factor in counselors’ decision-making about RESPECT. Finally, it is possible that counselors are using a proxy such as repeat testing, which was associated with race/ethnicity, as a marker for risk.

Together, the findings suggest that counselors use broadly defined risk factors and risk markers in order to provide the RESPECT to clients who are likely to need the intervention. As noted, this approach has inefficiencies. However, given the importance of identifying HIV-positive individuals in the population, an over inclusiveness that includes modest numbers of low-risk clients reflects good public health practice.

The diffusion of RESPECT into practice

Despite widespread adoption, the present study is the first to examine the diffusion of RESPECT into practice. Our findings suggest that CDC policies and training on program delivery have diffused across practice settings and counselors. That is, we found no interaction effects for agency type by risk factors or risk markers in relation to program fidelity. These findings underscore the strength of the national efforts to promote common standards for program delivery.

With regard to compliance fidelity, we found that the underlying support systems (e.g., training, quality assurance) have helped sustain relatively good fidelity across agency settings. One caveat is that female counselors at urban DPHs had lower compliance fidelity scores. We lack data to explain this finding. It is possible, however, that some agencies are providing less-than-optimal training or have experienced staff turnover, resulting in groups of counselors with deficits in program knowledge. Additional work is needed to explore these issues.

Environmental fit and program delivery

Environmental factors emerged in the qualitative findings as an important element in program delivery. When environmental fit is poor, it is difficult to deliver programs as intended. By “fit,” we mean the extent to which the parameters of the program can be faithfully integrated in the context or environment in which the program is delivered. Although RESPECT was initially developed for clinic settings, counselors discussed delivering the program in bars, at street fairs, and in mobile vans. At these sites, counselors adapted the program in ways that are inconsistent with training and policy and which reduce compliance fidelity. Achieving fit between programs and delivery settings is crucial for achieving compliance fidelity [15] and an area of importance for future investigations.

Limitations

There are limitations to this investigation. Although we constructed a purposive sample of agencies from all regions of the USA, data are based on convenience samples of agencies and of clients, and findings may not generalize. Our measure of fidelity also has limitations [also see 37]. We were unable to assess counseling strategies that clients may not recognize (e.g., teachable moment), nor did we assess quality of delivery (i.e., competence fidelity). Furthermore, it is possible that clients’ reports reflected a desire to portray their counselors in a positive light or that counselors changed their practices because of the knowledge that exit surveys were being conducted. Nonetheless, alternative measures (e.g., observation, counselor reports) have methodological challenges, including social desirability and Hawthorne effects [see 14, 15]. In a study using observational measures and client exit reports, Hitt and colleagues [7] found that both measures were sensitive to the effects of training programs designed to enhance program fidelity. Finally, client fidelity indices have been found to be reliable reports of what transpires in related settings [23, 26, 28, 29]. Thus, despite limitations, the current study provides substantial coverage of core program elements in the initial RESPECT session through client reports and uses counselor interviews to substantiate and expand on client data.

CONCLUSIONS

Our findings provide support for the CDC’s diffusion efforts and point to the value of brief interventions that can be adapted for individual clients. Future investigations should continue to focus on compliance fidelity and the factors that contribute to high-quality implementation and to program integration (i.e., fit).