Assessing Outcomes of a Stigma-Reduction Intervention with Venue-based Analysis

Abstract

Purpose

A randomized controlled trial with a matched design was conducted during October 2008 and February 2010, aiming at reducing HIV-related stigma in healthcare settings.

Methods

Forty county hospitals in Fujian and Yunnan provinces of China were matched into pairs and randomized to either an intervention condition or a control condition. Forty-four service providers were randomly selected from each hospital, yielding a sample of 1,760. Intervention outcomes were assessed at baseline, 6 and 12 months based on venue-based pair comparisons. We identified and trained 30 popular opinion leaders (POL) in each intervention hospital among service providers to disseminate stigma reduction messages to their peer providers.

Results

Hospital and participant characteristics were comparable between the intervention and control conditions. Thirteen out of twenty pairs of hospitals showed significant reduction in the stigma outcome measure at the 6-month follow-up assessment. For most hospitals, the intervention effects were maintained at the 12-month follow-up assessment. Among the 13 pair of hospitals which showed intervention effect at 6-months, eight were in Fujian and five were in Yunnan. The non-significant hospitals at 6-month had more beds than significant hospitals. However, the difference did not reach statistical significance.

Conclusions

A matched design and venue-based analysis provide more insight in assessing intervention effects for facility-based intervention trials. The identification of venue-based or hospital characteristics that are associated with intervention efficacy provides additional implications for the adaptation and implementation of future interventions.

INTRODUCTION

It was estimated that there were 780,000 (620,000 – 940,000) people are living with HIV (PLH) in China by the end of 2011 [1]. As HIV infection has evolved into a chronic disease, more and more PLH require regular medical check-ups and routine care in the Chinese health care systems. Previous studies revealed that Chinese service providers’ stigmatizing attitude and behavior were a key barrier for HIV testing and treatment including refusal of care, sub-optimal services, or breach of confidentiality, and were correlated with providers’ perception of social norms as well as concerns about their occupational safety [2–5]. Most stigma reduction intervention studies have been conducted at the individual level using strategies such as informational approaches, skill building, counseling/support, and testimonials [6]. Given the fact that stigma is a complex social phenomenon that is influenced by many factors (e.g., public policy, institutional governance, social norms), researchers recommend that an intervention with multiple components be used to achieve the maximum effectiveness in attitude and behavioral changes [7–10].

A randomized controlled trial was conducted in China to address HIV-related stigma and discrimination in healthcare settings. The intervention included both interpersonal and structural level components, with the aim to reduce HIV-related stigma among providers in the medical community. During the trial development, a design issue emerged as whether to use a simple randomization procedure to appoint the intervention and control groups, or alternatively to form matched pairs of clusters and then to allocate the intervention status randomly to one cluster in each pair [11]. The collaborative research team chose the matched-pair design, which increased the potential of achieving balanced intervention groups, thereby improving the power for detecting intervention effect [12,13].

The matched-pair design is not new to the HIV research and has been used in a number of intervention studies [14–16]. However, most reports on efficacy focus on intervention outcomes by comparing overall changes in outcome measures between the two intervention conditions or averaging differences between the two conditions across pairs of venues [17–19]. Few randomized controlled trials have studied intervention outcomes at the venue level, thus missing the opportunity to obtain supplementary data. For example, some venue-level factors might be associated with intervention outcomes in the venue, but conventional outcome analysis between intervention conditions will omit the opportunity to investigate such factors. In this study, the randomized control trial with the matched-pair design sowed an overall significant intervention effect and the findings were published [20]. We also observed that the intervention effects were not homogeneous across provinces and across matched-pairs of hospitals. These findings motivated us to further estimate the intervention effect at the matched-pair level, and investigate how the intervention effects differ across provinces and matched-pairs. It was also of interest to learn whether any of hospital-level characteristics were associated with the heterogeneous intervention effects. The study results have potential to inform other facility based intervention studies.

METHODS

Study site

The trial was conducted from October 2008 to February 2010 in Fujian and Yunnan Provinces, China. Yunnan province has one of the highest HIV infection rates in China, with 57,325 cases recorded by the end of 2007 [21]. In Yunnan, HIV transmitted through drug use drastically decreased from 100% in 1989 to 42.5% in 2007, while sexual transmission accounted for 47.4% of total cases [21]. In contrast, the reported number of HIV/AIDS cases in Fujian province was relatively low [22]. Fujian reported 1,387 cumulative HIV cases by the end of 2007, with about two-thirds of the cases spread through unprotected sexual acts [23]. Yunnan and Fujian were included in this trial in order to represent the varied HIV rates and infection routes seen in China.

Hospital sampling and matching

This study employed a cluster randomized controlled study design, stratified by province. Only county-level hospitals were included in the study because they are the most advanced medical centers within easy access of most Chinese residents, and many HIV cases are first detected in county hospitals. With administrative support from the Provincial Health Department, a total of 40 hospitals were randomly selected out of 214 county hospitals in the two provinces (20 hospitals from each province). A matched-pair design was applied to optimize the randomization [24,25]. The 20 hospitals in each province were matched into pairs based on 1) type of hospital (general or specialized); 2) number of beds; 3) number of staff; and 4) reported HIV cases. We also collected information about the provision of antiretroviral (ARV) drugs, the history of occupational exposure, and the availability of HIV post-exposure prophylaxis (PEP) drugs in these hospitals. However, these data were not used as matching criteria. Based on the primary criteria, the two hospitals within each pair were randomized to either intervention or control condition. Repeating this procedure resulted in 10 hospitals assigned to the control condition and 10 hospitals assigned to the intervention in each province. The geographic location of the hospitals was considered to avoid potential contamination.

Participant recruitment

Service providers in each hospital were randomly selected from a publicly available hospital staff roster. Providers who delivered direct services to patients, including doctors, nurses, and lab technicians, were included. The sampling ratio of doctors, nurses, and lab technicians was preset at 50%, 45%, and 5%, respectively, to represent the personnel profiles of the medical staff at the county hospitals. Providers had to be 18 years or older to participate. When approaching participants, trained recruiters fully explained the purpose of the study, procedures, voluntary nature of participation, and potential risks and benefits. Written informed consent was obtained before data collection. Forty-four providers were randomly sampled from each of the 40 selected hospitals, resulting in a total of 1,760 provider participants in the study. The refusal rate was about 3%.

Intervention

The intervention, known as White Coat, Warm Heart (WW), was developed based on the findings from previous studies [2,3,7]. The intervention integrated behavioral and structural level components. At the structural level, both intervention and control hospitals received the same amount of universal precaution supplies (e.g., disposable sharp containers, medical disposable clothes, waterproof aprons, protective goggles, and rubber gloves). This component was based on previous findings showing that limited access to universal precaution supplies and lack of precaution practices were related to stigmatizing attitudes and behaviors [2,26]. The behavioral components of the WW intervention were built upon Diffusion of Innovation theory, which states that new behavioral trends are most efficiently established when a “critical mass” of popular opinion leaders (POLs) adopt and endorse the new trend [27,28]. About 20% of service providers who were respected, trustworthy, and influential among peer providers were identified as POLs, recruited, and trained in two waves in each of the 20 intervention hospitals through gatekeeper recommendations and co-worker nominations. A total of 456 POLs were recruited from the 20 intervention hospitals with informed consent (refusal rate was less than 3%). The average age of POLs was 37.7 years at baseline; 69.3% of the POLs were female and 47.4% were doctors.

The POLs participated in four weekly group sessions over a one-month period and three reunion sessions during the 12 months after the initial training. The sessions, each lasted about 90 minutes, were conducted by well-trained local health educators. The contents of group sessions focused on complying with universal precaution principles, ensuring occupational safety, equal treatment to all patients, reducing stigma towards PLH and at-risk population, improving the provider-patient relationship, and making efforts to improve the medical environment. The WW intervention employed interactive strategies to encourage trainees’ full participation. During the training sessions, POL providers learned and practiced communication skills in diffusing intervention messages to their co-workers and serving as behavior change endorsers during their daily medical practice. The participation rate of the training and reunion activities was higher than 95%. Detailed information about the WW intervention is described elsewhere [20].

Data collection

The study procedure and materials were approved by the Institutional Review Boards (IRB) of the University of California, Los Angeles (UCLA), and the IRB of National Center for AIDS/STD Control and Prevention, Chinese Center for Disease Control and Prevention (NCAIDS/CCDC). Hospital characteristics, including location, number of beds, number of service providers, cumulative number of reported HIV cases, provision of ARV therapy, history of occupational exposure, and accessibility of PEP in the hospital, were obtained from public available records and confirmed with hospital gatekeepers. The intervention efficacy was evaluated at baseline and the 6- and 12-month follow-up assessments. At each assessment, all participating providers completed self-administrated paper/pencil questionnaires in a private room, with a trained interviewer present to make sure that the participants finish the questionnaire independently and to clarify the participants’ questions. The standardized survey took an average of 30 to 45 minutes to complete. Participants were compensated 50 yuan (U.S. $7.70) for each assessment. The assessment was not anonymous since the participants were followed up over multiple time points. However, the responses were kept strict confidential. The follow-up rate was higher than 99% across all study sites, and no significant difference was observed in attrition rates between the two intervention conditions (Fig. 1).

Figure 1.

Flow of the participants and venues used in the study.

Providers’ general stigmatizing attitude toward PLH was measured using an eight-item scale that was modified from a scale previously used and validated by our study team in China [2,29]. The statements used in this scale include “People who got HIV/AIDS through sex or drug use got what they deserved” “AIDS is a punishment for bad behavior” “People who behave badly should be blamed for the epidemic of AIDS” “PLH should have the right to marry” “I would feel ashamed if someone I know got HIV/AIDS” “I would feel ashamed if someone in my family got HIV/AIDS” “I would not buy food from a vendor who has HIV/AIDS” “I would not share eating utensils with a PLH.” The responses to each statement ranged from 1 (strongly agree) to 5 (strongly disagree). Some items were reverse coded so that a higher score indicates more severe stigmatizing attitude toward PLH (α = 0.75; range: 8–40). Demographic variables, including age, gender, professional category (doctor, nurse, or lab technician), and if ever had contact with PLH at work (yes or no), were also collected at baseline.

Statistical analysis

An intent-to-treat approach was used to analyze the intervention effects. Baseline differences between intervention and control conditions were compared for both hospital and provider characteristics by province and tested using Chi-square and t tests for categorical and continuous variables, respectively. A mixed-effects regression model that included the providers’ characteristics (age, gender, occupation, whether a provider had prior contact with PLH), hospital indicator, visit (baseline, 6- and 12-month), and hospital-by-visit interaction was used to assess the hospital-level and province-level intervention effects on stigmatizing attitudes through contrasts. The model included a provider-level random effect to account for repeated observations within each provider and a separate set of covariance parameters per province. This modeling approach allows us to estimate not only the intervention effect for each matched pair of hospitals (control vs. intervention), and also the province-level intervention effect by averaging the intervention effects across the matched pairs within the same province through the model contrast. Differences in reducing stigmatizing attitudes (with 95% confidence intervals), adjusting for the provider’s characteristics, between intervention and control were estimated for each pair at both 6- and 12-month follow-ups. All statistical analyses were carried out with the SAS System for Windows (Version 9.2) and all of the graphs were generated using the publicly available statistical software R (R Development Core Team, 2008).

RESUTLS

Hospital and service provider characteristics at baseline

Hospital characteristics and provider sample characteristics are presented in Table 1. The number of beds and providers were comparable between intervention and control hospitals at baseline. The hospitals in Yunnan had reported more HIV cases and occupational exposure accidents than the hospitals in Fujian. More hospitals in Yunnan had provided ARV and had PEP drugs available at baseline than those in Fujian. Approximately 73% of the service providers in Yunnan and 62% in Fujian were female. The average age of participants in Yunnan (38.0 years) and in Fujian (38.2 years) was comparable. However, in Fujian Province, more providers in the control group fell into the younger category (35 years or younger) than the intervention group. About half (48.3% in Yunnan and 50.1% in Fujian) of the participants were doctors. Twice as many (76.0%) providers in Yunnan reported previous contact with PLH. Stratified by province, there was no significant difference in gender, profession, or previous contact with PLH detected between the intervention and control group participants (Table 1).

Table 1.

Hospital characteristics by intervention condition at baseline

	Yunnan			Fujian
	Control N (%)	Intervention N (%)	P1	Control N (%)	Intervention N (%)	P1
Hospital characteristics
Number of Hospitals	10	10		10	10
Number of beds
200 or less	5 (50%)	7 (70%)	0.6499	4 (40%)	2 (20%)	0.6999
201–500	5 (50%)	3 (30%)		5 (50%)	6 (60%)
> 500	0	0		1 (10%)	2 (20%)
Number of service providers
100 or less	1 (10%)	1 (10%)	1.0000	1 (10%)	1 (10%)	0.8000
101–200	5 (50%)	6 (60%)		2 (20%)	0 (0%)
201–300	3 (30%)	3 (30%)		2 (20%)	3 (30%)
>300	1 (10%)	1 (10%)		5 (50%)	6 (60%)
Cumulative number of reported HIV
None	5 (50%)	5 (50%)	1.0000	4 (40%)	4 (40%)	0.1789
1–10	3 (30%)	4 (40%)		3 (30%)	6 (60%)
> 10	2 (20%)	1 (10%)		3 (30%)	0
Currently provide ART service	8 (80%)	7 (70%)	1.0000	1 (10%)	0	1.0000
Occupational exposure accidents in history	3 (30%)	1 (10%)	0.5820	0	0	1.0000
Currently have access to post-exposure prophylaxis	10	10	1.0000	1 (10%)	3 (30%)	0.5820
Service Provider characteristics
Number of Providers	440	440		440	440
Gender
Female	332 (75.5%)	311 (70.7%)	0.1105	279 (63.4%)	266 (60.5%)	0.3668
Age (Year)
Mean (SD)	38.2 (7.5)	37.8 (7.6)	0.53252	39.3 (89.9)	37.0 (8.7)	0.59522
35 or younger	163 (37.1%)	176 (40.0%)		254 (57.7%)	200 (45.5%)
36–45	198 (45.0%)	181 (41.1%)		118 (26.8%)	149 (33.9%)
46 or older	79 (18.0%)	83 (18.9%)		68 (15.5%)	91 (20.7%)
Profession
Doctor	206 (46.8%)	219 (49.8%)	0.5650	218 (49.6%)	223 (50.7%)	0.9386
Nurse	202 (45.9%)	195 (44.3%)		181 (41.1%)	176 (40.0%)
Technician/Other	32 (7.3%)	26 (5.9%)		41 (9.3%)	41 (9.3%)
Ever had contact with PLH at work	339 (77.1%)	330 (75.0%)	0.4773	171 (38.9%)	164 (37.4%)	0.6457

¹Chi-square or Fisher exact test;

²Two-group t-test.

Province-level intervention effects

Table 2 presents the results from the mixed-effects regression and the estimated province-level intervention effects. Providers who had prior contacts with PLH were significantly associated with a lower level of stigmatizing attitudes (P = 0.0054). The hospital-by-visit interaction was significant (P < 0.0001), meaning that the time trends of the stigmatizing attitudes among pairs were different, which can be seen in Figure 2. At 6 months, the estimated differences in alleviation of stigmatizing attitudes between the intervention and control condition were highly significantly for Fujian (2.95 ± 0.32, P <0.0001) and Yunnan (1.87 ± 0.32, P <0.0001). The differences in intervention outcome became larger between these two provinces at 12 months (5.36 vs. 2.20, respectively). Lastly, we estimated the clustering effect using an intra-class correlation (ICC) measure, which can be interpreted as the fraction of total variability explained by hospital-level clustering. We found almost 50% of the total variability was due to the hospital-level variation in Fujian (ICC=0.424) and in Yunnan (ICC=0.489).

Table 2.

Regression estimates of provider’s characteristics and intervention effects from mixed-effects regression model.

Parameter	Estimate	SE	P
Service Provider Characteristics
Age (in year)	−0.001	0.002	0.5201
Male gender	−0.326	0.167	0.0511
Doctor (vs. nurse/technician/other)	0.104	0.151	0.4911
Contact HIV	−0.389	0.139	0.0054
Tests of Main effects (F statistics)
Hospital (F[39,1720]=6.44)			<.0001
Visit (F[2,3424]=309.5)			<.0001
Hospital x Visit (F[78,3424]=6.44)			<.0001
Intervention Effects1 (by Province)
Fujian
6-Month Assessment	2.950	0.316	<.0001
12-Month Assessment	5.360	0.312	<.0001
Yunnan
6-Month Assessment	1.862	0.316	<.0001
12-Month Assessment	2.196	0.312	<.0001

¹Intervention effect was defined as the difference in reduction of perceived stigma between intervention and control. The intervention effects by province were estimated through model contrasts.

Figure 2.

Estimated difference in stigma reduction (with 95% CI) at 6-month (left) and 12-month (right) follow-up assessments.

The solid circle and the circle with plus symbol represent the estimated difference in stigma reduction for the intervention versus the control condition. The horizontal bar represents the 95% CI and the significance is indicated by one star (P < 0.05) or two stars (P < 0.0001).

Hospital-level intervention effects

Figure 2 presents the estimated differences in reducing stigmatizing attitudes (with 95% confidence interval) for each of the 20 pairs of hospitals from the mixed-effects regression model, adjusting for age, gender, occupation, and prior PLH contact experience. The intervention effects were non-homogeneous among the matched pairs of hospitals. First, there were 13 pairs of hospitals that showed significant reduction in stigmatizing attitudes in the intervention versus the control condition at the 6-month follow-up, with the estimated difference in stigma reduction ranging from 2.19 to 5.02. At the 12-month follow-up, there were 16 pairs of hospitals that showed significant intervention effects, with the estimated difference in stigma reduction ranging from 2.11 to 10.33. These intervention effects either became stronger or persisted except for two pairs located in Yunnan (Pair ID: 18 and 19). Second, among 7 pairs of hospitals that did not show significant intervention effects at 6 month, 5 of them showed significant intervention effects in reducing stigmatizing attitudes at 12 months (range: 2.11 to 3.36). Third, there were 2 pairs of hospitals (Pair ID: 8 and 13) that showed significant intervention effects at the 6-month follow-up but did not last at the 12 months. From these analyses, we observed various magnitudes and patterns of the intervention outcome across 20 pairs of hospitals in two provinces and over two follow-up periods.

Hospital characteristics associated with intervention outcome at 6-month follow-up

For exploratory purpose, we grouped the paired hospitals according to their significances at 6 months: 7 non-significant pairs of hospitals and 13 significant pairs of hospitals, and further examined the associations between hospitals characteristics and intervention effect. First, the average number of hospital beds for those hospitals with non-significant intervention effects was slightly larger than those with significant intervention effects (310 vs. 293, respectively; P=0.72). Specifically, the average size for the pairs of hospitals that achieved statistical significant differences was significantly smaller than those that did not achieve statistical significance in Yunnan (178 vs. 275, respectively, P=0.031). Second, we investigated the proportion of providers who had contact with PLH and its relationship with intervention effect. The proportions of providers with prior contact for those significant pairs of hospitals was significantly lower than those non-significant pairs (50.4% vs. 69.5%, respectively, P=0.048). Lastly, the number of HIV cases treated for the significant pairs of hospitals was on average smaller than those non-significant pairs (11.6 vs. 4.6, respectively; P=0.30). These findings, although preliminary, encouraged further investigations on if hospital-level characteristics were associated with the heterogeneous intervention effects across provinces and matched pairs.

DISCUSSION

This study affirmed the overall effectiveness of the WW intervention for HIV-related stigma reduction in healthcare settings [20]. Interventions that address HIV-related stigma at the facility level have the potential to be cost-effective because they can reach large numbers of members, as demonstrated by some HIV risk reduction interventions [30]. In a recent article, Trickett and colleagues (2011) challenged the research designs in randomized controlled trials [31]. One of the drawbacks identified was the difficulty in investigating diversity because of issues related to statistical power and limited resources [31]. Implementation science focuses not only on the effectiveness of interventions, but also on the translation of interventions into routine practice in diverse settings [32,33]. This requires researchers to identify variations across sites even when the overall effects of the intervention may be of central interest.

An important design feature of the intervention trial is that the 40 hospitals were pair-matched prior to random assignment on the basis of variables known to be correlated with HIV-related stigmatizing attitudes in healthcare settings [34–36]. In addition to confirming the efficacy of the WW intervention, we also compared the intervention outcomes within each pair as well as across 20 pairs and further explored the characteristics of hospitals that might be associated with intervention outcomes. Based on the findings, researchers can further explore core elements in the implementation process at the facility level and link them to varying levels of intervention effects. To use different approaches to demonstrate evidence of intervention efficacy reflects the complex social processes involved in facility-level interventions.

Our study revealed that there were regional differences in the effects of the WW intervention, with more intervention hospitals in Fujian showing desirable intervention outcomes at 6-month than hospitals in Yunnan. This was in line with our previous findings from a study [37]. In that study, service providers in Fujian, where HIV prevalence was low, reported a lower level of HIV knowledge and a higher level of stigmatizing attitude toward PLH as compared to providers in Yunnan. Another notable finding from the present study is that providers in Yunnan had a much higher rate of contact with PLH than providers in Fujian. Several researchers have shown that increased contact with PLH may decrease stigma and the desire for social distance and restrictions [38,39]. Thus, low HIV prevalence and lack of contact with PLH may partially explain the relatively high degree of stigmatizing attitude among providers in Fujian. This could be the explanation for the earlier intervention effect seen in Fujian (i.e., there was much room for improvement), whereas in Yunnan there may have been a “ceiling effect” (i.e., less room for improvement) [40,41]. This is another example where there is a need to address intervention outcomes at various levels of analysis [42]. Providers’ individual attitudes toward PLH could be associated with the regional prevalence of the HIV epidemic, and the magnitude of the problem might also influence the effects of the intervention as demonstrated in this study.

We also explored the difference in size of the hospital and its relationship with intervention effects. The hospitals which did not show effect at 6-months appeared to be larger than hospitals which showed effect at 6 months, but some of these hospitals eventually showed effect at 12-month. However, the differences varied by province. In Yunnan, the average size of hospital for the pairs of hospitals that achieved statistical significant differences was statistically smaller than those that did not show effects. The possible interpretation is that it might require longer time for intervention messages to be diffused from POLs to their co-workers in larger hospitals. This finding implies that, for the implementation of a POL intervention, the critical mass of POLs should be taken into consideration and the number of opinion leaders should be determined based on the size of the venue. This is consistent to previous findings that the proportion of “adopters” in the network affects the degree of exposure to the message, and will consequently alter the process of message diffusion [43]. It is important to realize that facility-level interventions targeting social norms and personal behavior need to be implemented for sufficient amounts of time to have a sustainable impact in communities.

The study has limitations. The data were collected only from county-level hospitals in the two provinces, so the results might not be generalizable to other levels of hospitals or hospitals in other geographic areas. Also, the outcome measures relied entirely on self-reported data, so issues surrounding recall accuracy and social-desirability bias can be raised. The stigma variable used in this study did not measure all the dimensions that previous literature identified [44, 45]; with the focus on the prejudicial attitude towards PLH, it was still unclear to what extent it actually reflected the behavior change by providers in serving PLH. The number of pairs of hospitals was relatively small even though some of the associations between intervention effect and the hospital level characteristics reached statistical significance. There were some other implementation factors, such as the fidelity of the training and the enthusiasm of the POLs, which might be more important in mediating the intervention outcome, were not measured or controlled. This study, despite these limitations, explored alternative ways to demonstrate intervention efficacy in stigma reduction that can also be applied to HIV intervention studies in other countries and contexts.