Changes in clients' care ratings after HIV prevention training of hospital workers in Malawi

Abstract

Objective

To examine the changes in clients' health-care ratings before and after hospital workers received an HIV prevention intervention in Malawi, which increased the workers' personal and work-related HIV prevention knowledge, attitudes and preventive behaviors.

Design

Pre- and post-intervention client surveys.

Setting

A large urban referral hospital in Malawi.

Participants

Clients at purposefully selected inpatient and outpatient units on designated days (baseline, n = 310 clients; final, n = 683).

Intervention

Ten-session peer-group intervention for health workers focused on HIV transmission, personal and work-related prevention, treating clients and families respectfully and incorporating HIV-related teaching.

Main Outcome Measures

Brief face-to-face clients' interview obtaining ratings of confidentiality of HIV, whether HIV-related teaching occurred and ratings of service quality.

Results

Compared with baseline, at the final survey, clients reported higher confidence about confidentiality of clients' HIV status (83 vs. 75%, P < 0.01) and more clients reported that a health worker talked to them about HIV and AIDS (37 versus 28%, P< 0.01). More clients rated overall health services as ‘very good’ (five-item mean rating, 68 versus 59%, P< 0.01) and this was true for both inpatients and outpatients examined separately. However, there was no improvement in ratings of the courtesy of laboratory or pharmacy workers or of the adequacy of treatment instructions in the pharmacy.

Conclusions

HIV prevention training for health workers can have positive effects on clients' ratings of services, including HIV-related confidentiality and teaching, and should be scaled-up throughout Malawi and in other similar countries. Hospitals need to improve laboratory and pharmacy services.

As a result of the HIV epidemic, health services throughout Southern Africa must cope with increased patient loads in a context of inadequate staffing and resources [1, 2]. Consequently, health workers suffer from demoralization and burnout [3–5]. Several studies, including our own work in Malawi, have documented that health workers benefit from interventions to improve their capacities for HIV prevention and AIDS care [6–12]. Benefits include greater HIV-related knowledge, less stigmatization and other more positive attitudes, reduced personal risky behaviors and more use of universal precautions to prevent HIV and other infections in the workplace [6–12]. However, no previous publications have examined whether these improvements for health workers are reflected in more positive ratings of health care by clients. In this paper, we explore clients' ratings of the health services they received before and after health workers received an HIV prevention intervention.

Client ratings of health care in African countries

Clients' ratings of health services have become a routine part of health-care quality evaluation in industrialized countries [13] and are increasingly a focus of attention in developing countries [14–16]. Clients' satisfaction has been related to greater adherence, return for follow-up services and willingness-to-pay for services [17–23], and so clients' ratings are both an important outcome of care and a useful indicator of health facility performance [14, 16, 24]. More than 20 years ago, Donabedian identified three main components of a client's rating of health services: technical adequacy (e.g. evidence-based diagnosis and treatment); interpersonal relationship with providers (e.g. good communication and emotional support) and organizational factors (e.g. wait time, cleanliness and medication availability) [24].

These same dimensions of health services are important in clients' ratings of health services in Africa. Interpersonal relationship with providers was an important dimension of clients' satisfaction in all African studies [13, 15, 22, 25–40]. In a few studies, technical adequacy factors such as lack of doctors or failure to receive a physical examination were also important [17, 22]. Organizational factors such as cleanliness, long waits and lack of medicines were also important in most studies [22, 31, 34, 41–46]. Several studies found that higher client education and acuity of illness were related to less satisfaction with services [25, 28, 42, 47–49], but other studies found that patient characteristics did not affect satisfaction [40, 50]. The HIV epidemic has added a new concern regarding the quality of health care related to stigmatizing attitudes [51]. Several studies of health workers in African countries have found avoidance of HIV-related discussion, discriminatory care and inadequate protection of confidentiality [3–11, 51]. These practises are unethical and discourage clients from seeking counseling, early testing and treatment.

The relatively few published reports of interventions or quality improvement projects in Africa suggest that health worker interventions can improve clients' ratings of health services [52–55]. However, there have been no descriptions of the impact on clients' care ratings of interventions not focused primarily on service improvement. In this report, we describe clients' ratings of health services before and after a peer-group HIV prevention intervention for hospital workers in Malawi. In previous publications we have documented higher HIV-related knowledge, less stigmatizing attitudes and more observed use of universal precautions and teaching after health workers received the intervention [9, 10]. We hypothesized that these changes in health worker attitudes and behaviors should be reflected in more positive reports regarding confidentiality of HIV status, HIV-related teaching and overall care ratings by clients surveyed before and after the intervention.

Methods

This study used a quasi-experimental pre–post test design with no control group. Clients' surveys were conducted at baseline and after the 6-month intervention. We did not use a control group because we expected health workers to talk about the intervention with each other, potentially contaminating a same-site control group. We could have used a different referral hospital as a control, but the three other referral hospitals in Malawi differ in size, region and social and economic differences in the clients served. Thus, none of the referral hospitals was sufficiently comparable to the study site to be a suitable control site. More information about the site, sample and intervention are available in the two previous publications examining the effects of the intervention on the hospital workers [6, 10].

Site and sample

The study site was a large, urban referral facility providing inpatient and outpatient care with a full complement of specialty areas. The hospital work force is around 850, including clinical and non-clinical workers. There are ∼700 inpatient beds and 560 outpatients daily. The acuity level is high, especially for inpatients. The most recent estimation was that 40% of all hospitalized patients were HIV positive [56]. Clients come from all over the city, and more complex cases are referred from health centers in surrounding districts. Most clients come from low-income neighborhoods.

To obtain a diverse sample of clients, we purposefully sampled from all in-patient and outpatient units at various times and different days of the week. Inclusion criteria were as follows: (1) present at the designated site and time, and (2) either an adult who received outpatient services or current inpatient, the parent/guardian of a child (under 12) receiving health services or an adolescent with a parent present to provide permission. Exclusion criteria were appeared acutely ill, in pain or distressed; or receiving health worker services at that time. We invited all clients to participate who met the inclusion/exclusion criteria. The study was explained and oral consent obtained. The survey was anonymous and no clients' demographics were collected to ensure that clients felt free to express their opinion.

We interviewed 310 clients at baseline and 678 clients at the final evaluation. There were more client surveys at the final evaluation because units had more clients at the final observation, and interviewers were instructed to invite all the clients present to participate. We are not sure exactly why the difference was so great, but there are substantial variations in morbidity and patient volume, such as the seasonal increase in malaria cases during the rainy season. Discussions with hospital administrators confirmed that there were no changes in the demographic characteristics of clients or the catchment area the hospital served.

The health worker intervention

Participation in the intervention was voluntary, and nearly all of the staff chose to participate. A total of 855 hospital workers received the Mzake ndi Mzake (Friend-to-Friend) peer-group intervention. The 10-session intervention was guided by a framework integrating the World Health Organization primary health-care model [57], Bandura's social cognitive learning model [58] and a formative evaluation as described elsewhere [59, 60]. A pair of trained health workers facilitated the sessions, usually one volunteer hospital worker and the project nurse. The first six sessions, developed for use with community members as well as health workers, were about HIV and AIDS transmission and prevention. These sessions focused on replacing myths with correct knowledge, reducing stigmatizing attitudes and building self-efficacy and skills to practise safer sex and to engage in community prevention activities. Four sessions focused on health worker issues. Sessions 7 and 8 discussed AIDS treatment and universal precautions. Session 9 focused on providing empathetic care that respects human dignity and preserves confidentiality for clients, including persons living with HIV. Session 10 was about increasing teaching for patients and families about HIV prevention and AIDS care. Sessions 9 and 10 also emphasized reducing stigmatization and discriminatory treatment. All sessions incorporated active learning strategies to build self-efficacy for behavioral change, including role modeling, rehearsal through role plays and other skill-building exercises.

This intervention was relatively low cost to provide for workers. A cadre of volunteer trainers was established; they received the intervention followed by a 2-week training ingroup facilitation skills with return demonstrations and feedback. Each facilitator had a manual outlining content and activities for each session. At this facility, the workers attended the 90–120 min sessions weekly or bi-weekly toward the end of a shift or on break, so that about half the session was work time and half was donated by the workers. The sessions were staggered so that unit coverage was maintained. No special equipment or off-site training was needed.

Variables and operational measures

The survey was an adaptation of a client’s satisfaction survey used in the USA for prenatal clinic visits [61] which we used in our earlier rural health worker study [9, 59]. We asked about specific aspects of the experience to make it easier to express less positive opinions. We established content validity with Malawian clinical experts. Bilingual team members translated the survey into Chichewa, the local language, using the consensus translation method [62] to facilitate conceptual equivalency. Interviews were conducted in Chichewa.

We asked about two HIV prevention-related health worker behaviors. Confidentiality was rated using a single item, ‘how would you rate the confidentiality of clients' HIV status at the hospital?’ HIV-related teaching during the clinic visit or hospital stay was assessed by asking whether any health worker talked about HIV and AIDS in general, condom use or safer sex, and getting an HIV test. We examined each item and created a summary measure of whether one or more topics were discussed.

Clients rated five general aspects of their care that included elements of all three dimensions of Donabedian's elements of care. Two items assessed technical adequacy: health providers' information and explanations about medicine and treatment and how well the health provider took care of their health problem. Two items looked at interpersonal relations: courtesy of registration staff and how well the health provider treated them as a person. Because the intervention focused only on health workers, we asked only one item related to organizational factors, the cleanliness of the facility. For clients' ratings of care, the three response categories (very good, good and fair or poor) were very positively skewed. Therefore, when we created a summary index of the five general care items, we counted the number of ‘very good’ responses for each client to capture even slightly less positive ratings. We also asked those clients who received services at the pharmacy and the laboratory, the two most common ancillary services, to rate the courtesy of the staff and how well pharmacy staff explained medication, using the same three-category rating scale.

We asked hospitalized patients to rate separately four items about their experience as inpatients: how well health workers helped them get healthy, how courteous the health worker was to them and to their family and how well the health workers showed their family member how to take care of them. In Malawi, as in many countries, hospitalized patients are usually accompanied by a family member who provides some physical care both during hospitalization and at home. We examined the separate items and combined them into an index that summed the ‘very good’ responses.

Data collection

Approval was obtained from the University of Illinois at Chicago ethical review board, University of Malawi College of Medicine ethical review board and the study site hospital director. Intervention and evaluation activities were planned collaboratively with the hospital's administrators and AIDS committee. Interviews were conducted anonymously with verbal consent to reduce clients' discomfort when rating services. We conducted oral surveys because of low literacy levels.

The data collection team was trained in the consent process and interviewing using mock interviews with corrective feedback. We explained the study to clients in the designated areas, obtained consent and conducted the surveys. After the baseline, we provided the peer-group intervention for HIV prevention to all interested workers. We then conducted the final client survey at around 6.5 months after the peer-group interventions were completed, using the same procedures as at baseline.

Analysis

We first examined the primary provider and location (inpatient or outpatient) for clients surveyed at baseline and final evaluations, using cross-tabulations with χ² tests of significance. We examined individual items and summary measures descriptively using cross-tabulation and comparisons of means. We then tested our hypotheses that there would be more positive reports regarding confidentiality of HIV status, HIV-related teaching and overall care ratings by clients surveyed before and after the intervention. We used gamma tests of significance for the ordinal individual items. For the summary measures of multiple items, we used t-tests, which are appropriate for these ordinal mean scores. Because there were different proportions of inpatient and outpatient clients at the baseline and final surveys, we also examined satisfaction ratings for these two groups separately.

Results

At both baseline and final surveys, 53% of the clients said that their main health provider was a nurse and a third (baseline 33%; final 35%) said their main provider was a clinical officer. Other providers included physicians and half were pharmacy, laboratory or X-ray technicians. There was no significant difference in the type of provider reported between the baseline and final client surveys.

At baseline, 67.7% of the clients were inpatients and at the final survey, 57.1% were inpatients. This difference was significant (χ² = 10.1; d.f. 1, P< 0.01), so we analyzed responses of inpatients and outpatients separately as well as the total of all clients.

HIV-related issues

We examined client confidence regarding confidentiality of clients' HIV status and whether providers discussed HIV-related topics in the health-care setting. At baseline, 74.8% of clients rated health workers as ‘very good’ in maintaining confidentiality of HIV status. After hospital staff received the intervention, rating of confidentiality were significantly higher (83%, P< 0.01). More hospitalized patients rated confidentiality as very good at the final evaluation (87%) than the baseline (76%). Among outpatients, 66% rating confidentiality as ‘very good’ at baseline and 76% rated confidentiality as ‘very good’ at the final evaluation, a difference that was not statistically significant.

Only 28% of participants reported that a health worker discussed at least one HIV or AIDS topic at baseline (Table 1). After health workers received the HIV prevention intervention, discussion increased significantly to 37%. At baseline only 2% of clients reported that their provider discussed HIV and AIDS and only 2.6% said their provider discussed condoms or HIV prevention. At the final evaluation, 24.7% reported discussion of HIV and AIDS and 23.7% reported discussion of condoms or HIV prevention. Discussion of HIV testing did not increase.

Table 1.

Percent of patients reporting HIV-related teaching

	Baseline (n)	Final (n)	P*
All patients	310	683
Health worker discussed at least one HIV/AIDS topic (a–c), %	28.1	37.3	0.003
(a) Talking about HIV/AIDS (% yes)	2.0	24.7	0.000
(b) Talking about condoms or HIV prevention (% yes)	2.6	23.7	0.000
(c) Talking about getting an HIV test (% yes)	27.6	29.6	0.538
Hospitalized inpatients	210	390
Health worker discussed at least one HIV/AIDS topic (a–c), %	29.1	33.9	0.225
(a) Talking about HIV/AIDS (% yes)	1.9	22.3	0.000
(b) Talking about condoms or HIV prevention (% yes)	3.4	19.1	0.000
(c) Talking about getting an HIV test (% yes)	28.9	26.6	0.539
Clinic outpatients	100	293
Health worker discussed at least one HIV/AIDS topic (a–c), %	26.0	41.9	0.003
(a) Talking about HIV/AIDS (% yes)	2.0	28.0	0.000
(b) Talking about condoms or HIV prevention (% yes)	1.0	29.9	0.000
(c) Talking about getting an HIV test (% yes)	25.0	33.6	0.099

*P for t-test comparing baseline and final.

Among inpatients, there were no significant differences between baseline and post-intervention in reported health worker discussion of HIV-related topics. However, among outpatients the proportion who reported that a health worker discussed an HIV-related topic increased significantly from 26% at baseline to 42% after the intervention. Both inpatients and outpatients reported significantly more discussion about HIV and AIDS and condom use at the final evaluation but little difference in HIV-testing discussion.

Clients' ratings of health-care services

Overall care ratings (mean for five items) were significantly higher at the final evaluation than the baseline for the total group and for both inpatient and outpatient clients (Table 2). Except for one item, whether treated respectfully, individual ratings also were significantly higher at the final evaluation.

Table 2.

Patients rating health-care services

	Baseline (n)	Final (n)	P
Summary rating—mean % very good on five items (a–e)
All patients	59.0 (310)	67.6 (683)	0.000*
Hospitalized inpatients	61.4 (210)	68.9 (390)	0.009*
Clinic outpatients	54.2 (100)	65.9 (293)	0.002*
Individual items
All patients
(a) Information about medicine and treatment
Fair or poor	18.3	13.0	0.020**
Good	21.4	19.1
Very good	60.3 (290)	67.9 (614)
(b) Taking good care of their health problem
Fair or poor	20.2	10.3	0.001**
Good	20.8	21.9
Very good	59.0 (307)	67.8 (671)
(c) Courtesy of registration staff
Fair or poor	12.1	3.5	0.004**
Good	15.3	16.5
Very good	72.6 (307)	80.0 (656)
(d) Provider treating them respectfully as a person
Fair or poor	11.0	6.0	0.063**
Good	23.3	23.3
Very good	65.7 (309)	70.7 (682)
(e) Cleanliness of the clinic
Fair or poor	18.3	13.1	0.000**
Good	43.1	35.1
Very good	38.6 (306)	51.8 (681)
Hospitalized inpatients
(a) Information about medicine and treatment
Fair or poor	12.8	15.0	0.911**
Good	20.4	16.7
Very good	66.8 (196)	68.2 (359)
(b) Taking good care of their health problem
Fair or poor	17.3	10.0	0.005**
Good	22.1	18.5
Very good	60.6 (208)	71.5 (389)
(c) Courtesy of registration staff
Fair or poor	12.1	4.1	0.012**
Good	13.5	13.3
Very good	74.4 (207)	82.6 (368)
(d) Provider treating them respectfully as a person
Fair or poor	9.1	6.9	0.178**
Good	25.4	22.3
Very good	65.6 (209)	70.8 (390)
(e) Cleanliness of the clinic
Fair or poor	17.8	12.9	0.011**
Good	41.8	36.2
Very good	40.4 (208)	50.9 (389)
Clinic outpatients
(a) Information about medicine and treatment
Fair or poor	29.8	10.2	0.000**
Good	23.4	22.4
Very good	46.8 (94)	67.5 (255)
(b) Taking good care of their health problem
Fair or poor	26.3	10.6	0.051**
Good	18.2	26.6
Very good	55.8 (99)	62.8 (389)
(c) Courtesy of registration staff
Fair or poor	12.0	2.8	0.078**
Good	19.0	20.5
Very good	69.0 (100)	76.7 (288)
(d) Provider treating them respectfully as a person
Fair or poor	15.0	4.8	0.203**
Good	19.0	24.7
Very good	66.0 (100)	70.5 (292)
(e) Cleanliness of the clinic
Fair or poor	19.4	13.4	0.002**
Good	45.9	33.6
Very good	34.7 (98)	53.1 (292)

*P for t-test comparing baseline and final; **P for gamma comparing baseline and final.

We also asked clients who used laboratory and pharmacy services about satisfaction with these services. The number of cases was smaller, so we did not examine inpatient and outpatient ratings separately. Ratings were lower after the intervention than at baseline for courtesy of laboratory staff (74 vs. 67%) and pharmacy staff (66 vs. 55%) and how well the pharmacy staff explained medications (74 vs. 64%), although differences were not significant.

Most inpatients experienced both a clinic visit and subsequent hospitalization. We asked them to rate their in-hospital experience separately. Overall ratings of inpatient services and each of the individual items were significantly higher at the final evaluation than the baseline (Table 3).

Table 3.

Hospitalized inpatients rating health workers

	Baseline (n= 208)	Final (n= 390)	P
Summary rating—mean % very good on four items	60.0	68.6	0.013*
Individual items
Health worker helped to get healthy
Fair or poor	16.9	8.7	0.001**
Good	23.7	19.0
Very good	59.4	72.3
Courtesy of health worker
Fair or poor	9.2	4.1	0.031**
Good	26.6	23.8
Very good	64.3	72.1
Showing family/guardian how to care
Fair or poor	19.7	26.3	0.167**
Good	24.5	22.0
Very good	55.9	61.8
Courteous treatment of family
Fair or poor	13.8	10.7	0.230**
Good	25.4	23.6
Very good	60.8	65.7

*P for t-test comparing baseline and final; **P for gamma comparing baseline and final.

Discussion

Client surveys before and after health workers in a large hospital received a peer-group intervention for HIV prevention showed higher clients' confidence about the confidentiality of their HIV status, more reported discussion with a health worker about HIV-related topics and higher ratings of general health services care for both outpatient and inpatient services at the final evaluation than at baseline. These client reports were congruent with the intervention's emphasis on greater teaching, less stigmatization, ethical conduct related to HIV status of clients, more courtesy and greater universal precaution use. The increased emphasis on universal precautions may have resulted in greater visible cleanliness of the facility. These improved ratings of services occurred despite the absence of system-level interventions such as changes in staffing levels, supplies or working conditions.

However, this increased satisfaction did not apply to laboratory and pharmacy services. These services are especially crowded, and the additional waiting time after already waiting to be seen in the clinic may be especially frustrating to clients. There is little privacy, and staff feel a great deal of pressure to keep up with demand, inhibiting discussion of procedures and medications with clients.

This study's results are congruent with the few previous published reports of health worker interventions or quality improvement projects in Africa. Only two prior studies related to HIV. In Eritrea, an infection prevention intervention increased staff knowledge and hand washing as reported by both providers and patients [52]. In Botswana, training for ‘opt-out’ HIV testing increased patient ratings of their care and reports of receiving HIV-risk information, HIV testing and a physical examination [53]. Quality improvement initiatives in Egypt and Nigeria that included health worker training also increased clients' perceptions of the quality of care, including polite reception, decreased wait time and health worker competence and attitudes [54, 55]. All these interventions included a health systems component, so improvements may not have been due only to health worker training.

Several other previous quality improvement projects in Africa resulted in higher clients' ratings of care but either did not include a health worker training component or did not provide details about the intervention [30, 63–65] In Malawi, a quality improvement project shared client audit-based recommendations with selected maternity care providers who shared this with all workers in their units, resulting in significantly improved client greetings, respect, cleanliness, using simple language, privacy and overall client's satisfaction [30]. The ‘Health Workers for Change’ intervention supported by WHO provided participatory workshops to empower health workers to improve services [63]. This model was tested in seven different primary health-care settings, including Nigeria, Tanzania, Ghana and Kenya, resulting in reduced waiting time at five sites and improved client–staff interactions at four sites [64]. However, in Tanzania the intervention did not improve provider–client relationships, which the authors related to unsatisfactory working conditions that remained unchanged [65].

In this study, outpatients reported less confidentiality of HIV status than inpatients. This may reflect the lower availability of private space in the outpatient setting. In the hospital, treatment rooms are available for confidential discussion with clients. No previous studies in Africa have compared ratings of services by outpatients and inpatients.

There were several limitations for this study. The study lacked a control group. The sample size and the proportion of hospitalized and outpatient clients differed at the baseline and post-intervention surveys, although the units surveyed, selection criteria and instructions to interviewers were the same at both times. Hospital administrators did not identify any differences in clients' characteristics, but seasonal variations in morbidity may have contributed to these differences. Also, our interviewers had never done this type of client survey before and may have been more successful recruiters at the final evaluation. We addressed difference in hospitalized and outpatient clients by examining client reports separately for the two groups. However, without a control group, we cannot identify to what extent nationwide changes in HIV awareness, testing and drug treatment may have affected our results. Another limitation is the lack of follow-up beyond the final survey ∼6.5 months after the intervention. The clients surveyed at baseline and follow-up were not the same persons.

Another set of limitations has to do with the nature of clients' care ratings. Clients tend to rate health services positively, as noted in this study and prior research [14], and ratings are influenced by cultural and socioeconomic variations in client expectations and preferences [14–16] and lack of expertise when rating technical adequacy [13–15, 24]. No standardized client satisfaction questionnaire previously used in African settings was available. Despite all these limitations, clients' ratings were useful in identifying the effect of the intervention on clients and areas where further improvement is needed.

Implications

This study has shown that a peer-group intervention for HIV prevention for health workers can improve clients' satisfaction with services, an important benefit not reported in previous studies. The intervention provided was relatively low cost. However, continued improvement in clients' services is needed, especially in the area of pharmacy and laboratory services. Also, <40% of clients reported having any HIV-related discussion with a provider. It seems likely that there are many more clients who would benefit from such discussion.

This intervention for individual workers might be made more effective by increasing content and skill building related to service delivery. Including an organizational change component may also increase the effectiveness of interventions to improve clients' ratings of health services. Organization-level changes feasible within the resource constraints of countries such as Malawi include the development of guidelines regarding HIV teaching and ensuring compliance with existing national policies supporting patient confidentiality and non-discriminatory care.

Funding

The World AIDS Foundation funded this research. The interpretations and conclusions are those of the authors and do not represent the position of the World AIDS Foundation. Earlier funding from the National Institute of Nursing Research, NIH [NR08058] supported the development of the health worker intervention and data collection instruments and a Fulbright African Regional Research Fellowship and sabbatical support from the University of Illinois at Chicago to Dr K. Norr in 1999–2000 provided partial support for initial development of the Mzake ndi Mzake intervention.