Preliminary Program Evaluation of Emergency Department HIV Prevention Counseling

Abstract

Objective

Controversy surrounds the linkage of prevention counseling with emergency department (ED)–based HIV testing. Further, the effectiveness and feasibility of prevention counseling in the ED setting is unknown. We investigate these issues by conducting a preliminarily exploration of several related aspects of our ED's HIV prevention counseling and testing program.

Methods

Our urban, academic ED provides formal client-centered prevention counseling in conjunction with HIV testing. Five descriptive, exploratory observations were conducted, involving surveys and analysis of electronic medical records and programmatic data focused on (1) patient perception and feasibility of prevention counseling in the ED, (2) patient perceptions of the need to link prevention counseling with testing, and (3) potential effectiveness of providing prevention counseling in conjunction with ED-based HIV testing.

Results

Of 110 ED patients surveyed after prevention counseling and testing, 98% believed privacy was adequate, and 97% reported that their questions were answered. Patients stated that counseling would lead to improved health (80%), behavioral changes (72%), follow-up testing (77%), and discussion with partners (74%). However, 89% would accept testing without counseling, 32% were willing to seek counseling elsewhere, and 26% preferred not to receive the counseling. Correct responses to a 16-question knowledge quiz increased by 1.6 after counseling (95% confidence interval 1.3 to 12.0). The program completed counseling for 97% of patients tested; however, 6% of patients had difficulty recalling the encounter and 13% denied received testing. Among patients undergoing repeated testing, there was no consistent change in self-reported risk behaviors.

Conclusion

Participants in the ED prevention counseling and testing program considered counseling acceptable and useful, though not required. Given adequate resources, prevention counseling can be provided in the ED, but it is unlikely that all patients benefit.

Introduction

Background

Historically, “client-centered” prevention counseling was recommended as a necessary adjunct to HIV testing.¹ This lengthy, individualized process augments provision of information or education² to include detailed risk assessment, identifying behaviors that place the patient at risk, and development of a negotiated risk reduction plan. Newly revised Centers for Disease Control and Prevention (CDC) recommendations for HIV screening in health care settings uncouple prevention counseling from testing,³ partly because of debate over counseling efficacy for patients receiving negative test results^4-7 and partly because resource requirements to conduct such extensive counseling may inhibit uptake of screening.^8-11 Given the significant influence of screening,¹² delinking prevention counseling from testing is consistent with the intent to remove exceptional requirements that slow screening efforts and further stigmatize HIV.^3,13,14

Despite momentum away from prevention counseling, controversy remains.^7,5 Although the CDC recommendations delink counseling from screening, they do not explicitly recommend against prevention counseling, particularly for individuals at high risk.³ Studies have shown that prevention counseling can reduce risky behaviors,¹⁵ and proponents suggest it should not be discarded.^6,16-18 Some argue that failure to provide prevention counseling with testing withholds a service with demonstrated benefit,⁶ with potential harm resulting (ie, false reassurance).¹⁹ Additional controversy is fueled by ambiguity in terminology.² For example, “counseling” in the context of HIV screening often denotes provision of information or education more limited in scope than formal prevention counseling,^2,20,21 yet many practitioners perceive that time, resource, and training requirements for prevention counseling still apply.

Importance

Even if prevention counseling can be highly efficacious, its effectiveness in the emergency department (ED) setting is not well understood. Although the ED appears to be a beneficial venue for screening, it may not be an optimal venue for prevention counseling. Moreover, prevention counseling in the ED might be effective for some patients and not others.

Goals of This Investigation

The HIV testing program in our ED is required to provide client-centered prevention counseling in conjunction with testing to receive funding from the state health department. We conducted a complementary series of investigations, using survey and chart review methodology to explore (1) patient perception and feasibility of prevention counseling in the ED, (2) patient perceptions of the need to link prevention counseling with testing, and (3) potential effectiveness of prevention counseling provided in conjunction with ED-based HIV testing.

Methods

Study Design

This preliminary program evaluation used a mixed-methods approach involving retrospective review of programmatic and clinical data and prospective patient survey. Five complementary, descriptive studies were conducted on an exploratory basis to inform 3 study questions:

1. Is the ED setting an acceptable venue for prevention counseling?

   • Study 1: Survey of patient perceptions about counseling and intent to change behavior

   • Study 5: Review of programmatic quality assurance data on operational metrics

2. Do patients perceive a need to couple prevention counseling to screening?

   • Study 1: Survey of patient perceptions about counseling and intent to change behavior

3. Does the prevention counseling provided by our program affect behavior and knowledge?

   • Study 1: Survey of patient perceptions about counseling and intent to change behavior

   • Study 2: Survey to assess patient knowledge before and after counseling

   • Study 3: Review of clinical program data to assess short-term recall of testing and counseling encounter

   • Study 4: Review of clinical program data to assess long-term change in behaviors self-reported during the counseling risk-assessment

All studies were reviewed and approved by the University of Cincinnati's institutional review board.

Setting

The HIV counseling and testing program operates in the ED of an urban, tertiary referral hospital. The annual ED census is approximately 85,000 patients, of whom 57% are black, 39% are white, and 0.5% are Latino; the local county population is composed of more than 800,000, of whom approximately 25% are black, 72% white, and 1.5 Latino.²² Almost all ED patients are aged 18 years or older, and there is an adjacent pediatric hospital.

The HIV counseling and testing program has been described elsewhere.²³ Briefly, trained counselors provided targeted, opt-in HIV counseling and testing to ED patients. The counseling staff included individuals with previous relevant work experience. Counselors were available from 8 AM to 12 PM from 2002 to 2006 and 24 hours per day after 2006. Patients were offered testing as a result of self-request, when clinical staff identified symptoms suggestive of HIV, or when clinical staff or counselors identified factors associated with increased risk of undiagnosed HIV infection. Before October 2008, conventional HIV testing was used; blood samples were analyzed off site and patients were called 7 to 10 days later with their results. In October 2008, rapid HIV testing was implemented. The program's electronic medical record includes process information, self-reported patient behaviors, and limited outcomes from the testing and counseling.²³

In conjunction with testing, counselors provided formal risk-assessment and prevention counseling with a structured, questionnaire-driven interview as a guide. Counselors sought to promote individualized, achievable plans for risk reduction. Sessions were planned to take about 30 minutes. Counselors were trained for several days in class according to the CDC guidelines for client-centered HIV prevention counseling. The training involved role playing, HIV education, and instruction in counseling methods. Then, counselors spent approximately 1 month shadowing experienced counselors. The program coordinator evaluated counselor performance by direct observation approximately every 6 months. Throughout the article, the term counseling refers to this approach to prevention counseling.

Overview of Studies

The 5 exploratory, descriptive studies included prospective patient surveys to assess patient perceptions; retrospective evaluation of programmatic data, including patient recall about their counseling and testing encounter; change in self-reported HIV risk behavior between visits; a description of the proportion of tested patients receiving counseling as intended; and the duration of counseling provided.

Study 1: Patient perceptions about counseling and intent to change behavior

Between April 2009 and May 2009, 110 patients were surveyed after undergoing a rapid HIV test, after completion of pre- and posttest counseling. Areas of query included perceptions of HIV counseling, the ED setting, and linkage of counseling to testing, and intention and readiness to change risk behaviors. To avoid extremes of sample heterogeneity, patients were excluded if they were pregnant, incarcerated, younger than 18 years, or rapid HIV reactive. The survey consisted of 22 questions (Appendix E1, available online at http://www.annemergmed.com). Though the survey was not fully validated, review from the first 10 questionnaires found responses to be internally consistent, with no unexpected difficulties. The survey took approximately 5 to 10 minutes to complete.

Study 2: Patient knowledge before and after prevention counseling

Between April 2009 and May 2009, a separate convenience sample of 100 patients were asked questions about their knowledge of HIV before and after undergoing HIV counseling and testing. Patients were not told about the pending retest before counseling. Questions were adapted from Merchant et al²⁴ and read verbatim. Exclusions were the same as in study 1 described above. Knowledge scores were calculated by summing the number of correct responses to the 16 questions. The survey took approximately 5 minutes to complete.

Study 3: Short-term recall about counseling and testing encounter

Between January 2006 and November 2006, conventional testing was conducted, with delayed result notification. Negative results were provided by telephone approximately 1 week after HIV testing; about 75% of tested patients were successfully reached. As part of the telephone encounter, patients were asked the open-ended question, “What do you remember being told about HIV when you were in the ED?” Counselors were instructed to record the exact response, though some paraphrasing likely occurred. Two coders categorized responses as (1) no response, (2) specific denial of having received testing or denial of having been given any HIV-related information, (3) response totally unrelated to HIV, or (4) any interpretable response related to HIV that did not specifically denote denial of counseling or testing. Responses related to HIV were additionally categorized as (1) being asked questions about HIV risks, (2) receipt of HIV information or discussion, (3) negotiated risk reduction plan or specific plan to change behavior, (4) referrals for related or follow-up services. Discrepancies between reviewers were adjudicated by a third reviewer.

Study 4: Long-term change in behaviors self-reported during counseling risk—assessment

Electronic program records from January 2003 to December 2007 were reviewed to identify patients with multiple visits that occurred at least 1 year apart. Only patients with more than 1 visit during the study period were included to assess change over time. Changes in self-reported risk behavior were ascertained. An individual was designated as engaging in a behavior if he or she participated in the activity at least once in the previous year. Given the complexity and interaction between various behaviors, we explored the proportion of patients who began or ended each activity and did not attempt to quantify overall behavior. For sex behaviors, change was also determined to occur if the frequency of condom use varied between never being used, sometimes being used, and always being used.

Study 5: Review of programmatic quality assurance data on operational metrics

Metrics describing program operation were reviewed for March 1, 2008, through December 31, 2008. There were no significant changes in personnel or program operations during that period. The proportion of patients who received counseling as intended, in addition to the duration of the counseling sessions, was extracted. The duration of counseling was obtained only for those patients for whom counseling was received as intended. Patients were considered to have received counseling as intended as long as at least three quarters of the risk questionnaire was completed.

Data Analysis

Data were managed with Microsoft Access (version 2003, Microsoft, Redmond, WA). Analyses were conducted with SPSS for Windows (versions 16.0 and 17.0; SPSS, Inc., Chicago, IL). Changes were assessed by estimating differences in proportions or scores, and 95% confidence intervals (CIs) of the differences are given. Where agreement was assessed, Cohen's κ was used.

Results

Study 1: Perceptions About Counseling and Intent to Change Behavior

One hundred ten patients were included in the survey; median age was 29 years (range 18 to 62 years), 46% were women, and 64% were nonwhite. Previous HIV testing was reported by 71% of participants, and 22% reported previous testing in an ED. Patients generally perceived that counseling improves health (80%), and more than 95% of patients perceived the ED environment as adequate, with sufficient privacy and ability to ask all questions and have them answered. Nine of 10 respondents indicated a willingness to be tested, even if the test were provided without counseling, and 26% preferred testing to be provided without discussion of HIV risk and prevention. About one third (32%) said they would be willing to go elsewhere to receive prevention counseling after an ED encounter that included testing only. The majority of patients stated an intention to alter their behavior (Table).

Table.

Counseling perceptions and intended behavior change.^*

Question	Yes, %	No, %	Not Sure, %	Decline/NA, %
Intentions
Plan on making changes as a result of conversation today	71.8	23.6	3.6	0.9
Plan to get another HIV test in 3-6 months	76.4	12.7	10.0	—
Talk to sex or needle-sharing partners about HIV	73.6	15.5	5.5	5.5
Perceptions of HIV counseling
Talking about HIV will improve your health	80.0	10.9	8.2	0.9
Still take test if no discussion, only test	89.1	3.6	7.3	—
Preferred test without discussion of HIV risk and prevention	25.5	57.3	17.3	—
Go somewhere else for talk about HIV risk and prevention	31.8	52.7	15.5	—
ED role
Comfort with privacy	98.2	1.8	—	—
Able to ask all questions and receive answers about HIV	97.3	0.9	0.9	0.9
Tested for HIV before	71.8	28.2	—	—
Tested in ED before	22.7	34.5	1.8	25.5

^*Responses to a patient survey of perceptions of ED HIV counseling, testing, and intent to change behavior administered at completion of ED HIV counseling and testing encounter.

Study 2: Patient Knowledge Before and After Prevention Counseling

There were 100 participants; median age was 28 years (range 18 to 53 years), 44% were women, and 74% were nonwhite. The mean precounseling knowledge score was 10.34 (range 5 to 15) and the mean postcounseling score was 11.96 (range 4 to 16), with an increase of 1.62 (95% CI 1.26 to 1.97). The proportion of correct responses to each question and change in the proportion of correct responses before and after counseling are shown in Table E1 (available online at http://www.annemergmed.com).

Study 3: Short-Term Recall About Counseling and Testing Encounter

There were 3,210 patients contacted by telephone and provided with a negative test result during the study period. Recall responses were recorded for 1,510 patients (47%); data were missing because the question was not asked, the patient did not respond to the question, or the counselor did not record the response. Mean age was 30 years (SD 10.8 years), 47% were men, and 70% were black. Unprotected sex and multiple partners were reported by 51%, 6% were men having sex with men, and 7% were intravenous drug users. Agreement in coding responses was high (93.6% for primary categories; κ 0.816; 95% CI 0.779 to 0.853). Overall, 1,189 (78.7%) respondents remembered something about being counseled or tested, 199 (13.2%) denied being tested for HIV or did not recall the encounter, and 122 (8.1%) did not provide an interpretable response related to HIV. For individuals with some recollection of the encounter, 39 (3.3%) mentioned questioning, 1,154 (97.1%) recalled some element of education or counseling, 5 (0.4%) recounted a specific action plan, and 15 (1.3%) indicated that they were provided with a referral.

Study 4: Long-Term Change in Behaviors Self-reported During Counseling Risk

Assessment

There were 7,898 counseling sessions during the study period. There were 335 patients with 2 counseling sessions at least 1 year apart with risk data available. The mean age was 30 years, 51% were men, and 84% were black. A decrease in risk behavior was observed for sex with an HIV-positive partner (Table E2, available online at http://www.annemergmed.com). There were no other consistent changes in risk behavior, with some patients demonstrating increased risk and others demonstrating decreased risk.

Study 5: Review of Programmatic Quality Assurance Data on Operational Metrics

There were 3,918 consents for HIV testing and counseling in 2008, of which all but 107 (2.7%) patient encounters had at least 3 of 4 pages completed. Therefore, the counseling process was considered complete for 2,989 cases (97%). For these patients, the mean time spent with a patient was 25.5 minutes.

Limitations

Our findings should be tempered by several limitations. Most important, the design of our study was descriptive and exploratory, rather than a structured investigation directed to answer a specific study question. It is possible that our findings are biased by the particular format of the program when measurements were obtained. However, core principles of the program, particularly with respect to the prevention counseling provided, remained relatively constant since inception. In addition, although program data were collected prospectively in a structured format, they were gathered for clinical, not research, purposes. Accordingly, measures of service delivery are crude and do not allow rigorous determination of quality of service or consistency of delivery. Our assessment of behavioral outcomes is also limited because of use of self-report and exclusion of the majority of patients who were not treated repeatedly. However, if prevention counseling led to a reduced need for repeated testing for some patients, our results would be biased against the effectiveness of prevention counseling.

Our prospective surveys were small, were not validated, and did not include patients who did not consent to testing. It is possible that patients declined testing because of concerns over the ED environment. However, previous work suggests this is not likely.^23,25 We also did not survey patients who were pregnant, incarcerated, or younger than 18 years or who received a reactive HIV test result.

Although our study intended to explore the appropriateness of delinking counseling and testing, the 2 were linked in the program under evaluation. As such, we do not know to what extent our findings would be applicable to prevention counseling provided separately from testing. Response to counselor-administered questionnaires may also have been limited by social desirability, recall bias, and individual counselor variation. Finally, several questions refer to opinions, predicted actions, and theoretical scenarios, which may not reliably reflect how patients would actually conduct themselves.

Discussion

Overall, our results demonstrate that many patients find prevention counseling valuable and perceive the ED to be an adequate setting to receive this service. Furthermore, many patients show improved knowledge after counseling, are ready to consider behavioral change, and report intentions for specific actions as a result of the counseling. It is unclear how patients would receive prevention counseling if it were not offered in the ED because our study suggests that most patients would not choose to access such services. Among patients receiving repeated targeted screening, there was no trend toward reductions in self-reported behaviors, raising concerns about provision of nonselective prevention counseling for patients tested in the ED. Indeed, some patients do not even recall the counseling and testing encounter, and others specifically prefer testing without counseling.

Ability to Perform Prevention Counseling in the ED

The ED is a challenging environment in which to offer prevention counseling. However, the findings that counselors spent approximately 25 minutes with patients and almost all sessions were completed suggest that, where resources are sufficient, it is possible to overcome the intrinsic barriers to delivering prevention counseling. However, these process measures do not indicate the adequacy of the ED environment for sensitive risk-assessment and behavioral intervention; notably, patients who were uncomfortable with the setting may have been less likely to consent to testing and thus would not be included in our survey. Our findings must also be placed in the context of our overall screening program, which offers testing to only a minority of ED patients. Further research is needed to better understand the public health tradeoff between risk behavior counseling and widespread testing and identification of HIV.

Linking Prevention Counseling and Testing

The importance of prevention counseling relative to screening remains controversial. The recent focus on screening and reduction of barriers to screening in health care settings has outpaced emphasis on behavioral prevention efforts. Screening without counseling may be a missed opportunity to provide prevention services.^6,15,17,18 However, requirements for counseling may create prohibitive barriers that prevent screening altogether, particularly in EDs in which resources are scarce.³ The majority of patients in our program would still consent to testing for HIV if counseling were not provided, and some would rather not receive prevention counseling, supporting the call for a more selected application of prevention counseling than that used for screening.

Potential for Effectiveness of Prevention Counseling

We found that some patients are extremely unlikely to have benefited from prevention counseling. A small but significant minority seemed not to recall any aspects of the encounter and even denied that they were tested. For example, patient responses included “I was heavily sedated, and I don't remember much,” “I can't remember; I was really sick,” “I can't remember, stressful situation,” and “I can't remember; I had a concussion.” For such patients, it is highly unlikely that a behavioral change would result from the ED risk assessment and behavioral intervention. Conversely, many reported intent to change behavior, receive subsequent testing, or talk with partners about risk. In our qualitative assessment of patient recall during result notification, some patients indicated that they had been significantly affected by the encounter. For example, patient responses included “told all my friends to get tested,” and “she gave me a lot of information that I'm gonna take heed to.” Further, we found increased knowledge as a result of counseling. If even a minority follow through on plans to change, or act on their new knowledge, the public health influence could be significant.

We assessed changes in self-reported behavior for that subset of patients who were tested by our program repeatedly. There was no consistent indication of decreased risk among repeated testers, with roughly equal proportions increasing, decreasing, or remaining static in their self-reported behavior. This may be in part because patients were more likely to receive repeated testing specifically because they had ongoing risk. Thus, although our findings do not rule out the possibility that prevention counseling changes behavior for some patients, they are suggestive that effects are small to negligible for others.

Conclusion

With sufficient resources, it is possible to provide prevention counseling in association with a large-scale ED testing program. In general, patients are receptive to prevention counseling in the ED and consider it useful. Most patients are willing to accept a separation of counseling from testing but would be unwilling to seek counseling elsewhere. Measures of potential counseling effectiveness were mixed. Although measures of knowledge about HIV risk behavior and self-reported intent to change behavior increased overall, counseling was unlikely to be beneficial for all patients, with some patients having no recall of the encounter and a significant minority preferring to receive testing only. This suggests that prevention counseling should not be a requirement of ED-based testing, even if resources are sufficient to provide it. Although our findings support delinking of counseling and testing, they also suggest that a lack of prevention counseling for a subset of ED encounters could represent a missed opportunity for beneficial intervention.

Supplementary Material

Table E1. Knowledge quiz responses before and after counseling.

Table E2. Change in behavior for patients undergoing repeated testing in an ED-based HIV counseling and testing program.

Appendix E1

Survey about ED HIV counseling and testing.

Intentions

1. Do you plan on making any changes in your life as a result of our conversation today?

   • □ Yes □ No □ Not Sure □ declines to answer

     • 1b) If yes, what changes do you plan to make, what is your plan? (record exact wording)

     • □ declines to answer

2. Why do you (not) want to make changes? (record exact wording)

   • □ declines to answer

3. Do you plan to get another HIV test in the next 3 to 6 months?

   • □ Yes □ No □ Not Sure □ declines to answer

4. Are you going to talk to your sex or needle-sharing partners about HIV?

   • □ Yes □ No □ Not Sure □ declines to answer □ not applicable

Perceptions of HIV Counseling

• 5) Did you think that talking today about HIV will help you to improve your health?

  • □ Yes □ No □ Not Sure □ declines to answer

• 6) If we did not discuss HIV risk and prevention and only provided information about the test itself, would you still have taken the HIV test?

  • □ Yes □ No □ Not Sure □ declines to answer

• 7) Would you have preferred to get the test without our longer discussion about HIV risk and prevention?

  • □ Yes □ No □ Not Sure □ declines to answer

• 8) If we had given you the HIV test today without any information and then asked you to go somewhere else to talk about HIV risk and prevention, would you have done this?

  • □ Yes □ No □ Not Sure □ declines to answer

ED Role

• 9) Were you comfortable with your privacy when you were talking about HIV?

  • □ Yes □ No □ Not Sure □ declines to answer

• 10) Were you able to ask all of your questions about HIV and have them answered?

  • □ Yes □ No □ Not Sure □ declines to answer

• 11) Have been tested for HIV before?

  • □ Yes □ No □ Not Sure □ declines to answer

    • If yes

    • 11a) was this in the ED

      • □ Yes □ No □ Not Sure □ declines to answer

Readiness to Change²⁶

I am now going to read a series of statements as if you were saying them. Please tell me if you think they are true or false for you.

• 1) I don't think I am at risk for HIV.

  • □ True □ False □ declines to answer

• 2) I am trying to use methods that reduce my risk for HIV.

  • □ True □ False □ declines to answer

• 3) Sometimes I think I need to better protect myself from getting HIV.

  • □ True □ False □ declines to answer

• 4) It is a waste of time worrying about getting HIV.

  • □ True □ False □ declines to answer

• 5) I have just recently changed my sexual and or drug habits (using condoms, getting tested, etc.).

  • □ True □ False □ declines to answer

• 6) Anyone can talk about not wanting to get HIV, but I am actually doing something about it.

  • □ True □ False □ declines to answer

• 7) I am at the point where I should think about protecting myself from getting HIV.

  • □ True □ False □ declines to answer

• 8) Not protecting myself from HIV is a problem.

  • □ True □ False □ declines to answer

• 9) There is no need for me to protect myself from HIV.

  • □ True □ False □ declines to answer

• 10) I am actually changing my HIV prevention methods now.

  • □ True □ False □ declines to answer

• 11) Ways to reduce my risk for HIV would be pointless for me.

  • □ True □ False □ declines to answer

Survey about Ed Hiv Counseling and Testing

Intentions

• 1) Do you plan on making any changes in your life as a result of our conversation today:

  • □ Yes □ No □ Not Sure □ declines to answer

• 2) Why do you (not) want to make changes? (record exact wording)

  • □ declines to answer

• 1b) If yes, what changes do you plan to make, what is your plan? (record exact wording)

  • □ declines to answer

• 3) Do you plan to get another HIV test in the next 3 to 6 months?

  • □ Yes □ No □ Not Sure □ declines to answer

• 4) Are you going to talk to your sex or needle sharing partners about HIV?

  • □ Yes □ No □ Not Sure□ declines to answer □ not applicable

Perceptions of HIV Counseling

• 5) Did you think that talking today about HIV will help you to improve your health?

  • □ Yes □ No □ Not Sure □ declines to answer

• 6) If we did not discuss HIV risk and prevention and only provided information about the test itself, would you still have taken the HIV test?

  • □ Yes □ No □ Not Sure □ declines to answer

• 7) Would you have preferred to get the test WITHOUT our longer discussion about HIV risk and prevention?

  • □ Yes □ No □ Not Sure □ declines to answer

• 8) If we had given you the HIV test today without any information, and then asked you to go somewhere else to talk about HIV risk and prevention, would you have done this?

  • □ Yes □ No □ Not Sure □ declines to answer

ED Role

• 9) Were you comfortable with your privacy when you were talking about HIV?

  • □ Yes □ No □ Not Sure □ declines to answer

• 10) Were you able to ask all of your questions about HIV and have them answered?

  • □ Yes □ No □ Not Sure □ declines to answer

• 11) Have been tested for HIV before?

  • □ Yes □ No □ declines to answer

    • If yes

    • 11a) was this in the ED

      • □ Yes □ No □ Not Sure □ declines to answer