Improving quality of PrEP counseling for adolescent girls and young women in Kenya with standardized patient actors: a dose-response analysis

Abstract

Background:

Health care worker (HCW) training using standardized patient actors (SPs) is an evidence-based approach for improving patient-provider interactions. We evaluated whether SP training among HCWs in Western Kenya improved quality of PrEP counseling for adolescent girls and young women (AGYW).

Methods:

We conducted a 2-day SP training intervention among HCWs providing PrEP counseling for AGYW. Six trained SPs role-played one encounter each with HCWs following scripts depicting common PrEP-seeking scenarios. SPs used checklists to report and discuss domains of adherence to national PrEP guidelines, communication, and interpersonal skills using validated scales after each encounter. HCWs presented to each case in random order. Overall and domain-specific mean score percentages were compared between the first and subsequent case encounters using generalized linear models, clustering by HCW.

Results:

During 564 training cases among 94 HCW, the overall mean quality of PrEP counseling score was 83.1 (standard deviation [SD]:10.1); scores improved over the course of the 6 encounters (p<0.001). Compared to the first case encounter, mean scores for the fourth were significantly higher (79.1 vs. 85.9, p<0.001). Mean scores plateaued from the fourth to the sixth case (85.2). While HCWs demonstrated high baseline communication (95.3) and interpersonal skills (83.7), adherence to PrEP guidelines at baseline was suboptimal (57.6). By the 4th case, scores increased significantly (p<0.001) for all domains.

Conclusions:

SP training improved PrEP counseling overall and in domains of interpersonal skills, use of guidelines, and communication with AGYW and could be useful in efforts to improve quality of PrEP counseling for AGYW.

Background

In high HIV-burden settings in sub-Saharan Africa (SSA), adolescent girls and young women (AGYW) age 15–24 are at high risk for HIV infection, with eight times the risk of males in the same age range.¹ Globally, an estimated 320,000 AGYW newly acquired HIV in 2019.² Pre-exposure prophylaxis (PrEP) is highly effective at preventing HIV transmission when adherence is optimal,^3–8 therefore a promising HIV prevention approach for AGYW.

PrEP uptake and adherence among AGYW in SSA is insufficient to achieve global HIV elimination targets.^9–11 Poor quality of counseling interactions with health care workers (HCW) can dissuade AGYW from engaging in sexual and reproductive health services, including HIV prevention.^12–17 AGYW experience judgmental attitudes by HCWs and lack of confidentiality, discouraging initiation or continuation of PrEP.^12–14 Concurrently, HCWs report gaps in their knowledge and skills to effectively serve AGYW, particularly with HIV prevention.^14,16–18 It is critical to improve the quality of PrEP counseling for AGYW to optimize uptake.

Standardized patient actors (SPs) are utilized within clinical training as an evidence-based approach to improve provider competency and quality of care.^19–26 SPs pose as patients, evaluating provider compliance with clinical guidelines. Provider competencies in numerous topics, including quality of care,^21,22,26 have been improved by SPs across diverse settings, including South Africa and Kenya.^19,20,25,27

We implemented a training intervention using SPs posing as AGYW in role-played clinical scenarios to improve quality of PrEP counseling by HCWs. We evaluated changes in quality of PrEP services provided by HCW across multiple role-played scenarios to identify the number of SP sessions needed to achieve sufficient competency in PrEP counseling for AGYW.

Methods

Study setting and population

This analysis is nested in a cluster randomized controlled trial (RCT) of an SP-led training intervention: “Simulated Patients to Improve PrEP Counseling for Adolescent Girls and Young Women in Kenya” (“PrIYA-SP”). The study protocol was previously published.²⁸ The PrIYA-SP study was conducted within 24 large family planning and maternal health facilities providing PrEP counseling and care services to AGYW in Kisumu County, Kenya. All HCWs who were current employees at the study sites, were at least 18 years old, able to provide informed consent, and whose job duties included provision of PrEP services to AGYW were approached for enrollment. Our analysis includes data from HCWs who completed the training intervention.

Standardized patient-led training intervention

The training intervention was delivered through two-day training events involving groups of 5–10 HCWs from intervention sites. The training started with didactic lectures covering adolescent health and development, Kenya national guidelines for HIV prevention and PrEP delivery²⁹, and how to structure a high-quality patient-provider encounter. Interactive group activities included a values clarification and patient-centered communication. These activities offered opportunities for HCW to practice reducing the influence of their personal beliefs within patient care and engaging patients in their own health decision-making processes. Training materials were informed by frameworks for clinical communication skills and high quality patient-provider interactions,^24,30–37 and Kenyan national guidelines for PrEP delivery.²⁹

Following didactic training, HCWS completed an applied component consisting of role-play sessions with trained SPs enacting case-scripted scenarios depicting common experiences of AGYW seeking PrEP in the region. SPs were trained by an expert trainer according to the Association of Standardized Patient Educators (ASPE) standards of best practice³⁸, which involved standardized performance review. Case scripts were developed in a standard format by a team comprised of a SP expert, Kenyan and US clinicians, and AGYW researchers who worked together to depict common challenges faced by PrEP-seeking AGYW within prior studies conducted by the group.^19,39,40 Cases involved themes including but not limited to sexual activity among young adolescents, transactional sex, multiple concurrent partners, and having an HIV-positive partner. HCWs rotated through six cases (one SP per case) in random order (Supplemental Table 1). A brief conversation between the SP and HCW followed each encounter where SPs shared HCW strengths and areas for improvement based on a standard, tablet-based checklist. Training events concluded with a review of video-recordings of SP role-play case encounters during a group debriefing session where study staff facilitated synthesis of feedback from peers and SPs.

Data collection procedures

Study staff administered surveys to HCWs to ascertain demographics; training history; beliefs about HIV,³² AGYW, and PrEP;¹⁸ self-reported competency in PrEP counseling to AGYW; and self-reported knowledge of PrEP services. After each role-play case encounter with an HCW, SPs used a tablet-based checklist to record perceptions of the interaction and guide discussion with the HCW (Appendix 1).

Outcome measures

The outcome of PrEP counseling quality by HCWs during the SP encounters was defined by three domains: adherence to PrEP delivery guidelines, communication skills, and interpersonal skills.⁴¹ Sub-scores for the three domains were combined to produce an overall score. Adherence to guidelines was evaluated with a checklist containing 12 questions using binary ‘done/not done’ response options to indicate whether a PrEP counselling message was delivered by the provider during the case encounter according to the Kenyan guidelines (sub-score range: 0–12) (Appendix 1).²⁹ Communication skills were assessed with seven questions using Likert response options (strongly agree, agree, disagree, and strongly disagree) (sub-score range: 0–28), which were informed by the Kalamazoo Consensus statement on essential elements of communication^30,42 and adapted from the Adolescent Patient-Provider Interaction Scale³⁷ to suit the study setting (Cronbach’s alpha = 0.81)^34,43 We assessed HCW interpersonal skills within case encounters using a validated tool developed to evaluate medical graduates during standardized patient encounters.⁴⁴ The IPS tool evaluates 14 items under four dimensions (interviewing and collecting information, counseling and delivering information, rapport, and personal manner) using Likert response options (item score range: 1 – 4; sub-score range: 14–56).

Statistical analysis

We described demographic, employment, and training characteristics of the HCW using counts and proportions or median and interquartile ranges (IQR). Mean sub-scores per domain were calculated by dividing each raw sub-score by its respective total possible score and rescaling by 100. Overall quality was computed by combining those three values, then dividing by three to achieve the weighted mean score.

To evaluate change in overall quality of PrEP counseling score for each case encounter compared to the first case encounter, we used generalized linear models with a Gaussian distribution and identity link including case encounter order as a categorical variable, clustering by HCW and adjusting for time since initial training event (days) to account for temporal changes in training implementation. Analyses were repeated for each domain sub-score. Analyses were performed in Stata version 15.0.

Ethical considerations

The study was approved by the Kenyatta National Hospital Ethics Research Committee and University of Washington Human Subjects Review Committee. Approval was additionally obtained by the Kisumu County Department of Health and health administrators in the health facilities involved. All participants provided written informed consent.

Results

Healthcare worker characteristics

Among the 94 HCW from 12 health facilities who completed the SP-led training intervention, over half (59%) were female and median age was 32 years (interquartile range [IQR]: 29–36) (Table 1). Over a third (37%) of HCW were stationed within HIV Comprehensive Care Clinics, and 37% worked in Maternal and Child Health clinics. Most HCW (83%) completed college as their highest level of education. Just over half (54%) were nurses, and 41% were clinical officers. HCWs had a median of 4 years of experience providing HIV prevention services to AGYW (IQR: 2–7).

Table 1.

Characteristics of healthcare workers participating in SP-led training intervention (n=94)

Characteristics	N or Median	% or IQR
Age (years)	32	29–36
Sex (Female)	55	59%
Clinical station
CCC	35	37%
MCH	35	37%
PMTCT/HTS	8	9%
Other /(FP)	16	17%
Highest education achieved
College	78	83%
University	15	16%
Primary/Secondary	1	1%
Employment title
Nurse	51	54%
Clinical officer	38	40%
Other/(Adherence counselor)	5	5%
Number of years at current clinic	2	1–4
Number of years of experience in HIV prevention services	4	2–8
Number of years working with AGYW	4	2–7
Number of AGYW clients served per week	4	3–6
Ever received training in HIV prevention for AGYW	56	60%
Ever received training in prescribing PrEP	67	71%
Ever received training in prescribing PrEP to AGYW (n=67)	35	37%

Quality of PrEP counseling for AGYW during training intervention

During 564 individual training encounters among 94 HCW, the overall mean score was 83.1 (standard deviation [SD]:10.1) and scores improved after sequential encounters (p<0.001) (Figure 1) taking place over the course of the 2-day training. The first case encounter had the lowest mean score of 79.1 (SD:10.5). Statistically significant improvement was achieved by the third compared to the first case (82.4 vs. 79.1, coef: 3.28, 95% confidence interval [CI]: 0.93–4.84), adjusting for time since initial training event. The fourth case had the highest mean score (85.9, SD:6.1). Compared to the first case encounter, mean scores for the fourth were 6.9 points higher (fourth vs. first case coef:6.9, 95%CI: 4.5–9.2), and mean scores plateaued from the fourth to the sixth case (85.2).

Figure 1. Quality of PrEP counseling overall mean score percent* and results from linear regression models (n=94 HCW, n=564 case scripts).

*Case script coefficients from generalized linear models with a Gaussian distribution and identity link, clustering by individual, case order included as categorical variable, adjusted for number of days since initial training event

**”Overall” coefficient from generalized linear model with Gaussian distribution and identity link, clustering by individual, case order included as a continuous variable, adjusted for number of days since initial training event

Sub-domains of PrEP counseling for AGYW

The domain of HCW adherence to PrEP guidelines had the lowest mean score of 66.1 across all encounters. The mean sub-score significantly improved by 14 points from the first to the fourth case (fourth vs. first case coef:14.1, 95%CI: 8.9–19.3, p<0.001), and mean sub-scores plateaued from the fourth to sixth case.

We saw a similar pattern in the mean sub-scores for the communication skills and interpersonal skills domains. HCWs had the highest mean score for the communication skills domain with 96.2 overall. We observed statistically significant improvement from the first to the fourth case (97.2 vs. 95.3, coef: 1.85, 95% CI: 0.53–3.16) which plateaued through the sixth case. Overall, HCW demonstrated mean score percent of 86.9 for interpersonal skills. The highest sub-score for interpersonal skills was achieved in the fourth case (89.1) which was about five points higher than the first case (coef: 5.4, 95% CI: 3.3–7.5). In sensitivity analyses, we did not find differences in quality of PrEP counseling by HCW characteristics.

Discussion

In our training evaluation with over 500 role-play interactions, we found that quality of PrEP counseling for AGYW improved after HCW training involving standardized patient actors. HCW competency in adherence to national PrEP guidelines, communication skills, and interpersonal skills improved after only three encounters, were highest after four, and plateaued after that. These findings indicate that a shorter training with four cases may be sufficient to elicit improvement in PrEP counseling for AGYW, allowing for lower resource allocation to achieve sustained improvement in counseling quality.

Quality of PrEP counseling assessed in the initial SP role-play sessions was high among this sample of HCW, including domains of communication and interpersonal skills. However, adherence to national PrEP guidelines for AGYW counseling sessions was relatively low. Adherence to PrEP guidelines showed the greatest improvement, yet HCW only reached a mean score of 72, demonstrating further room for improvement.⁴⁵ All domains showed statistically significant improvement following SP-led training encounters.

SPs are especially useful at assessing and improving patient-centered communication – a key component of quality of care.⁴⁶ Global guidelines from groups such as the World Health Organization and US. President’s Emergency Plan for AIDS Relief (PEPFAR)^47,48 increasingly call for a focus on provider competency and patient-centered communication as key components of adolescent and HIV care provision.^49,50 On the path toward global HIV elimination, PEPFAR promotes patient-centered services⁴⁷ that are hospitable and friendly⁴⁸ as an important last mile approach to ensuring hard-to-reach key populations receive HIV prevention and treatment. As high-HIV burden countries rapidly expand efforts to promote PrEP uptake among groups with high HIV risk, such as AGYW, SPs could be engaged within clinical training to operationalize policy and guideline implementation.

Our study was limited by the self-reported nature of SP checklists used to quantify quality of PrEP counseling. However, SPs were trained in ASPE best practices and underwent expert performance evaluation. Implementation of the intervention may have changed over time from the first to the final event, thus we adjusted our results for time elapsed. HCWs were aware they were being evaluated during training encounters, which may have influenced their performance. To address this, our ongoing trial utilizes unannounced SPs, or “mystery shoppers” to measure quality outcomes in real-world clinical settings within intervention and control sites.

In conclusion, simulated patient training improved provider guideline adherence and interpersonal communication skills in PrEP counseling for AGYW. Improvement was achieved following at least three simulated case encounters between HCWs and SPs, and highest quality was seen after four case encounters. Findings suggest SP training is useful to strengthen provider PrEP knowledge and interpersonal skills to engage and counsel AGYW on PrEP for HIV prevention. The forthcoming PrIYA-SP trial will further inform effectiveness of this SP-led training intervention in improving quality of PrEP counseling for AGYW. As sub-Saharan Africa strives toward global HIV elimination targets, additional research is needed to determine whether gains in quality care subsequently result in improved PrEP uptake and continuation.