Comparing three methods to assess learning outcomes for a suicide prevention training program for pharmacy staff

Abstract

Objective

To examine whether pharmacists and pharmacy staff who complete a suicide prevention gatekeeper training program (Pharm-SAVES) react similarly to a written patient case, a live simulated patient (SP), and a prerecorded SP case.

Methods

After completing the 30-min Pharm-SAVES training, participants completed a written patient case via survey and then, 1 month later, completed a prerecorded SP and live SP interaction via Zoom. For each assessment type, we documented whether the participant asked about suicide and referred the patient to the Suicide and Crisis Lifeline (988).

Results

Participants (n = 12) asked about suicide in 8 (67 %) written patient cases, 9 (75 %) prerecorded SP cases, and 8 (67 %) live SP cases. Participants referred patients to 988 in 8 (67 %) written patient cases, 5 (42 %) prerecorded SP cases, and 10 (83 %) live SP cases.

Conclusion

The number of participants who asked about suicide was similar regardless of assessment type; however, referrals to the Suicide & Crisis Lifeline happened less often with the prerecorded SP cases.

Innovation

This is the first study to compare key learning outcomes of pharmacy suicide prevention gatekeeper training across written, live, and prerecorded SP encounters.

1. Introduction

Learning through case-based scenarios has been shown to improve real-world application of learning objectives as well as improve learners' critical thinking and decision-making skills. [[1], [2], [3]] In training programs, case-based scenarios often mimic real-world situations or problems to allow learners to apply training in practice. [4] Written patient cases are text descriptions of patient scenarios where respondents provide written responses describing how they would react. This assessment format is often used due to its low cost and simplicity. [5] Prerecorded SP cases are video-based scenarios with actors portraying patients that learners watch and then react to orally or in written format. This assessment type is useful given that it presents the case in a standardized manner and limits variation in the SPs' presentation; however, there is limited interactivity between the SP and the learner. Live SP cases involve realistic real-time interactions between learners and actors who portray patients. [6] The use of live SPs can increase learners' knowledge and confidence in assessing and counseling individuals; however, they are usually resource-intensive, and SPs can vary in their delivery of the scenario. [7,8,9].

Comparison between the three patient case modalities can help educators decide which case type is most feasible, impactful, and the best fit for learners. Studies have not directly compared written patient cases to prerecorded SP and live SP formats to determine if learners' reactions to the cases are similar. Our objective is to address this research gap through the evaluation of pharmacist responses to three patient case scenarios (written case, prerecorded SP and live SP) that were part of the assessment of the Pharm-SAVES training module. Pharm-SAVES is a skills-based gatekeeper training for pharmacy staff, focused on two key learning outcomes: directly asking about suicide and referring patients to the Suicide & Crisis Lifeline (988 in the U.S.). Individuals in crisis may not reveal suicidal ideation until asked directly [[10], [11], [12], [13], [14]] and 988 provides local, customized support for individuals experiencing suicidal ideation with the goal to reduce suicidality. [15]

2. Methods

This is a cross-sectional study comparing three assessment methods: a written patient case, a prerecorded SP case, and a live SP case. Data were collected as part of a randomized controlled trial that examined the effectiveness of Pharm-SAVES from March 2022 to April 2023. [16] This trial was approved by the Institutional Review Board (IRB) at the University of North Carolina (UNC) at Chapel Hill (IRB approval #: 21–1062).

Pharm-SAVES is a 30-min online suicide prevention training specifically for community pharmacy staff that has been described previously. [17]

Participant recruitment is described in detail elsewhere. [18] In brief, adult English-speaking community pharmacy staff from four southeastern states (North Carolina, South Carolina, Georgia, Tennessee) were recruited. After completing Pharm-SAVES, participants completed written patient cases on the immediate post-training survey and received a USD $20 incentive.

Approximately one month later, participants were emailed a follow-up survey, with an option to participate in the prerecorded and live SP assessments. The live and prerecorded assessments were videorecorded via Zoom and took approximately 10 min total. Participants were asked to respond to the prerecorded SP and live SP as they would a patient physically in their pharmacy, and the order of the prerecorded SP and live SP were randomized between participants. Patient cases closely mirrored scenarios presented in the Pharm-SAVES training (Table 1). [18] The live SP had the same demographic characteristics listed in Table 1.

Table 1.

Descriptions of the three assessment methods.

Case Type	Description:
Written Case	Angie is a 55-year-old Black female whose spouse recently died from cancer. She comes to the pharmacy often and is new to living alone. Since her spouse died, her personality has changed from bubbly to quiet and sad. She approaches the pharmacy counter, and the pharmacist begins the conversation. Pharmacist: Hi, Angie. How may I help you today? Angie: (slightly unsteady and uncertain) Hi, my doctor gave me this new prescription to help me feel better, but I'm not sure if this is really going to help me. Pharmacist: Ok, Angie, give me a second while I look up the prescription. So, this medication is for depression, and it can take several weeks for you to start to feel better since it needs to build up in your system. Angie: How many would I need to take to really feel better? Pharmacist: Angie – what do you mean by ‘really feel better?’ Angie: Enough so I don't have to wake up tomorrow. I can't take another day like this. In the box below, please write what you would say to Angie after she said, “Enough so I don't have to wake up tomorrow. I can't take another day like this.” (open response) Would you refer Angie to any specific resources or organizations? If yes, which resources/organizations? (open response)
Prerecorded Simulated Patient	The prerecorded video SP portrayed a young White college-aged male patient picking up a prescription for Adderall, noting he was in a hurry, and asks, “Do you think if I take more than prescribed, it will help me focus better for the exam?” The video was paused, and the participant was asked to respond. After the response, the prerecorded video SP continued to say, “Well hey, I guess if I take a full bottle, I don't have to worry about the exams at all.” The participant was again asked to respond.
Live Simulated Patient	The live SP scenario was of a young Black woman at the pharmacy to pick up a new medication for her “mood”. She noted that she was concerned she may take too many. The SP then paused for a response to that statement. The SP then said, “I just don't want to feel anything anymore, sometimes it feels like nothing is going to get better” and paused for response. She then said, “With the world the way that it is, it might be nice to just drift off to sleep without having to worry anymore,” and paused for response.

2.1. Measures

On the baseline survey, respondents reported their demographic information (See Table 2).

Table 2.

Participant characteristics (n = 12).

Characteristic	N (%)
Age
18–24	1 (8.3)
25–29	0
30–34	3 (25)
35–39	1 (8.3)
40–44	3 (25)
45–54	2 (16.6)
55–64	2 (16.6)
Gender
Female	9 (75)
Male	3 (25)
Race
White	10 (83.3)
Asian	2 (16.7)
Role at Pharmacy
Pharmacist	9 (75)
Technician	3 (25)
Previous suicide prevention training
Yes	2 (16.7)
No	10 (83.3)

Using an observation guide based on previous research [19,20], two blinded coders independently coded the cases to document (yes/no) whether the participant: 1) asked the patient a direct question about suicide (used the words “suicide” or “killing yourself” in their question) and 2) referred the patient to 988. Coders reached 100 % consensus on all interactions.

Quality of verbal (8 items) and nonverbal (7 items) communication was assessed for the prerecorded and live SP cases using a scale of 0 to 2 (0: skill not demonstrated, 1: skill needs development, 2: skill demonstrated with competence). A mean quality of communication score was calculated for each item, with higher scores indicating higher quality verbal and nonverbal communication.

2.2. Data analysis

Descriptive statistics were calculated using IBM SPSS Version 28 (Armonk, NY). Means and standard deviations were calculated for each communication skill.

3. Results

Thirteen respondents completed the SP encounters; however, one experienced technical difficulty and their recording was not analyzed.

Trainees directly asked about suicide in 8 (66.7 %) written patient cases, 9 (75 %) prerecorded SP cases, and 8 (66.7 %) live SP cases. Table 3 shows example questions participants asked.

Table 3.

Example questions participants asked about suicide across the three assessment types.

ID Demographics	Written patient case Depicts 55-year-old black female	Prerecorded SP Depicts college- aged white male	Live SP Depicts young black woman
102 White female technician	“When you say those things, I become concerned. Let's go over here and talk about what's going on. Are you ok?”	“Are you talking about killing yourself? Are you talking about suicide?”	“Are you thinking about doing anything, like hurting yourself?”
4201 White female pharmacist	“Would you like to talk about anything? I'm here but there are also places that can help”	N/A	“OK, so tell me more about that. What makes you feel that way?”
4401 White male technician	N/A	“Have you spoken to your doctor about them (your side effects) about what you're experiencing?”	“So, what, what prompted you to contact the doctor to get prescribed a medication like this?”
1002 Asian female technician	“Are you contemplating suicide?”	“Ask um if he has been um contemplating killing himself.”	“Have you been feeling suicidal or hopeless lately?”
2101 White female pharmacist	“Angie, I know things are bad now but there are people who care about you. What you are feeling is normal after losing someone and there are people who can help you through this.”	“Well, I really have concerns when you talk about taking a full bottle. Are you really talking about committing suicide?”	“So, are you talking about suicide? (raised eyebrow)”
3001 White female pharmacist	“Are you talking about suicide?”	“I would ask him if he's considering suicide.”	“I understand what you're saying. Are you maybe talking about suicide?”
4402 Asian Male pharmacist	“Are you talking about suicide?”	“Hey, are you thinking of committing suicide?”	“Jen, are you talking about committing suicide, because that is never a good option for anyone”

988 referrals were more varied, with 8 (66.7 %) referrals in the written case, 5 (41.7 %) in the prerecorded video SP case, and 10 (83.3 %) in the live SP case.

For many items, verbal and nonverbal communication was similar between the prerecorded SP and the live SP (Table 4). Participants demonstrated the skill of acknowledging the patient's feelings with competence more frequently with the live SP in comparison to the prerecorded SP.

Table 4.

Mean verbal and nonverbal communication skills comparing interactions with the live SP and the prerecorded video SP.

	Live SP Mean (SD)	Prerecorded SP Mean (SD)
Verbal Communication Skills
Natural tone	1.92 (0.29)	1.67 (0.49)
Participant had clear enunciation	1.92 (0.29)	1.92 (0.29)
Appropriate rate of speech	1.67 (0.49)	1.75 (0.62)
Spoke clearly without clutter	1.33 (0.49)	1.33 (0.65)
No use of jargon	2 (0)	2 (0)
Participant responded perceptively, genuinely, and appropriately	1.67 (0.65)	1.17 (0.84)
Participant spoke confidently about suicide	1.17 (0.84)	1.17 (0.84)
Participant acknowledged the patient's feelings	1.67 (0.78)	1.08 (0.79)
Nonverbal Communication Skills
Confident posture	1.42 (0.67)	1.42 (0.67)
No distracting hand movements	1.5 (0.67)	1.58 (0.67)
Appropriately used silence	1.25 (0.45)	0.92 (0.79)
Maintained caring demeaner	1.75 (0.62)	1.33 (0.65)
Gave full attention to patient	1.92 (0.29)	1.5 (0.67)
Maintained eye contact	1.42 (0.69)	1.58 (0.79)
Open body language	1.58 (0.69)	1.33 (0.78)

Table note: Response options were 0: skill not demonstrated, 1: skill needs development, 2: skill demonstrated with competence.

4. Discussion

Results suggest that participants directly asked about suicide at similar rates across a written patient case, a prerecorded SP case, and a live SP case. Participants made more frequent referrals to 988 in the written patient case and the live SP case.

Pharmacy staff acknowledged patient feelings more commonly with the live SP than the prerecorded SP. Students have previously reported that live patient simulations enabled them to practice therapeutic communication skills and enhanced their confidence to interact with patients. [21] This study supports previous literature, which found that a live SP encounter is more realistic than a prerecorded encounter. [22,23] Validating patients' feelings is a key step in Pharm-SAVES. Supporting patient disclosure, particularly in the community setting, by supporting identification and referral of patients who may not otherwise interact with a provider while in crisis has the potential to save lives. Further research with larger sample sizes should evaluate whether our results are replicable.

4.1. Limitations

This study had several limitations. The written cases were completed immediately after the training; however, the prerecorded and live SP cases were completed approximately one month after training. The written patient case prompted participants to refer the patient to alternative resources, which, in addition to the timing of the assessment, may have biased participants to make a referral. The prerecorded SP and live SP encounters were conducted over Zoom, so participants' nonverbal communication could be different than an in-person encounter. Although instructed to speak to the SP as they would a patient physically in their pharmacy, two respondents spoke in the third person. The small sample size limits generalizability. Varying demographic characteristics of the patients depicted in the cases could have influenced participant responses.

4.2. Innovations

This study is the first to compare written patient case scenarios directly with live SP and prerecorded SP encounters in the implementation of suicide prevention training for pharmacy staff. Live SPs can be cost-restrictive for many educational programs, so evaluating whether less resource intensive prerecorded SPs are an effective alternative to live SPs is important.

This research could help justify the use of prerecorded SPs in lieu of live SPs in training evaluation given similar outcomes. However, participants' competence with some communication skills, such as acknowledging the patients' feelings, were higher with a live SP, suggesting that live SPs are useful depending on which skills are being evaluated. However, students were not graded on their performance, and if graded may have been more likely to validate the prerecorded SP's feelings.

5. Conclusion

Written cases or prerecorded SP cases could be considered by programs that do not have enough resources to hire and train live SPs for evaluation of suicide prevention training. This study provides preliminary data that prerecorded SPs result in similar learning outcomes compared to live SPs or written cases; however, live SPs seemed to provide students more realistic opportunities to demonstrate verbal and nonverbal communication skills. Overall, additional research is needed to examine whether these study results are replicable in studies with larger samples.

Funding

Development of the Pharm-SAVES curriculum was supported by Grant Number SRG-1-025-19 from the American Foundation for Suicide Prevention. The authors also acknowledge the generous support provided by the Eshelman Institute for Innovation at the UNC Eshelman School of Pharmacy and the Center of Excellence for Suicide Prevention, Department of Veterans Affairs. The content is solely the responsibility of the authors and does not represent the views of the funder, the Department of Veterans Affairs, or the United States Government.

CRediT authorship contribution statement

Grace Marley: Writing – review & editing, Writing – original draft, Formal analysis. Jill E. Lavigne: Writing – review & editing, Conceptualization. Wendi Cross: Writing – review & editing, Investigation, Conceptualization. Abigail Gamble: Writing – review & editing, Project administration. Zhuying Zhang: Writing – review & editing. Delesha M. Carpenter: Writing – review & editing, Project administration, Methodology, Funding acquisition, Formal analysis, Conceptualization.

Declaration of competing interest

GM, DMC, JEL, WC, ZZ, and AG do not have any conflicts of interest to declare.