Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2023 Oct 27.
Published in final edited form as: Transl Issues Psychol Sci. 2022 Mar;8(1):153–163. doi: 10.1037/tps0000305

How to incorporate lived experience into evidence-based interventions: assessing fidelity for peer-delivered substance use interventions in local and global resource-limited settings

Morgan S Anvari 1,*, Jennifer M Belus 1, Mary B Kleinman 1, CJ Seitz-Brown 1, Julia W Felton 2, Dwayne Dean 1, Nonceba Ciya 3, Jessica F Magidson 1
PMCID: PMC10611440  NIHMSID: NIHMS1893906  PMID: 37900977

Abstract

Interventions led by peer recovery specialists (PRSs) have rapidly expanded in response to a global shortage of access to substance use treatment. However, there is a lack of guidance on how to incorporate PRSs’ lived experience into the delivery of evidence-based interventions (EBIs). Moreover, few resources exist to assess fidelity that integrate both content fidelity, peer competence, and incorporation of lived experience (i.e., PRS role fidelity). This study aimed to: (1) describe a novel PRS fidelity monitoring approach to assess both content and PRS role fidelity; (2) compare independent rater and PRS-self-reported content fidelity; (3) examine associations between content and PRS role fidelity; and (4) assess whether the PRS role fidelity was associated with substance use at post-treatment. This study was conducted across two PRS-led behavioral intervention trials conducted in global resource-limited settings: Baltimore City, US, and Khayelitsha, South Africa. A significant difference was found between PRS- and independent rater content fidelity in both interventions, with PRSs reporting significantly higher content fidelity in both sites. PRS role and content fidelity were not significantly correlated, suggesting greater adherence to the PRS role is not associated with lower adherence to structured EBI content. PRS role fidelity was not significantly associated with substance use at post-treatment. This study provides an important step towards understanding how to assess PRS role fidelity in the context of EBIs for underserved individuals with SUD that also incorporates their lived experience.

Keywords: fidelity, behavioral intervention, peer recovery specialist, substance use, global mental health

Approximately 31 million people globally have a substance use disorder (SUD; World Health Organization, 2019). Despite the global recognition that SUD is a significant public health priority (UNAIDS, 2015), evidence-based treatment for SUD remains difficult to access globally (Salwan & Katz, 2014; Keynejad et al., 2017). Fewer than 5% of people with SUD in low- and middle-income countries have access to minimally adequate treatment (Degenhardt et al., 2017). In 2018 alone, 18.9 million people in the US needed, but did not receive, SUD treatment (SAMHSA, 2019).

To address this global SUD treatment gap, the workforce of peer recovery specialists (PRSs), also known as peer recovery coaches or peer recovery advocates depending on geographical location, has rapidly expanded in the US, particularly to meet the needs of underserved, low-income, ethnoracial minority individuals (Bassuk et al., 2016). In the US, a PRS is a trained, often certified, individual with lived substance use experience, who can incorporate this lived experience into interactions with clients. PRSs typically provide a variety of services, such as linkage to resources (e.g., treatment, basic needs, etc.), case management, and assistance in navigating health care systems, and may work in a variety of settings such as hospitals and community resource centers. Across settings, while PRSs are typically trained in motivational interviewing (MI), their delivery of other structured, evidence-based interventions (EBIs) has not been as widely implemented (Reif et al., 2014; Bassuk et al., 2016; Eddie et al., 2019; Satinsky et al., 2021), in part because their role has largely been viewed as “non-clinical”. Expanding the PRS role to include dissemination of EBIs may help address the SUD treatment gap, particularly for underserved and ethnoracial minority individuals. There is precedent in the field of global mental health for PRSs to assume a greater role in delivering EBIs, in which PRSs and lay counselors have been successfully trained and supervised to deliver EBIs, such as cognitive behavioral therapy (CBT; Verhey et al., 2020), where, for example, PRSs have been integrated into outpatient clinics in India as well as community settings in Pakistan in order to provide CBT for women with perinatal depression (Sikander et al., 2015). There has also been an increased focus in the US to better understand how PRSs can deliver EBIs locally, such as behavioral activation (Satinsky et al., 2020).

Fidelity to an EBI refers to the degree to which an interventionist carries out an intervention in the way in which it was intended (Gearing et al., 2010). By establishing high fidelity, PRS models may continue to gain credibility. With the expansion of PRS-delivered EBIs for substance use, such as MI, it is necessary to improve methods for evaluating fidelity of other PRS-delivered interventions. A limited number of studies in the US have measured content fidelity of PRS-led interventions for substance use (Wolfe et al., 2013; Tracy et al., 2012), for instance through checklists and independent ratings. Only a few studies in low or middle-income countries have formally assessed content fidelity in peer-led interventions (Morojele et al., 2014; Sikander et al., 2015), although assessing content fidelity has been successfully conducted in other lay health worker interventions to improve mental health outcomes (Murray et al., 2011; Murray et al., 2014; Patel et al., 2011).

While PRS delivery of EBIs may be a promising strategy, EBIs may not typically incorporate opportunities for sharing lived experience, which is a critical element of peer-delivered services. Fidelity assessments in therapist-led interventions have examined both content fidelity and therapist competence (i.e., non-specific factors; Carroll et al., 2000; Madson & Campbell, 2006; Hogue et al., 2008); yet, there have been few efforts to develop a fidelity assessment that incorporates both content fidelity (i.e., adherence to intervention content) and elements specific to delivering PRS interventions that include shared lived experience (i.e., competency in common factors and self-disclosures). Whereas shared experience is a defining quality of a PRS, and PRS training in the US typically contains content/modules on appropriate utilization of self-disclosures/shared experience in PRS interactions, as well as avoiding stigmatization of clients, these behaviors may be important to capture as well. It is important to capture PRS’ competence, self-disclosures, and potentially stigmatizing and de-stigmatizing behaviors in order to understand the extent to which PRSs are fulfilling their unique role. It is also important to understand how allowing for sharing of lived experience relates to being able to maintain content fidelity for PRS-delivered EBIs.

The goals of the current study were to (1) describe an approach to assessing fidelity to intervention content and the PRS role through the adaptation of existing measures; (2) explore differences between independent- and PRS-rated content fidelity; (3) examine associations between content and PRS role fidelity; and (4) explore whether PRS role fidelity was significantly associated with substance use at post-treatment. Data were drawn from two PRS-delivered behavioral intervention trials in global resource-limited settings, one in a community resource center in Baltimore City, US and the other in public primary care in Khayelitsha, South Africa. Based on prior psychotherapy fidelity work that has found an association between treatment adherence and therapist competence (Carroll et al., 2000), we expected that there would be a positive association between content fidelity and PRS’ overall competency (i.e., in common factors such as empathy and being non-judgmental). On the other hand, as an exploratory hypothesis, for the other component of PRS role fidelity, the use of self-disclosures, we predicted that a greater frequency of self-disclosures may be associated with lower content fidelity, demonstrated by a negative relationship between frequency of self-disclosures and content fidelity, due to challenges integrated self-disclosures within the constraints of a 60-minute, prescribed, manualized intervention. Finally, we predicted that higher PRS role fidelity would be associated with greater reductions in substance use at post-treatment. The overall aim of this work is to support the training, supervision, and assessment of PRS-delivered interventions, including EBIs, in resource-limited settings globally.

Methods

Participants and Procedures

The PRS fidelity approach was piloted in two PRS-delivered behavioral intervention trials, both evaluating delivery of a specific EBI (behavioral activation; Lejuez et al., 2011; see Table 1), one in a public primary care clinic in Khayelitsha, South Africa (Project Khanya) and the second in a community resource center in Baltimore, MD (Peer Activate).

Table 1.

Overview of interventions

Project Khanya Peer Activate
Setting Primary care setting, Cape Town, South Africa Community resource center, Baltimore, Maryland
Major intervention components Life Steps, behavioral activation, mindfulness Behavioral activation
Total intervention sessions 6 8
Primary outcome variable HIV antiretroviral medication adherence, substance use risk Linkage to substance use treatment
Open in a new tab

Project Khanya (South Africa).

Project Khanya was a Type 1 hybrid effectiveness-implementation trial to evaluate a six-session, PRS-delivered, behavioral activation (BA) intervention focused on reducing substance use and increasing medication adherence for people living with, and in care for, HIV (N=61; n=30 participants randomized to intervention condition) in Khayelitsha, South Africa. The intervention was compared with a referral to a local evidence-based substance use treatment center (Magidson et al., 2021). This trial took place in a public primary care setting in order to evaluate the feasibility and acceptability to integrate a PRS-led intervention into HIV care, given the lack of availability of trained mental health and substance use professionals in South Africa (Pasche et al., 2015). Given the use of task sharing models in HIV care in South Africa, integrating substance use treatment into an HIV clinic presents an opportunity to extend these models to substance use treatment, and apply these lessons to resource-limited contexts in the US. Seventy percent of participants completed all intervention sessions; of the sample included for fidelity monitoring, only two participants lost contact with the research team and dropped out. See Magidson et al. (2021) for additional study details and Magidson et al. (2020) for detailed description of the intervention.

Peer Activate (US).

Peer Activate was an open-label pilot feasibility trial evaluating an eight-session, PRS-delivered BA intervention focused on linking individuals (N=6) to substance use treatment from a community resource center in Baltimore City. Four of the six participants (66.7%) completed all intervention sessions; one participant lost contact with the research team and did not report reasons for discontinuing, and the other chose to discontinue due to demands on their time from other treatment engagement. There was flexibility in session content based on client need; this decision was based on formative qualitative work involving stakeholders, who noted that a less-structured intervention would better meet the needs of the target population, due to competing psychosocial and structural needs (Satinsky et al., 2020).

PRS interventionists.

Both studies included one peer interventionist with lived substance use experience and close connections to the local communities. In South Africa, there was not a formal PRS certification process, however, the interventionist did receive prior clinical training in delivering a trauma-focused treatment. The interventionist in this study identified as a Black African woman and was in her early-40s. The interventionist for Peer Activate had formal PRS training and certification, identified as a Black male and was in his mid-40s. In order to promote interventionist fidelity and participant engagement and comprehension of intervention content, the interventionist in South Africa was given a flipchart to use during each session. The flipchart is a two-sided visual aid, with one side visually depicting the concepts for clients and the other side a guide to the core content to assist the interventionist in content delivery (see Magidson et al., 2020). For Peer Activate, it was not as feasible to have a structured flipbook given the need to promote flexibility in the intervention delivery, both in terms of content and where the intervention was delivered (i.e., not always seated or face-to-face). However, flipbook components were developed into patient handout materials to support selected content.

A PhD-level clinical psychologist trained both interventionists, which included intensive training in the respective treatment components, common factors, and role plays. During the intervention, both interventionists received weekly, hour-long supervision from a PhD-level clinical psychologist and/or a clinical psychology doctoral student who reviewed audio recordings. The interventionists identified particularly challenging sessions to review in supervision. For Project Khanya, these sessions, in addition to those randomly selected for fidelity monitoring, were translated into English from isiXhosa by a South Africa-based research assistant fluent in both languages. Though session recordings (for supervision and fidelity monitoring) were not back-translated, all intervention content and materials were back-translated. Supervisors provided general feedback on both content and process, including balancing these two elements of the intervention. Supervision positively reinforced appropriate use of self-disclosures (i.e., supervisors encouraged the use of self-disclosures, gave examples of where interventionists could have utilized self-disclosures, and provided praise when they were used appropriately). Similarly, supervisors identified examples of, and discussed, stigmatizing language or behaviors throughout the course of the interventions. After each intervention session, both interventionists self-reported their fidelity to treatment content using a checklist of intervention components (see Measures for more detail on the checklist).

Session selection and fidelity coding.

Twenty percent of intervention sessions in each study were randomly selected for fidelity coding as part of the larger intervention trials. For Project Khanya, sessions were randomly selected for fidelity monitoring prior to completion of the intervention trial, and, thus, if a session did not occur due to participant drop-out, a new session was randomly chosen. This resulted in 36 sessions for Project Khanya (n=19 unique participants). For Peer Activate, sessions were randomly selected for fidelity monitoring after the intervention trial was completed, and 20% of completed (versus possible) sessions were randomly selected, resulting in eight sessions from four participants. Given the small sample size for Peer Activate, we then randomly selected one session from each of the remaining two participants in order to ensure that each participant (N=6) had at least one randomly selected session to assess, yielding a total of ten sessions. Across interventions, 46 total sessions were selected.

Raters trained by a PhD level clinical psychologist and a clinical psychology doctoral student listened to recordings of individual sessions and coded the session based on a standardized form (see Measures section below). Questions about any content or process items were discussed and resolved by group consensus, with four-to-five team members including raters and the aforementioned trainers.

Measures

Fidelity monitoring approach

Content fidelity.

We followed standard procedures to assess content fidelity (i.e., adherence to intervention content), which involved reviewing the content pertinent to each intervention’s treatment protocol (Breitenstein et al., 2010). After every session, the PRSs in both interventions completed this measure based on their recall of the session. The content fidelity checklist for Project Khanya contained 15–19 items, depending on the session. The Peer Activate form contained 12 content items that could be flexibly delivered across the entirety of the intervention. Items covered in each session were indicated by checking them off a list (e.g., “was the behavioral cycle reviewed,” “was Life Steps reviewed”).

PRS role fidelity.

There were seven items to capture PRS role fidelity (i.e., competency in common factors and self-disclosures), which were only assessed by the independent raters. We included four items from the Enhancing Assessment of Common Therapeutic factors (ENACT) scale (Kohrt et al., 2015). The ENACT scale is an 18-item measure developed to assess competence among non-specialists delivering mental health interventions. Each item is scored on a 3-point Likert scale of (2) done well, (1) done partially, and (0) needs improvement. Higher scores represent higher clinical competency. We included items from the ENACT relevant to the unique elements of the PRS role—empathy, quality/appropriateness of self-disclosures, exploration and normalization of feelings, and verbal communication skills such as validation and avoiding judgement. Items from the scale that were not included in the current study focused on non-verbal communication (which could not be assessed in the current study due to the use of audio recordings only), and other intervention-specific factors (e.g., assessing client coping mechanisms, explanation of confidentiality, etc.).

As the ENACT scale is for non-specialists broadly, we also included additional items specific to the PRS role (e.g., frequency of disclosures). These items were measured in every session in order enhance the assessment and measure items more specific to the PRS role. We chose to measure both the frequency and content of self-disclosures (i.e., the PRS sharing their experience with substance use and/or recovery with clients) and stigmatizing behaviors (e.g., stigmatizing language, tone, etc.), whereas utilizing one’s shared experience through disclosures would be seen as consistent with the PRS role and stigmatizing behaviors would be seen as inconsistent. Examples of stigmatizing behaviors may include using stigmatizing terms when referring to substance use (e.g., “dirty”), or stigmatizing a client’s recovery pathway (e.g., viewing a client who receives medication for opioid use disorder as not being in recovery). A quality self-disclosure may include stating one’s personal experience (e.g., “when I was in active use, I experienced a similar situation”), followed by relating it back to the client and/or intervention content. Raters used a dichotomous yes/no response to indicate whether a stigmatizing behavior or self-disclosure occurred in the session. If yes, the rater included a verbatim transcription of the event and included details of tone when possible (only relevant for English language sessions).

Patient Outcomes

Substance Use.

The Alcohol, Smoking and Substance Involvement Screening Test (WHO-ASSIST), a self-report assessment of substance use (alcohol, cannabis, cocaine, opiates, amphetamines, hallucinogens, and other drugs), was measured at baseline and post-treatment (approximately two-months post-treatment for Peer Activate, and approximately three-months post-treatment for Project Khanya) in both studies (WHO ASSIST Working Group, 2002). The WHO-ASSIST measures frequency of use, substance-related problems across domains (e.g., financial, interpersonal), dependency symptoms, and injection drug use. Individuals were categorized into low (0–3 for illicit drugs/ 0–10 for alcohol), moderate (4–26 for illicit drugs/ 11–26 for alcohol), or high risk (>26 for illicit drugs and alcohol) for their substance use related problems, thus, individuals with higher scores indicated more risks and problematic behaviors associated with their substance use. We reported on percentage of individuals scoring moderate or higher when reporting demographics, and used the score for the substance in which they scored highest as a measure of substance use risk, which is considered to be standard practice.

Data Analysis

We calculated descriptive statistics on percentage completed content fidelity and PRS role fidelity, which included total peer-disclosures and stigmatizing behaviors, mean overall ENACT score per session, and individual ENACT item scores per session, to describe the fidelity monitoring results (aim 1). In order to explore differences between independent- and PRS-rated content fidelity (aim 2), a paired t-test using the content fidelity percentage values from independent raters and PRSs was performed. In order to meet the assumption of a paired t-test that the dependent variable should be approximately normally distributed, an arcsine transformation was performed on the content fidelity variable.

In order to examine associations between content fidelity and PRS role fidelity (aim 3), we calculated Pearson correlations between content fidelity (i.e., the proportion of completed intervention content) with average ENACT score and total self-disclosures per session. We also conducted a Student’s t-test to evaluate differences in percentage of content fidelity in sessions containing a peer disclosure versus those that did not, in order to observe if disclosures may relate to increased or decreased content fidelity. Given the flexible delivery of Peer Activate and distinct approach to assessing content fidelity compared to Project Khanya, a sensitivity analysis was also conducted examining only Project Khanya data and results did not differ. We predicted that average ENACT score would be positively correlated with content fidelity, while peer disclosures would be negatively correlated with content fidelity.

Finally, to assess whether PRS role fidelity was significantly associated with substance use at post-treatment (aim 4), we conducted a multiple linear regression analysis to predict post-treatment WHO-ASSIST score using peer disclosures and ENACT average score, controlling for baseline WHO-ASSIST score and intervention (Project Khanya versus Peer Activate). All assumptions of the linear model were met. For participants who had multiple sessions (n = 9), each predictor variable (i.e., average ENACT score, peer disclosures) was averaged across the sessions, resulting in one observation per participant (n = 25). We also conducted a sensitivity analysis to include content fidelity as a covariate, and results did not differ. Models presented do not include content fidelity. We predicted that higher PRS role fidelity would be related to lower post-treatment substance use risk.

Results

Demographic data are presented in Table 2; participants in both studies primarily identified as Black and male. Descriptive analyses to explore overall content fidelity and PRS role fidelity are presented in Table 3. For Project Khanya, the independent rater indicated high average content fidelity (M=92%) and ENACT total score (M=1.69, on 1–3 scale). More than half of sessions included at least one peer disclosure, and only one stigmatizing behavior was recorded across sessions. For Peer Activate, approximately 70% of all intervention components were delivered on average per session, and the independent rater reported high average ENACT total score (M=1.98). Seventy percent of sessions included at least one disclosure, and three total stigmatizing behaviors were recorded across all sessions. Examples of self-disclosures related to session content from Project Khanya includes the PRS describing the use of mindfulness techniques to help slow down thoughts in a high-risk situation. In Peer Activate, an example disclosure was explaining urges that the PRS had experienced early in his recovery, such as the urge to spend excess money on drugs. For both Project Khanya and Peer Activate, the independent rater content fidelity was significantly lower than the peer interventionist’s self-reported fidelity (Project Khanya: t(35) = −4.12 p < .001; Peer Activate, t(9) = −3.98, p = .003).

Table 2.

Demographic Intervention Participant Data

Project Khanya (N = 19) Peer Activate (N = 6)
Variable n % n %
% Male 11 57.9% 5 83.3%
Age (M, SD) 40.32 (9.64) -- 41.83 (14.95) --
% Black or African American 19 100% 4 66.7%
% Mixed race -- -- 1 16.7%
% Other race -- -- 1 16.7%
% High school degree/GED or above 5 26.3% 4 66.7%
WHO-ASSIST highest score (M, SD), baseline 25.63 (5.95) -- 36.33 (2.66) --
WHO-ASSIST highest score (M, SD), post-treatment 23.50 (10.40) -- 6.80 (10.23) --
% moderate to high risk on WHO-ASSIST*
 Alcohol 19 100% 4 66.7%
 Opioids -- -- 5 83.3%
 Cannabis 2 10.5% 2 33.3%
 Cocaine -- -- 3 50.0%
Open in a new tab
*

Categories are not mutually exclusive and therefore percentages do not add up to 100%

Table 3.

Descriptive Data on PRS Role and Content Fidelity Per Session

Project Khanya Sessions (N = 36) Peer Activate Sessions (N = 10)
M (SD) Range M (SD) Range
ENACT total average score 1.69 (.28) 1.00–2.00 1.98 (.08) 1.75–2.00
ENACT verbal communication skills 1.61 (.49) 1–2 2.00 (0) --
ENACT self-disclosure quality 1.90 (.32) 1–2 1.88 (.35) 1–2
ENACT exploration, interpretation and normalization of feelings 1.61 (.49) 1–2 2.00 (0) --
ENACT empathy, warmth, and genuineness 1.81 (.40) 1–2 2.00 (0) --
Peer disclosures .97 (1.90) 0–11 1.30 (1.34) 0–4
% Sessions with at least 1 peer disclosure (n) 53% (19) -- 70% (7) --
Stigmatizing behaviors .03 (.17) 0–1 .30 (.48) 0–1
Content fidelity – independent rater* .92 (.13) .13–1.00 .69 (.25) .23–1.00
Content fidelity – interventionist* .97 (.07) .63–1.00 .97 (.09) .70–1.00
Open in a new tab
*

Content fidelity is reported as a proportion of completed intervention content

Next, we examined the association between content fidelity, peer disclosures, and average session ENACT scores. All correlations were small and not statistically significant: correlation between content fidelity and average ENACT score (r = −.10, p = .49), content fidelity and peer disclosures (r = −.04, p = .76), and peer disclosures and average ENACT score per session (r = .07, p = .64). Content fidelity did not differ between sessions where a peer disclosure occurred versus not (t(44) = −.13, p = .89).

The overall model explaining post-treatment substance use using ENACT average score and peer disclosures, controlling for baseline substance use and study (Project Khanya or Peer Activate) was significant (F(4, 18) = 3.185, p= .04). However, as demonstrated in Table 4, controlling for pre-treatment and study, ENACT average score did not significantly predict post-treatment substance use (B = −.72, p = .96) nor did peer disclosures (B = 3.41, p = .19).

Table 4.

Regression Analysis Model Summary for PRS Role Fidelity Predicting Patient Outcomes

Post-treatment WHO-ASSIST score
B(SE) p-value
Peer disclosures 3.41 (2.51) .19
ENACT total average −.55 (6.15) .97
Baseline WHO-ASSIST score .49 (.42) .25
Intervention (Project Khanya) 23.24 (7.36) .01
Open in a new tab

Note. B = unstandardized beta. Overall model R2 = .415.

Discussion

This study describes a novel approach for evaluating fidelity of a PRS-delivered intervention, which captures both fidelity to elements unique to the PRS role in addition to traditional content fidelity. Capturing this data allowed for the exploration of how the PRS role may be associated with intervention outcomes, and its relationship with content fidelity. Moreover, fidelity monitoring in traditional settings highlight the importance of measuring both intervention adherence and interventionist competence (Carroll et al., 2000; Madson & Campbell, 2006; Hogue et al., 2008). Across interventions, high content and PRS role fidelity were reported, though a significant difference was found between independent and interventionist-rated content fidelity. Further, there also was no significant difference between content fidelity in sessions with peer disclosures versus those without, suggesting that PRS role fidelity may not come at the cost of sacrificing content fidelity. Finally, PRS role fidelity was not significantly related to substance use at post-treatment.

Results demonstrated high content fidelity and PRS role fidelity across both projects, and significant differences between independent ratings and interventionist self-reported content fidelity. Our results suggest that independent ratings do offer distinct information on fidelity compared to self-reported fidelity. It is possible that this difference is in part due to inaccurate retroactive recall by PRSs, whereas independent raters scored fidelity while listening to session recordings. Discrepancies may also result from training in the use, and purpose of, the fidelity monitoring approach, or distinct perceptions of what constituted fulfilling a component of the intervention. However, fidelity was high for both types of ratings.

While PRS role fidelity (i.e., ENACT scores and self-disclosures) was similar across studies, content fidelity presented as higher in Project Khanya than Peer Activate. Although the Peer Activate content fidelity scores were lower than Project Khanya, this was a reflection of how content fidelity was assessed differently across studies. Given the flexible nature of Peer Activate, content fidelity could not be measured as a proportion of session-specific content delivered, but rather was measured using items that could be flexibly delivered throughout the intervention, which is expected to be lower than a proportion of fixed content covered in each session. Finally, there was also a low frequency of stigmatizing behaviors from PRSs across both studies. Although researchers have speculated that peer interventionists are less stigmatizing than other providers (Jack et al., 2018), there is limited empirical data on this topic.

Although the PRS role was not significantly associated with post-treatment substance use risk, the unique role of a PRS is their shared, lived experience, affording them the ability to relate to clients on a more personal level, which may also help foster empathy, a stronger therapeutic alliance (Leonard et al., 2018) and improved substance use outcomes, including better retention (Meier et al., 2005; Miller & Moyers, 2015; Moyers & Miller, 2013). Thus, PRS role fidelity in task-sharing interventions remains important to monitor. It will be important to evaluate the relationship between content fidelity, PRS role fidelity, and elements of the PRS role in relation to substance use outcomes in larger sample sizes.

This study presents a novel approach to assessing PRS fidelity, utilizing multiple samples, and contributes to the growing evidence-base for PRS-led interventions. However, these findings must be considered in the context of key study limitations. First, we recognize that an important limitation of this study was the small sample size, which likely resulted in limited power to detect effects, and generalizability is limited due to each intervention utilizing only one interventionist. Additionally, the small sample size resulted in an inability to test for factors associated with dropout versus intervention completion. However, given the lack of fidelity on peer-delivered interventions, and the increasing global workforce of PRSs (Satinsky et al., 2021), preliminary data on this topic is sorely needed. Future directions for both studies include planned dissemination efforts in partnership with local community partners and key stakeholders, with an ultimate aim for this work to sustainably shape how PRSs are trained and supervised in these contexts.

Further, we acknowledge the limitations associated with combining data from two distinct trials. While we compared PRS and content fidelity between interventions, the sample sizes, especially for Peer Activate, were quite small. Descriptive comparisons were made across groups; however, the small sample size did not allow for formal statistical comparisons of differences between these studies. We also acknowledge that we used distinct approaches to measuring content fidelity in Peer Activate versus Project Khanya, given the flexible nature of delivering Peer Activate, which resulted in content fidelity representing the proportion of all possible intervention content delivered per session (versus the session-by-session checklist that is typically used in assessing content fidelity). Thus, content fidelity for Peer Activate may appear as lower than it would in a traditional structured intervention. Nonetheless, this flexibility was a key strength of the intervention and was regarded as necessary for feasibility and acceptability. How to appropriately monitor content fidelity in interventions with built-in flexibility while also allowing for peer shared lived experience should continue to be explored. Moreover, future research may consider if PRS role fidelity predicts other relevant clinical outcome variables, including retention and reduced harms associated with use, especially using larger samples, to continue to support the expansion of PRS interventions and inform future PRS trainings.

Conclusion

Fidelity assessments are one of the best indicators of implementation quality (Breitenstein et al., 2010; Wolery, 2011). Establishing PRS fidelity is vital to support more widespread implementation of PRS-delivered services, particularly EBIs. With few studies reporting how to appropriately measure fidelity in PRS interventions (Wolfe et al., 2013; Tracy et al., 2012), particularly beyond traditional content fidelity, this study provides an important step towards understanding how to assess PRS role fidelity in the context of EBIs for underserved individuals with SUD that also incorporates their lived experience and attention to issues of stigma.

Public significance statement:

In order to establish fidelity of peer recovery specialist (PRS) models, it is vital to appropriately measure PRS’ adherence to delivering evidence-based interventions (EBIs), while incorporating PRS lived experience. This paper describes a novel fidelity monitoring approach, piloted in two PRS-led behavioral intervention trials for substance use. Findings suggest that adherence to the PRS role does not take away from adherence to the evidence-based intervention, and that there is value in having an independent rater observe fidelity, in addition to interventionist-reported fidelity.

Acknowledgments

This work was supported by the National Institute of Health (K23DA041901; PI: Magidson) and the UMB-UMCP Research Innovation Seed Grant (PIs: Felton, Doran, Magidson). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

References

  1. Bassuk EL, Hanson J, Greene RN, Richard M, & Laudet A (2016). Peer-delivered recovery support services for addictions in the United States: A systematic review. Journal of Substance Abuse Treatment, 63, 1–9. https://doi-org.proxy-um.researchport.umd.edu/10.1016/j.jsat.2016.01.003 [DOI] [PubMed] [Google Scholar]
  2. Belus JM, Rose AL, Andersen LS, Ciya N, Joska JA, Myers B, Safren SA, & Magidson JF (accepted). Adapting a behavioral intervention for alcohol use and HIV medication adherence for lay counselor delivery in Cape Town, South Africa: A case series. Cognitive and Behavioral Practice. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Bowen S, Chawla N, & Marlatt GA (2011). Mindfulness-based relapse prevention for addictive behaviors: A clinician’s guide. Guilford Press. [Google Scholar]
  4. Breitenstein SM, Gross D, Garvey CA, Hill C, Fogg L, & Resnick B (2010). Implementation fidelity in community-based interventions. Research in nursing & health, 33(2), 164–173. 10.1002/nur.20373 [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Carroll KM, Nich C, Sifry RL, Nuro KF, Frankforter TL, Ball SA, … & Rounsaville BJ (2000). A general system for evaluating therapist adherence and competence in psychotherapy research in the addictions. Drug and alcohol dependence, 57(3), 225–238. [DOI] [PubMed] [Google Scholar]
  6. Daughters SB, Magidson JF, Anand D, Seitz-Brown CJ, Chen Y, & Baker S (2017). The effect of a behavioral activation treatment for substance use on post-treatment abstinence: a randomized controlled trial. Addiction, 113(3), 535–544. 10.1111/add.14049 [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Degenhardt L, Glantz M, Evans-Lacko S, Sadikova E, Sampson N, Thornicroft G, et al. (2017). Estimating treatment coverage for people with substance use disorders: An analysis of data from the World Mental Health Surveys. World Psychiatry, 16(3), 299–307. 10.1002/wps.20457 [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Eddie D, Hoffman L, Vilsaint C, Abry A, Bergman B, Hoeppner B, Weinstein C, & Kelly JF (2019). Lived Experience in New Models of Care for Substance Use Disorder: A Systematic Review of Peer Recovery Support Services and Recovery Coaching. Frontiers in psychology, 10, 1052. 10.3389/fpsyg.2019.01052 [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Gearing RE, El-Bassel N, Ghesquiere A, Baldwin S, Gillies J, & Ngeow E (2011). Major ingredients of fidelity: a review and scientific guide to improving quality of intervention research implementation. Clinical psychology review, 31(1), 79–88. 10.1016/j.cpr.2010.09.007 [DOI] [PubMed] [Google Scholar]
  10. Gouse H, Magidson JF, Burnhams W, Remmert JE, Myers B, Joska JA, & Carrico AW (2016). Implementation of cognitive-behavioral substance abuse treatment in Sub-Saharan Africa: treatment engagement and abstinence at treatment exit. PloS one, 11(1), e0147900. 10.1371/journal.pone.0147900 [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Hogue A, Dauber S, Chinchilla P, Fried A, Henderson C, Inclan J, … & Liddle HA (2008). Assessing fidelity in individual and family therapy for adolescent substance abuse. Journal of substance abuse treatment, 35(2), 137–147. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Jack HE, Oller D, Kelly J, Magidson JF, & Wakeman SE (2018). Addressing substance use disorder in primary care: the role, integration, and impact of recovery coaches. Substance abuse, 39(3), 307–314. 10.1080/08897077.2017.1389802 [DOI] [PubMed] [Google Scholar]
  13. Kemp CG, Petersen I, Bhana A, & Rao D (2019). Supervision of Task-Shared Mental Health Care in Low-Resource Settings: A Commentary on Programmatic Experience. Global health, science and practice, 7(2), 150–159. 10.9745/GHSP-D-18-00337 [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Keynejad RC, Dua T, Barbui C, & Thornicroft G (2017). WHO Mental Health Gap Action Programme (mhGAP) Intervention Guide: a systematic review of evidence from low and middle-income countries. Evidence Based Mental Health, 21(1), 30–34. 10.1136/eb-2017-102750 [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Kohrt BA, Ramaiya MK, Rai S, Bhardwaj A, & Jordans MJD (2015). Development of a scoring system for non-specialist ratings of clinical competence in global mental health: a qualitative process evaluation of the Enhancing Assessment of Common Therapeutic Factors (ENACT) scale. Global Mental Health, 2. 10.1017/gmh.2015.21 [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Lejuez CW, Hopko DR, Acierno R, Daughters SB, & Pagoto SL (2011). Ten Year Revision of the Brief Behavioral Activation Treatment for Depression: Revised Treatment Manual. Behavior Modification, 35(2), 111–161. 10.1177/0145445510390929 [DOI] [PubMed] [Google Scholar]
  17. Leonard HD, Campbell K, & Gonzalez VM (2018). The relationships among clinician self-report of empathy, mindfulness, and therapeutic alliance. Mindfulness, 9, 1837–1844. 10.1007/s12671-018-0926-z [DOI] [Google Scholar]
  18. Livingston JD, Milne T, Fang ML, & Amari E (2012). The effectiveness of interventions for reducing stigma related to substance use disorders: a systematic review. Addiction, 107(1), 39–50. 10.1111/j.1360-0443.2011.03601.x [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Madson MB, & Campbell TC (2006). Measures of fidelity in motivational enhancement: a systematic review. Journal of substance abuse treatment, 31(1), 67–73. [DOI] [PubMed] [Google Scholar]
  20. Magidson JF, Joska J, Belus JM, Andersen LS, Regenauer KS, Rose A, Myers B, Majokweni S, O’Cleirigh C, & Safren SA (2021). Project Khanya: Results from a pilot randomized type 1 hybrid effectiveness-implementation trial of a peer-delivered behavioral intervention for art adherence and substance use in HIV care in South Africa. Journal of the International AIDS Society, 24. doi: 10.1002/jia2.25720 [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Magidson JF, Joska JA, Myers B, Belus JM, Regenauer KS, Andersen LS, … Safren SA (2020). Project Khanya: a randomized, hybrid effectiveness-implementation trial of a peer-delivered behavioral intervention for ART adherence and substance use in Cape Town, South Africa. Implementation Science Communications, 1(1). 10.1186/s43058-020-00004-w [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Marlatt GA, & Gordon JR (1985). Relapse prevention: A self-control strategy for the maintenance of behavior change. New York: Guilford, 85–101. [Google Scholar]
  23. Meier PS, Barrowclough C, & Donmall MC (2005). The role of the therapeutic alliance in the treatment of substance misuse: A critical review of the literature. Addiction, 100(3), 304–316. 10.1111/j.1360-0443.2004.00935.x [DOI] [PubMed] [Google Scholar]
  24. Miller WR, & Moyers TB (2015). The forest and the trees: Relational and specific factors in addiction treatment. Addiction, 110, 401–413. 10.1111/add.12693 [DOI] [PubMed] [Google Scholar]
  25. Morojele NK, Kitleli N, Ngako K, Kekwaletswe CT, Nkosi S, Fritz K, & Parry CD (2014). Feasibility and acceptability of a bar-based sexual risk reduction intervention for bar patrons in Tshwane, South Africa. SAHARA-J: Journal of Social Aspects of HIV/AIDS, 11(1), 1–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Moyers TB, & Miller WR (2013). Is low therapist empathy toxic? Psychology of Addictive Behaviors, 27, 878–884. 10.1037/a0030274 [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Murray LK, Dorsey S, Bolton P et al. (2011). Building capacity in mental health interventions in low resource countries: an apprenticeship model for training local providers. International Journal of Mental Health Systems, 5, 30. 10.1186/1752-4458-5-30 [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Murray LK, Dorsey S, Haroz E, Lee C, Alsiary MM, Haydary A, … & Bolton P (2014). A common elements treatment approach for adult mental health problems in low- and middle-income countries. Cognitive and behavioral practice, 21(2), 111–123. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Pasche S, Kleintjes S, Wilson D, Stein DJ, & Myers B (2015). Improving addiction care in South Africa: Development and challenges to implementing training in addictions care at the University of Cape Town. International Journal of Mental Health and Addiction, 13, 322–332. 10.1007/s11469-014-9537-7 [DOI] [Google Scholar]
  30. Patel V, Weiss H, Chowdhary N, Naik S, Pednekar S, Chatterjee S, … Kirkwood B (2011). Lay health worker led intervention for depressive and anxiety disorders in India: Impact on clinical and disability outcomes over 12 months. British Journal of Psychiatry, 199(6), 459–466. doi: 10.1192/bjp.bp.111.092155 [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Rao D, Elshafei A, Nguyen M et al. (2019). A systematic review of multi-level stigma interventions: state of the science and future directions. BMC Med, 17, 41. 10.1186/s12916-018-1244-y [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Reif S, Braude L, Lyman DR, Dougherty RH, Daniels AS, Ghose SS, Salim O, & Delphin-Rittmon ME (2014). Peer recovery support for individuals with substance use disorders: Assessing the evidence. Psychiatric Services, 65(7), 853–861. https://doi-org.proxy-um.researchport.umd.edu/10.1176/appi.ps.201400047 [DOI] [PubMed] [Google Scholar]
  33. Regenauer KS, Myers B, Batchelder AW, & Magidson JF (2020). “That person stopped being human”: Intersecting HIV and substance use stigma among patients and providers in South Africa. Drug and Alcohol Dependence, 216, 108322. 10.1016/j.drugalcdep.2020.108322 [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Rose-Clarke K, Bentley A, Marston C, & Prost A (2019). Peer-facilitated community-based interventions for adolescent health in low-and middle-income countries: A systematic review. PloS one, 14(1), e0210468. 10.1371/journal.pone.0210468 [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Safren SA, Otto MW, & Worth JL (1999). Life-steps: Applying cognitive behavioral therapy to HIV medication adherence. Cognitive and Behavioral Practice, 6(4), 332–341. 10.1016/S1077-7229(99)80052-2 [DOI] [Google Scholar]
  36. Salwan J, & Katz CL (2014). A Review of Substance Use Disorder Treatment in Developing World Communities. Annals of Global Health, 80(2), 115. 10.1016/j.aogh.2014.04.010 [DOI] [PubMed] [Google Scholar]
  37. Satinsky EN, Doran K, Felton JW, Kleinman M, Dean D, & Magidson JF (2020). Adapting a peer recovery coach-delivered behavioral activation intervention for problematic substance use in a medically underserved community in Baltimore City. PloS one, 15(1), e0228084. 10.1371/journal.pone.0228084 [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Satinsky EN, Kleinman MB, Tralka HM, Jack HE, Myers B, & Magidson JF (2021). Peer-delivered services for substance use in low-and middle-income countries: A systematic review. International Journal of Drug Policy, 95, 103252. 10.1016/j.drugpo.2021.103252 [DOI] [PubMed] [Google Scholar]
  39. SAMHSA. (2019). Key Substance Use and Mental Health Indicators in the United States: Results from the 2018 National Survey on Drug Use and Health. Retrieved from https://www.samhsa.gov/data/sites/default/files/cbhsq-reports/NSDUHNationalFindingsReport2018/NSDUHNationalFindingsReport2018.pdf.
  40. Sikander S, Lazarus A, Bangash O, Fuhr DC, Weobong B, Krishna RN, … & Patel V (2015). The effectiveness and cost-effectiveness of the peer-delivered Thinking Healthy Programme for perinatal depression in Pakistan and India: the SHARE study protocol for randomised controlled trials. Trials, 16(1), 534. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Tracy K, Burton M, Miescher A, Galanter M, Babuscio T, Frankforter T, Nich C, & Rounsaville B (2012). Mentorship for Alcohol Problems (MAP): A peer to peer modular intervention for outpatients. Alcohol and Alcoholism, 47(1), 42–47. https://doi-org.proxy-um.researchport.umd.edu/10.1093/alcalc/agr136 [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. United Nations. #Envision2030 Goal 3: Good Health and Well-being Enable. (2015). Retrieved from https://www.un.org/development/desa/disabilities/envision2030-goal3.html.
  43. Verhey IJ, Ryan GK, Scherer N, & Magidson JF (2020). Implementation outcomes of cognitive behavioural therapy delivered by non-specialists for common mental disorders and substance-use disorders in low-and middle-income countries: a systematic review. International Journal of Mental Health Systems, 14(1), 1–14. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Volkow ND, Frieden TR, Hyde PS, & Cha SS (2014). Medication-assisted therapies—tackling the opioid-overdose epidemic. New England Journal of Medicine, 370(22), 2063–2066. 10.1056/NEJMp1402780 [DOI] [PubMed] [Google Scholar]
  45. WHO ASSIST Working Group. (2002). The alcohol, smoking and substance involvement screening test (ASSIST): development, reliability and feasibility. Addiction, 97(9), 1183–1194. [DOI] [PubMed] [Google Scholar]
  46. Wolery M (2011). Intervention research: The importance of fidelity measurement. Topics in Early Childhood Special Education, 31(3), 155–157. 10.1177/0271121411408621 [DOI] [Google Scholar]
  47. Wolfe H, Haller DL, Benoit E, Bolger KW, Cancienne JC, Ingersoll KS, & Sharp V (2013). Developing PeerLink to engage out-of-care HIV+ substance users: training peers to deliver a peer-led motivational intervention with fidelity. AIDS care, 25(7), 888–894. 10.1080/09540121.2012.748169 [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. World Health Organization. Facts and figures. (2019, December 3). Retrieved from https://www.who.int/substance_abuse/facts/en/.

RESOURCES