Psychometric performance of the Perceived Stigma of Substance Abuse Scale (PSAS) among patients on methadone maintenance therapy in Vietnam

Abstract

Background

People who inject drugs (PWID) and patients on methadone maintenance therapy (MMT) face severe stigma and discrimination. However, there are limited measures assessing stigma towards individuals with substance use disorders or MMT patients, particularly in Vietnam, and few studies have examined the psychometric properties of existing measures. This study aims to examine the performance of the eight-item Perceived Stigma of Substance Abuse Scale (PSAS), created in the United States, among a population of MMT patients in Vietnam.

Methods

A cross-sectional survey was conducted among 400 adult patients enrolled in an urban MMT clinic. Substance use stigma was measured using the PSAS. We conducted confirmatory factor analysis (CFA) and assessed construct validity, reliability and known-group validity.

Results

The original eight-item scale was unidimensional, but yielded mixed goodness-of-fit indicators. Ultimately, dropping two items and allowing the errors of the items related to childcare to correlate improved the goodness-of-fit indicators. (RMSEA=0.025; CFI=0.999; and SRMR=0.018). The shortened scale demonstrated acceptable reliability (Cronbach’s α= 0.766). The mean stigma scores were significantly higher among individuals not working, but did not significantly vary by length of time in MMT, marital status, education attainment, HIV status, depression, concurrent injection drug use or missed methadone dose.

Conclusions

The shortened six-item scale demonstrated good construct validity and acceptability reliability, but it did not demonstrate many a priori hypothesized known-group validity associations. Further research should consider a mixed-methods validation approach.

1. Introduction

“‘Stigma’ is a complex social process that leads to the social disqualification and economic exclusion of individuals or groups based on real or perceived characteristics and is recognised as a key determinant of health and health inequity.(Hatzenbuehler et al., 2013; Weiss et al., 2006) Stigma is conceptualized as a social process that includes: (1) distinguishing and labelling differences, (2) associating negative attributes to those identified differences, (3) separation and distancing, and (4) status loss and discrimination.(Link and Phelan, 2001) While there are several different types of stigma, perceived stigma can be understood as the stigmatized individual’s perceptions of the prevalence of stigmatising attitudes among different groups of people, such as family members, friends, the general community or health care providers.(Nyblade et al., 2021) Stigma constitutes a major barrier to health care seeking and adversely impacts health outcomes, quality of life and economic achievement.(Kane et al., 2019; Sharac et al., 2010; Stangl et al., 2019)

People who inject drugs (PWID) are one such marginalized group that faces severe stigma and discrimination due to negative attitudes towards and lack of understanding around opioid use disorder, as well as the association with criminalization and HIV(Bayat et al., 2020; Lloyd, 2013). However, PWIDs who enroll in opioid use disorder treatment may face additional stigma from seeking treatment and being unable to self-manage their condition. As PWIDs are at heightened risk for HIV, Hepatitis C, other blood-borne pathogens, and a heavy burden of mental health disorders,(Cornford and Close, 2016) there is a critical need to understand the magnitude of stigma and its role in care for PWIDs and patients receiving treatment for opioid use disorder.

The need for such an improved understanding is especially great in Vietnam, where injection drug use plays a critical role in the country’s HIV epidemic, particularly in light of unprotected sex, group injecting and needle sharing practices.(Ahmed et al., 2015; Do et al., 2012) It is estimated that 30% of people living with HIV (PLHIV) in Vietnam also inject drugs.(Nguyen et al., 2018; Tran et al., 2018) In response to the growing problem of injection drug use in the country, the Vietnamese government has invested in harm reduction programs including methadone maintenance therapy (MMT) and needle exchange programs.(Nguyen et al., 2012) Since 2008, when Vietnam began offering MMT programs, MMT has been shown to reduce heroin use and improve quality of life of PWIDs and their families.(Hoang et al., 2015; Tran et al., 2018) However, MMT patients in Vietnam continue to face high levels of stigma and discrimination.(Lan et al., 2018; Tran et al., 2016) There are limited measures of stigma towards individuals with opioid use disorders or MMT patients, particularly in Vietnam, and few studies have examined the psychometric properties of existing measures.(Luoma et al., 2010; Tuliao and Holyoak, 2020)

In order to better characterize stigma among MMT patients and understand its impact on health care engagement and health outcomes, validated drug use-related stigma measures are needed. This study aims to examine the performance of the Perceived Stigma of Substance Abuse Scale (PSAS) among a population of patients on methadone maintenance therapy in Vietnam.(Tuliao and Holyoak, 2020)

2. Methods

2.1. Study Design

A cross-sectional survey was conducted among adult patients enrolled in an urban MMT clinic in Hanoi, Vietnam. Clinic staff approached patients at the clinic to invite them to participate in the survey. Patients who provided informed consent were interviewed in a private room at the clinic. The interviewers included a trained research assistant and a physician. Methods of this study have been previously published.(Mughal, 2021, Pending Publication)

2.2. Measures

Substance use stigma was measured using the eight-item PSAS.(Luoma et al., 2010) This scale was originally developed and validated among a population of patients in treatment for substance use related problems in the United States. In the original development and validation study, the items were adapted from Link’s perceived discrimination–devaluation measures, content validity was assessed by review from substance use stigma experts, and the scale was correlated with internalized shame, self-concealment, internalized stigma, and depression.(Link et al., 1997; Luoma et al., 2010) All items in the scale are rated on a four-point Likert scale (1 = Strongly disagree to 4 = Strongly agree). A higher score indicates a higher level of perceived stigma after reverse coding the positively worded items. Major depressive disorder was identified with the relevant section of the Mini-International Neuropsychiatric Interview (MINI).(Sheehan et al., 1998) Suicidality was identified among those who endorsed either of the MINI questions about suicidal ideation or attempt, “Did you repeatedly think about death, or have any thoughts of killing yourself, or have any intent or plan to kill yourself? Did you attempt suicide?” Any injection drug use in the last three months was defined as reporting injecting drugs on at least one day in the past three months. A missed methadone dose was defined as reported missing at least one dose of methadone in the last month. Educational attainment, employment, HIV status, concurrent injecting drug use, and missed methadone dose were all self-reported.

2.3. Analysis

We conducted confirmatory factor analysis (CFA) using structural equation modeling (SEM) to examine the PSAS and test the pre-specified 1-factor 8-item model. We then investigated whether a shorter scale would perform more satisfactorily than the original. After evaluating the initial model, we re-specified the model using a process of selectively eliminating items with low standardized factor loadings, and then re-assessed model fit. A factor loading threshold of ≥ 0.40 was utilized for retention.(Garson, 2010)

2.3.1. Goodness of fit

To assess model goodness of fit, we computed root mean square error of approximation (RMSEA), comparative fit index (CFI), and standardized root mean square residual (SRMR) goodness-of-fit statistics. (Kline, 2015) RMSEA values ≤ 0.06 were considered desirable, CFI values ≥ 0.9 were consideredto be indicators of good model fit, and SRMR values ≤ 0.05 were c (Hu and Bentler, 1999)

2.3.2. Reliability

We computed Cronbach’s alphas to assess the internal consistency of items.(Gadermann et al., 2012)

2.3.3. Known-group validity

Known-group validity was assessed by examining a priori hypothesized associations between stigma and length of time in MMT, depression, education, employment, HIV status, concurrent injecting drug use, and missed methadone dose.(Hoang et al., 2015) Specifically, based on the literature, we hypothesized that higher stigma would be associated with shorter length of time in MMT, depression, lower education, not working, living with HIV, reporting concurrent drug use and missing a methadone dose.(Do et al., 2021; Lim et al., 2013; Tran et al., 2016; Van Nguyen et al., 2017) To operationalize perceived substance use stigma, we created mean sum-of-item stigma scores. We assessed the correlation between the factor scores and the sum-of-item scores, to ensure that summing item scores was acceptable (Appendix 1).

Analyses were conducted using STATA and MPlus.

2.4. Ethical Considerations

This study received ethics approval from the University of North Carolina at Chapel Hill Institutional Review Board (IRB# 18–3105) and all participants provided verbal consent.

3. Results

3.1. Participant Characteristics

Four hundred patients in MMT participated in this survey. The mean age was 41.3 years, nearly all participants were male, and on average, participants had been in MMT for 4.2 years. Around 73% of participants had a partner, 85% completed at least some secondary school, and 82% were working at least part-time. Around one-fifth of participants were living with HIV and a tenth had depression as indicated by the MINI. (Table 1) For context and comparison, a nationally representative sample of MMT patients found 96.8% of patients were male, 79.8% of patients were under the age of 40, 52.0% were married or living with a partner, 53.9% were working at least part-time, and 33.5% were living with HIV(Hoang et al., 2018).

Table 1.

Study participants (N=400)

Mean (SD) or n (%)
Age	41.3 (7.2)
Sex at birth
Male	397 (99.3%)
Female	3 (0.8%)
Years in MMT (Range 0–11)	4.2 (3.1)
Marital Status
Single	75 (18.8%)
Married or partnered	293 (73.3%)
Widowed/Divorced/Separated	32 (8.0%)
Education
Primary or less	23 (5.8%)
Secondary	340 (85.0%)
Tertiary	37 (9.3%)
Employment
Working at least part-time	329 (82.3%)
Unemployed	64 (16.0%)
Retired	7 (1.8%)
HIV Status
Positive	87 (21.8%)
Negative	313 (78.3%)
Depression	42 (10.5%)
Suicidality	12 (3.0%)
MMT Engagement
Injecting drug use	106 (26.5%)
Missed dose	108 (27.1%)

3.2. Stigma

Participants reported perceiving substance use stigma by endorsing or reporting “strongly agree” or “agree” on the negatively worded items (items 5 and 7) or reporting “strongly disagree” or “disagree” on the positively worded items. (Table 2). Around 45% of respondents endorsed at least half of the items and 5% of respondents endorsed every item. Around 46% of participants agreed most people think less of a person who has been in treatment for substance use. The majority (63–72%) of participants disagreed that someone would be accepted to teach (item 3) or hired to care for children (item 4).

Table 2.

Stigma Items (N=400)

Item Statements	n(%) Strongly disagree	Disagree	Agree	Strongly agree
1. Most people would willingly accept someone who has been treated for substance use as a close friend.	1%	27%	71%	2%
2. Most people believe that someone who has been treated for substance use is just as trustworthy as the average citizen.	3%	36%	59%	2%
3. Most people would accept someone who has been treated for substance use as a teacher of young children in a public school.	9%	54%	37%	1%
4. Most people would hire someone who has been treated to, sub e use to take care of their children.	7%	65%	28%	0%
5. Most people think less of a person who has been in treatment for substance use.	2%	52%	45%	1%
6. Most employers will hire someone who has been treated for substance use if he or she is qualified for the job.	1%	24%	73%	3%
7. Most employers will pass over the application of someone who has been treated for substance use in favor of another applicant.	0%	44%	54%	2%
8. Most people would be willing to date someone who has been treated for substance use.	1%	46%	50%	4%

3.3. Construct Validity

To arrive at a final model, we first examined the original eight-item model, then selectively eliminated items with low standard loadings, and finally examined model modification indices. Table 3 shows the individual survey items with standardized factor loadings of all models examined. While the original pre-hypothesized eight-item model yielded acceptable CFI and SRMR values (CFI=0.927 and SRMR=0.069), the RMSEA was > 0.06, and thus higher than desirable (RMSEA=0.147). Selectively eliminating items with low standard loadings – item 7 and then item 8 – improved the goodness of fit indices, though the RMSEA remained high. An assessment of the model modification indices of the six-item model indicated the errors of items 3 and 4 (capturing acceptance as a teacher of young children and hiring someone to take care their children) were correlated. Allowing the errors of these two items to correlate further improved the goodness-of-fit indicators. The final six-item model that allowed for residual covariance of items 3 and 4 demonstrated improved goodness of fit indices (RMSEA=0.025; CFI=0.999; and SRMR=0.018). Factor scores based on the six and eight-item models were highly correlated (Spearman’s rank-order correlation=0.975), suggesting that little information is lost with fewer items. (Appendix 2)

Table 3:

Confirmatory Factor Analysis Results for the 1-factor models

	Step 1: 8-item		Step 2: 7-item		Step 3: 6-item		Step 4: 6-item*
Item	SFL	SE	SFL	SE	SFL	SD	SFL	SE
1. Close friend	0.762	0.032	0.760	0.035	0.765	0.035	0.788	0.036
2. Trustworthy	0.840	0.027	0.826	0.028	0.831	0.028	0.905	0.030
3. Teacher	0.834	0.021	0.850	0.021	0.847	0.022	0.704	0.030
4. Childcare	0.771	0.026	0.786	0.026	0.789	0.026	0.584	0.042
5. Think less of	0.448	0.043	0.452	0.043	0.429	0.043	0.454	0.045
6. Employers will hire	0.630	0.045	0.560	0.048	0.562	0.048	0.602	0.050
Employers will pass over	0.200	0.047	Dropped		Dropped		Dropped
8. Date	0.699	0.035	0.270	0.051	Dropped		Dropped
Goodness-of-fit In dices
RMSEA	0.147		0.143		0.151		0.025
SRMF	0.069		0.063		0.054		0.018
CFI	0.927		0.941		0.957		0.999

^*residual covariance between item 3 and 4; Standardized factor loading (SFL); Standard Error (SE); Root mean Squared Error of Approximation (RMSEA); Standardized Root Mean Square Residual (SRMR); Comparative Fit Index (CFI)

3.4. Reliability

The original pre-hypothesized eight-item model demonstrated acceptable internal consistency (Cronbach’s α= 0.775). The final six-item scale also demonstrated acceptable internal consistency (Cronbach’s α= 0.766), with the slight reduction likely due to the reduced number of included items.

3.5. Known-group validity

The mean stigma score of the six-item scale was 2.47 (SD: 0.38; Range: 1.5–3.83). There was a small, but significant difference in mean stigma scores by employment; mean stigma scores were significantly higher among individuals not working (mean 2.57, SD 0.47) compared to those working at least part-time (mean 2.45, SD 0.35), but did not significantly vary by length of time in MMT, marital status, education attainment, HIV status, depression, concurrent injection drug use or missed methadone dose. (Table 4)

Table 4:

Mean stigma scores

Mean (SD)		p-value
In MMT care for		0.483
< 1 Year (n=75)	2.50 (0.38)
≥ 1 Year (n=324)	2.46 (0.38)
Marital Status		0.878
Single (n=75)	2.47 (0.32)
Married or partnered (n=293)	2.47 (0.39)
Widowed/Divorced/Separated (n=32)	2.44 (0.36)
Education		0.628
Primary or less (n=23)	2.49 (0.49)
Secondary (n=340)	2.47 (0.37)
Tertiary (n=37)	2.41 (0.34)
Employment		0.017
Working at least part-time (n=329)	2.45 (0.35)
Unemployed/Retired (n=71)	2.57 (0.47)
HIV Status		0.424
Positive (n=87)	2.50 (0.35)
Negative (n=313)	2.46 (0.38)
Depression		0.087
Yes (n=42)	2.56 (0.35)
No (n=358)	2.46 (0.38)
Injecting drug use		0.154
Yes (n=106)	2.51 (0.37)
No (n=294)	2.45 (0.38)
Missed Methadone dose		0.586
Yes (n=108)	2.48 (0.38)
No (n=290	2.46 (0.37)

t-test or one-way ANOVA

4. Discussion

In this cross-sectional survey among MMT patients in Vietnam, substance use stigma was measured using the PSAS. While the original eight-item scale performed as a unidimensional measure and yielded acceptable CFI and SRMR values, the RMSEA was higher than desirable. Ultimately, dropping two items and allowing the errors of the items related to childcare to correlate improved the goodness-of-fit indicators without losing much information. Both the original and shortened scales demonstrated acceptable internal consistency. However, while mean stigma scores were significantly higher among individuals not working compared to those at working at least part-time, scores did not significantly vary by length of time in MMT, marital status, education attainment, HIV status, depression, concurrent injection drug use or missed methadone dose.

Our final six-item version of the PSAS performed as a unidimensional measure and demonstrated good construct validity and acceptable reliability. Other studies in Vietnam and across the globe have also used or drawn from the PSAS to measure substance use stigma. Another study in Vietnam that used both the PSAS and the Perceived Devaluation and Discrimination Questionnaire (Link et al., 1997) found that the PSAS had adequate internal consistency (α = 0.73), but did not evaluate other psychometric properties of the scale.(Lan et al., 2018) A different study in Vietnam used questions from the PSAS to develop a new 5-item measure.(Van Nguyen et al., 2017). Another study developed and validated a Taiwanese version of the PSAS, but added an item asking about the Chinese concept of “losing face”.(Chang et al., 2020) In Chang et al.’s psychometric analysis, the scale also performed as a unidimensional model, but ultimately item 7 – the item about employers passing over an application- was also dropped.(Chang et al., 2020) Elsewhere in the world, a study in the United States that used the PSAS also confirmed the scale’s performance as a unidimensional measure; similarly to our study, they also dropped item 7 and added residual covariance between items 3 and 4 to improve the construct validity and reliability of the scale.(Tuliao and Holyoak, 2020) From a substantive standpoint, it is unsurprising that these two items are highly correlated as they ask about different aspects of childcare. This body of evidence suggests that the more parsimonious PSAS scale may be perform better than the full scale in the Vietnam setting.

Our final six-item PSAS did not perform as expected in the known-group validity analyses. Only one of the a priori hypothesized associations – that individuals who were not working would have higher stigma scores compared to those working at least part-time – proved to be statistically significant in the study population. While the difference in mean stigma scores was small, other studies in Vietnam support this trend. A qualitative study in Vietnam found that stigma created major barriers to employment, resulting in poverty and inability to be self-reliant.(Trang et al., 2020) A another study in Vietnam found that part-time employed PWID living with HIV reported more experiences and perceptions of drug-related stigma compared to full-time employed PWID living with HIV.(Lim et al., 2013) Stigma scores did not vary significantly by length of time in MMT, marital status, education attainment, HIV status, depression, concurrent injection drug use or missed methadone dose as expected. This contrasts with other studies in Vietnam that found that higher stigma associated with depression or anxiety and being an MMT for a shorter period of time.(Tran et al., 2016; Van Nguyen et al., 2017) However, similar to this study, another study in Vietnam that included PWIDs both in and out of MMT found that drug-related stigma was not associated with utilization of MMT.(Lan et al., 2018) In regards to HIV status, it is possible that perceived substance use stigma is less relevant to PLHIV than HIV stigma or intersectional HIV-substance use stigma. Several studies in Vietnam among substance using or MMT populations with HIV have recognized the intersectional and compounding natures of HIV and substance use stigma.(Li et al., 2020; Lim et al., 2013; Rudolph et al., 2012) More robust measures of different types and forms of stigma may offer useful insights into their role in care.

4.1. Limitations

This study highlights the distribution of perceived substance use stigma among the particularly vulnerable population of MMT patients, who have a heavy burden of both HIV and depression. However, there are several limitations inherent to this study. The study was limited by the relatively small sample size and convenience sampling strategy. We did not collect retest data and could not evaluate test-retest reliability. There is the possibility of measurement error in the known-group validity assessment as HIV status, concurrent drug use and missing a methadone dose were all self-reported and depression was not diagnosed, but identified through a positive screen on the MINI. While this study strove to assess the validity of the PSAS in a culture quite different than the United States, additional qualitative work could have provided greater insights into the content validity of the scale and confirmed whether the PSAS does indeed capture the key aspects of perceived substance use stigma in Vietnam. In the inherent absence of a gold standard criterion, future use of this stigma scale may benefit from mixed-methods validation work, to ensure the scale is adequately capturing the intended stigma construct.

5. Conclusion

This cross-sectional study documents the prevalence of perceived substance use stigma among MMT patients in Vietnam using the PSAS and evaluates the scale’s psychometric properties. Ultimately, dropping two items and allowing the errors to correlate between the two childcare items improved the scale’s construct validity. The shortened six-item scale demonstrated acceptability reliability. However, as higher stigma scores were only associated with not working, the scale did not demonstrate many a priori hypothesized associations in the known-group validity analyses. While this shortened measure will allow for reliable assessment of the burden of perceived substance use stigma among MMT patient, future use of this stigma scale may benefit from mixed-methods validation work. Finally, future research using the PSAS might consider modifying the scale to capture parallel community-level stigma by adapting the questions to assess the degree of these stigmatizing attitudes among non-opioid using community members. As substance use stigma has the potential to disrupt healthcare seeking and MMT engagement as well as create barriers to employment and adversely impact quality of life,(Lan et al., 2018) further investment in efforts to address the deleterious effect of substance use stigma are urgently needed.

Highlights.

• Stigma scale assessed among methadone maintenance patients in Vietnam

• Reduced six-item scale demonstrated good construct validity and acceptability reliability

• Stigma Scores only associated with employment

• Further mixed-methods validation work recommended