A Systematic Review and Psychometric Appraisal of Instruments Measuring Tuberculosis Stigma in Sub-Saharan Africa

Abstract

Tuberculosis (TB) stigma is one barrier to TB testing, treatment uptake and treatment completion. Therefore, stigma measurement must be approached through rigorous scientific methodology in order to accurately and reliably estimate the impact of TB stigma on treatment outcomes. The aim of this systematic review is to evaluate the methods and instruments used to measure TB stigma and interrogate strategies used to culturally validate measures of TB stigma in global research. Two reviewers used the PRISMA method to extract and analyze the existing body of literature on TB stigma in Sub-Saharan Africa. A thorough search was performed using three data bases generating 2,302 independent studies. After systematic screening, this review includes 28 studies. Of those studies, 13 used a psychometrically validated instrument while 15 used informal questionnaires or proxy variables to measure stigma. Psychometric appraisal was limited due to the number of studies that measured stigma using unvalidated questionnaires or proxy variables. The Patient and Community Perceptions of TB scales validated by Van Rie et al. were the most commonly used instruments to measure TB stigma; additionally, many instruments were not culturally or linguistically validated in Sub-Saharan Africa. Our appraisal emphasizes the need for reliable and valid instruments to measure TB stigma in low- and middle-income countries most affected by TB.

Background

One quarter of the world’s population is living with tuberculosis (TB) (MacNeil et al., 2020). Although effective treatment and prevention methods exist, TB persists at epidemic proportions in low- and middle-income countries (LMICs). According to the World Bank, 22 of the world’s 29 low-income economies are in Sub-Saharan Africa, where more than half of the world’s high-burden TB countries are also located (The World Bank, 2020; World Health Organization, 2019). Antimicrobial treatment completion is the most effective way to disrupt TB transmission and decrease the risk for infection. TB treatment is often complicated by its association with HIV/AIDS which contributes to additional morbidity and mortality in patients living with both TB and HIV (Falzon et al., 2017). TB treatment is further complicated by the emergence of multi-drug resistant and extensively drug resistant strains of TB increase treatment duration and require physically taxing treatment courses. Patients may also experience judgement and shame due to culturally held fears and preconceptions of what TB is and who it affects. Perceived, anticipated, internalized and experienced TB stigma are significant impediments to TB testing, linkage and treatment adherence (Cremers et al., 2015; A. Daftary et al., 2007; Amrita Daftary & Padayatchi, 2012; De Schacht et al., 2019; Gebremariam et al., 2010; Møller et al., 2011; Wachira et al., 2014). Despite ample evidence that people living with TB in Sub-Saharan Africa perceive and experience stigma, there is poor consensus within the medical research community about how to quantify TB stigma accurately and reliably.

Stigma is a social determinant of health which hinders TB eradication efforts, including contact tracing for high-risk persons, uptake of isoniazid preventive treatment, and successful completion of TB treatment (Cremers et al., 2015; Amrita Daftary & Padayatchi, 2012; Gebremariam et al., 2010; Méda et al., 2014). Social determinants of health are the born or lived environmental conditions that shape differences in health and health experiences (US Department of Health and Human Services, 2010). Each person is born into a cultural environment with implied normative behaviors, characteristics and traits. Cultural norms contribute to stigma when the “normal” or default status provides the reference point for good, positive, or moral character. In comparison, “othered” people, who exist apart from the normative group, are aberrations. In his foundational work on stigma, Erving Goffman asserts that stigma dehumanizes the othered person. This dehumanization justifies increased morbidity and mortality of people living with TB stigma and results in disparate healthcare access (Goffman, 1963).

Measuring and understanding stigma as a deterrent to care is integral to advancing TB eradication efforts. In 2015, The World Health Organization outlined a strategy to end TB worldwide by the year 2035. Their strategic goals for TB elimination include decreasing TB-related deaths, decreasing new TB infections, and eliminating extreme debts due to the costs of treating TB (The World Health Organization, 2015). Initiating and completing a four-drug, anti-TB treatment regimen is the most effective way to halt the chain of transmission, promoting both individual and public safety (Nahid et al., 2016). Improving rates of TB treatment completion will decrease the death rate associated with tuberculosis and prevent new infections by reducing the reservoir of infectious hosts.

Decreasing TB stigma may increase access to TB care and retention in treatment. To accurately measure changes in TB stigma, researchers must utilize well-validated instruments that fully capture the experience of stigma, while differentiating stigma from related constructs. Unlike manifest variables, latent constructs, like TB stigma, cannot be directly measured or observed. Researchers use indicators to represent the underlying concept and thereby inferentially measure the latent construct.

In order to ensure that the measurement model accurately reflects the underlying latent variable, rigorous validation is necessary. Content, criterion and construct validity are three generally accepted measures that represent the broadest interpretation of overall instrument validity. Content validity is a measure of whether or not the instrument items adequately and completely represent the construct and is often assessed in a content validity index completed by subject matter experts (DeVellis, 2016). Subject matter experts may include scholars and, or people most familiar with the construct of interest. Here, people living with TB who perceive, experience or internalized stigma, may be the most qualified experts. Criterion validity is often measured by comparing a new instrument to a “gold standard,” or the currently accepted measurement standard (DeVellis, 2016). Criterion validity may be difficult to assess in latent variable measurement. Because researchers cannot directly measure the latent construct, they must compare novel instruments to existing measures which may themselves include systematic measurement error. Finally, construct validity compares the findings of the instrument to other measures within the nomological network. Construct validity is highly theoretical and attempts to validate the instrument through its positive or negative associations with other theoretically related constructs (DeVellis, 2016). To evaluate the instrument items, psychometric validation uses statistical techniques to analyze item groupings and their association with related constructs. To maximize both the use and impact of latent scales, methodological precision is essential. As we appraise the studies included in this review, we consider whether the instruments used to measure TB stigma were tested for content, criterion and, or construct validity.

This systematic review synthesizes the available quantitative literature on TB stigma evaluating the methods and instruments used to measure TB stigma and the steps that researchers take to ensure cultural and linguistic validity. This review will direct future TB stigma research by identifying gaps in the current research methodology.

Methods

This review used the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) method for guidance in searching, assembling and synthesizing the literature. No new data was generated for this analysis and was therefore IRB exempt.

Search Methods

For this review of literature, we worked with a library informationist to develop a search strategy for three databases: Cumulative Index to Nursing and Allied Health Literature (CINAHL), PubMed and Embase. Keywords, phrases and truncations for the search strategy included variations of the terms “Tuberculosis or TB” and “stigma” which produced an initial 2,302 articles for title and abstract screening. After eliminating duplicates, studies conducted outside of Sub-Saharan Africa and articles that addressed only HIV stigma, 91 studies moved forward for full text screening.

Inclusion and Exclusion Criteria

Quantitative studies were included in the review if they disclosed the methods used to measure stigma, if they measured TB stigma as the primary outcome of interest, or as an independent variable contributing to a clinical outcome among TB patients. This review includes articles that explore TB stigma from within persons with TB, excluding studies that pertain to general community level stigma or healthcare provider stigma. Other inclusion criteria were adult study population, publication in a peer-reviewed journal, and research conducted in Sub-Saharan Africa due to regional variation in stigma based on sociocultural values and religious and political norms. This review also excluded systematic reviews, integrative reviews and meta-analysis. We excluded gray literature including abstracts, speaker presentations, and editorials, as well as studies that were not published in English.

Procedure

Two independent reviewers (A.B. and K.M.) first reviewed study titles and abstracts, and then full text articles, resolving conflicts through discussion and consensus. The reviewers individually appraised each study that met the inclusion criteria and extracted data using a pre-developed form to ensure the review’s integrity. No studies were excluded due to poor or low quality upon appraisal. Studies that used single variables or unvalidated questionnaires to measure stigma had an increased risk of bias and are therefore evaluated separately. Figure 1 demonstrates the screening process following PRISMA guidelines.

Figure 1.

Prisma Diagram

For the purposes of this review, we defined rigorously validated instruments as instruments that 1) had a formal assessment of content validity including an appraisal of clarity and relevance; and that 2) evaluated construct validity using statistical data reduction techniques to evaluate the covariance among either the instrument items or the participants (factor analysis or latent class analysis). Questionnaires that were constructed based on literature review, expert review or qualitative analysis alone, did not meet the criteria for “rigorously validated”. Because latent constructs often lack a measurement “gold standard,” we did not include criterion validity in our definition of rigorous validation. Using these criteria, reviewers divided the studies into two groups for synthesis, 1) studies that measured stigma with a rigorously validated instrument and 2) those that used questionnaires or proxy indicators to measure stigma.

Results - Study Characteristics

This search generated 28 studies in total. Most of the studies were conducted in Ethiopia (Adenager et al., 2017; Ambaw et al., 2019; Asefa & Teshome, 2014; Assefa et al., 2017; Ayana et al., 2019; Duko et al., 2015, 2019; Mohammedhussein, Alenko, et al., 2020; Mohammedhussein, Hajure, et al., 2020; Molla, Mengesha, et al., 2019; Molla, Mekuriaw, et al., 2019;), but this review also includes research from ten additional countries in Sub-Saharan Africa. 13 of the 28 studies used a rigorously validated instrument to measure TB or TB/HIV stigma (Bresenham et al., 2020; Duko et al., 2015; E. Hayes-Larson et al., 2017; Eleanor Hayes-Larson et al., 2017; Kautzky & Tollman, 2008; Mbuthia et al., 2020; Mohammedhussein, Alenko, et al., 2020; Mohammedhussein, Hajure, et al., 2020; Molla, Mekuriaw, et al., 2019; Molla, Mengesha, et al., 2019; Naidu et al., 2020; Nkambule et al., 2019; Zetola et al., 2012) and are summarized in table one. Of the remaining studies, five used proxy indicators or single items to identify stigma (Abioye et al., 2011; Assefa et al., 2017; Muture et al., 2011; Nyangoma et al., 2020; Peltzer et al., 2012) and ten used questionnaires specifically created for their study to measure stigma (Adenager et al., 2017; Ambaw et al., 2019; Asefa & Teshome, 2014; Ayana et al., 2019; Chileshe & Bond, 2010; Cremers et al., 2015; Levin et al., 2006; Méda et al., 2014; Osei et al., 2015); all of these proxy variables and questionnaires are summarized in table two. Although several studies used qualitative exploration or pilot tests to evaluate their instruments, none of the studies using proxies or questionnaires conducted a formal assessment of content validity or used statistical data reduction to evaluate the psychometric properties of their instruments. Table two includes details about if and how qualitative exploration contributed to questionnaire development, as well as pilot testing. This review will compare and contrast the instruments, questionnaires and proxy variables, used to measure TB stigma.

Table 1.

Studies measuring stigma with rigorously validated instruments

First Author, Year, Cronbach’s α	Country	Stigma scale used	Subscales and # of indicators used	Adaptations to the original scale? Y/N	Qualitative exploration of the latent construct for cultural relevance in context? Y/N	Did authors perform a statistical assessment of cultural relevance in context? Y/N	Stigma measurement (dichotomous, categorical, or continuous)	Stigma mechanism (perceived, anticipated, enacted, internalized)
Bressenham, 2020, α = 0.86-0.93	South Africa	Van Rie TB Stigma Scale	Subscale: Community perspectives of TB Scale – 8 of 11 indicators	Y – based on formative work 3 items were removed	Y – conducted cognitive interviews to explore instrument interpretation	Y – factor analysis performed within target population led to scale adaptation	Continuous	Perceived TB Stigma
Duko, 2015, α = 0.89	Ethiopia	Van Rie Perceived TB Stigma Scale	Subscale: Patient Perspectives of TB Stigma – 12 of 12 indicators	N	N	N	Dichotomous – used the sample mean value as a cut point	Perceived TB stigma
Duko, 2019, α = 0.82-0.91	Ethiopia	Van Rie Perceived TB Stigma Scale	Subscale: Patient Perspectives of TB Stigma – 12 of 12 indicators	N	N	N	Dichotomous – used the sample mean value as a cut point	Perceived TB stigma
E. Hayes-Larson 2017, α = N/A	Lesotho	Van Rie TB Stigma Scale & 3 novel indicators developed in Lesotho to measure internalized stigma	Subscale: Community Perspectives of TB Stigma – 11 of 11 indicators 3 novel items added for internalized stigma	N	N	Y – factor analysis performed on all 14 items within the target population	Continuous	Perceived & internalized TB stigma
Eleanor Hayes-Larson 2017, α- N/A	Lesotho	Van Rie TB Stigma Scale	Subscale: Community perspectives of TB Stigma – 11 of 11 indicators	N	N	Y – validated using exploratory factor analysis published by E. Hayes Larson et al 2017	Continuous	Perceived & internalized TB stigma
Mbuthia 2020, α = 0.87 (community) & 0.86 (patient)	Kenya	Van Rie TB Stigma Scale	Full scale: Patient & community perspectives of TB stigma. 23 of 23 items	N	Y – qualitative exploration within the target population	Y – factor analysis performed within the target population	Continuous	Perceived & felt TB stigma
Mohammed-hussein & Hajure 2020α =0.84	Ethiopia	Somma et al. perceived TB stigma scale	Full scale – 11 indicators	N	N	N – however the scale was pretested among an independent sample of 20 people living with TB in Oromo & Amharic languages	Dichotomous – used the sample mean value as a cut point	Perceived TB stigma
Mohammed-hussein & Alenko, 2020α =0.78	Ethiopia	Somma et al. perceived TB stigma scale	Full scale - 11 indicators	N	N	N– however the scale was pretested among an independent sample of 20 people living with TB in Oromo & Amharic languages	Dichotomous – used the sample mean value as a cut point	Perceived TB stigma
Molla & Mengesha, 2019a, α = 0.92	Ethiopia	Van Rie TB Stigma Scale	Subscale: Patient Perspectives of TB Stigma – 12 of 12 indicators	N	N	N	Dichotomous – used the sample mean value as a cut point	Perceived TB stigma
Molla & Mekuriaw, 2019b, α = N/A	Ethiopia	Van Rie TB Stigma Scale	Subscale: Patient Perspectives of TB Stigma – 12 of 12 indicators	N	N	N	Dichotomous – used the sample mean value as a cut point	Perceived TB stigma
Naidu, 2020 α = 0.83 (community), α = 0.85 (patient)	South Africa	Van Rie TB Stigma Scale	Full scale: Patient & community perspectives of TB stigma - 23 of 23 indicators	N	N – an isiZulu speaking psychologist evaluated the instrument for face validity	N	Continuous	Perceived TB stigma
Nkambule 2019, α = 0.9 (community) α = 0.83 (patient)	Swaziland	Van Rie TB Stigma Scale	Full Scale: Patient & community perspectives of TB stigma - 23 of 23 indicators	N	N	Y – confirmatory factor analysis performed within target population	Continuous	Perceived & felt TB stigma
Zetola, 2012, α = N/A	Botswana	Van Rie TB Stigma Scale	*Instrument was “similar” to the Patient Perspectives of TB Stigma, unknown number of indicators	Y – The number & type of adaptations were not specified	N	N	Continuous	Felt TB Stigma

Table 2.

Studies measuring stigma with questionnaires or proxies

First Author, Year, Cronbach’α	Country	How was stigma measured?	Instrument was pilot tested prior to implementation (Y/N)?	Instrument was developed following qualitative exploration?	Statistical data reduction techniques utilized?	Stigma mechanism
Abioye, 2011 α – not applicable	Nigeria	Self-report of experienced stigma	Not applicable	Not applicable	N	Experienced TB stigma
Adenager, 2017, α - not reported	Ethiopia	10 questions measured on a 5-point Likert scale (ex: feeling ashamed of having TB)	Y – the questionnaire was pilot tested on a sample equal to 5% of the sample size in similar health facilities.	N	N	Not reported
Ambaw, 2019, α = 0.84	Ethiopia	Macq adapted perception index* - 10 question instrument developed for a Nicaraguan population.	Y – piloted on 68 people living with TB to ensure understandability	N – original study used qualitative methods in Nicaragua	N	Perceived TB Stigma
Asefa, 2014, α – not reported	Ethiopia	4 questions on a 3-point Likert scale including shame, disclosure, isolation, and relationship	Y – instrument was pretested, no additional details provided	N	N	Not reported
Assefa, 2017, α – not applicable	Ethiopia	TB disclosure status was a proxy for TB stigma	Not applicable	N	N	Perceived TB stigma
Ayana, 2019, α = 0.92	Ethiopia	9-item stigma scale measured using a 4-point Likert system. Instrument was developed by Xu et al. for a Chinese population.	N	N – original study used qualitative methods in China	N	Experienced TB stigma
Bond, 2017, α – not reported	Zambia & South Africa	10 items generated from qualitative literature and informed by Nyblade et al. in their pilot study of HIV stigma indicators.	Y – piloted in 4 Zambian and 2 South African communities in a variety of local languages which helped to select final indicators	Y – extensive expert review and multi country qualitative work	N	Internalized & experienced TB stigma
Cremers, 2015, α - not reported	Zambia	3 questions evaluating positive/negative attitudes or perceptions towards TB. 1) Do you feel shy/shame coming to the clinic? 2) What do people in the place you live/in your neighborhood think about TB? 3) How do they compare HIV and TB? The	Y – All of the structured interview questions including the 3 stigma questions were reviewed by a Zambian physician and Zambian nurses then pilot tested.	N – although this was a mixed methods study, the qualitative interviews did not inform the questionnaire development.	N	Internalized, perceived & experienced TB stigma
Levin, 2006 α – not reported	South Africa	12 Agree/disagree questions related to perceived TB stigma, and general knowledge and believes related to TB and transmission.	N	N	N	Perceived TB stigma
Meda, 2014, α – not reported	Burkina Faso	Macq adapted perception index - 10 question instrument developed for a Nicaraguan population.	Y – instrument was pretested, no additional details provided	N – original study used qualitative methods in Nicaragua	N	Perceived TB stigma
Muture, 2011, α – not applicable	Kenya	Self-report of experienced stigma	Not applicable	Not applicable	N	Experienced TB stigma
Nyangoma, 2020 α - not reported	Uganda	Measured experiences of enacted stigma following TB status disclosure. Predefined negative stigmatizing experiences included criticism, isolation, withdrawal of household support, and marital separation. Participants could also report negative experiences outside of the defined list.	N	N	N	Experienced TB stigma
Osei, 2015, α - not reported	Ghana	Measured using a 4-point Likert scale, did not disclose number of instrument items. Questions including feelings of shame, TB status disclosure, and relationship impact at the individual and community level.	Y – piloted at a non-study site for clarity and acceptability	N	N	Not reported
Peltzer, 2012, α - not applicable	South Africa	Stigma was a response option for the following question: “the main reason for not going for an HIV test”	Not applicable	Not applicable	N	N/A
Westaway, 1989 α - not reported	South Africa	“Social stigma” was measured using 5 multiple choice questions. Questions asked participants to describe the types of people most often affected by TB.	N	N	N	Social (perceived) stigma

Stigma Measured using Proxy Variables

Five studies measured TB stigma using a proxy or single indicator. These studies were conducted across Sub-Saharan Africa in Nigeria (Abioye et al., 2011), Ethiopia (Assefa et al., 2017), Kenya (Muture et al., 2011), Uganda (Nyangoma et al., 2020) and South Africa (Peltzer et al., 2012).

One study used TB status disclosure as a proxy for stigma, where any patients who failed to disclose their TB status met the study definition for stigma, and all patients who disclosed their TB status met the definition for no stigma (Assefa et al., 2017). Three additional studies relied on patient report of experienced stigma for their study definitions of stigmatization (Abioye et al., 2011; Muture et al., 2011; Nyangoma et al., 2020). The final study asked patients their reason for avoiding HIV testing, and included “stigma” as a response choice (Peltzer et al., 2012).

Stigma Measured using Questionnaires

Ten studies measured stigma using an unvalidated questionnaire. Four of these studies were conducted in Ethiopia (Adenager et al., 2017; Ambaw et al., 2019; Asefa & Teshome, 2014; Ayana et al., 2019). The remaining studies were conducted in Burkina Faso (Méda et al., 2014), Ghana (Osei et al., 2015), Zambia (Cremers et al., 2015), South Africa (Levin et al., 2006; Westaway, 1989), and one took place in both Zambia and South Africa (Bond et al., 2017).

Seven studies developed and used their own questionnaires to evaluate stigma without a thorough evaluation of content, or construct validity(Adenager et al., 2017; Asefa & Teshome, 2014; Bond et al., 2017; Cremers et al., 2015; Levin et al., 2006; Osei et al., 2015; Westaway, 1989). These studies included varying levels of detail describing item generation and how their authors measured and analyzed stigma. Bond and colleagues provided the most detail in their description of item generation and measurement; they considered key literature, expert review and qualitative data in an effort to adequately represent the latent variable (2017). They also discussed a preference for a validated measure of TB stigma, but at the time of data collection, no validated instrument was available (Bond et al., 2017). Levin and colleagues provided a copy of their questionnaire which included questions about perceived TB as well as general TB knowledge (2006); however, they do not offer any insight into how they developed their questions.

Some of the studies in this review used a previously cited but unvalidated instrument for measurement of TB stigma. Two studies in this review used an adapted [TB] perception index developed by Macq and colleagues, to measure TB stigma (Ambaw et al., 2019; Méda et al., 2014). The adapted perception index by Macq et al. was grounded in a review of literature and qualitative exploration, then later pilot tested among people living with TB in Nicaragua. However, the reviewers could find no evidence that the instrument was formally evaluated in context for clarity and relevance. Furthermore, Macq et al. did not statistically evaluate the relationships between items or test the instrument’s association with other related constructs. This index was developed for a Nicaraguan population in 2006 and is frequently cited in TB stigma literature. The adapted perception index has not been explored qualitatively or psychometrically interrogated in Ethiopia or Burkina Faso where the studies were conducted. Similarly, Ayana and colleagues used a stigma scale developed by Xu et al. for a Chinese population. The instrument was not evaluated for content, criterion or construct validity and has not been evaluated for cultural or linguistic consistency in Sub-Saharan Africa (2019).

Only two studies using questionnaires reported data on the internal consistency of their questionnaires. Ambaw and colleagues reported Cronbach’s α = 0.84 (2019) and Ayana and colleagues reported Cronbach’s α=0.92 (2019). Both of these values indicate excellent internal reliability, but alpha values greater than 0.90 may indicate inter-item redundancy (Tavakol & Dennick, 2011).

Because none of these studies or their sources used validation techniques to evaluate their questionnaires or proxies, we cannot assume that these measures accurately or adequately represent the construct of stigma. Therefore, there is no way to systematically evaluate their instrument quality. For that reason, the remainder of this review will focus on the 13 studies that used a validated instrument to quantify TB stigma.

Stigma Measured using a Validated Instrument

Among these 13 studies which used a validated instrument, most were conducted in Ethiopia (Duko et al., 2015, 2019; Mohammedhussein, Alenko, et al., 2020; Mohammedhussein, Hajure, et al., 2020; Molla, Mekuriaw, et al., 2019; Molla, Mengesha, et al., 2019). Two studies were conducted in Lesotho (E. Hayes-Larson et al., 2017; Eleanor Hayes-Larson et al., 2017), two in South Africa (Bresenham et al., 2020; Naidu et al., 2020), one was conducted in Kenya (Mbuthia et al., 2020), one in Swaziland (Nkambule et al., 2019) and one was conducted in Botswana (Zetola et al., 2012). Within this group, most reported the internal consistency within the respective data sets using Cronbach’s alpha; however, three did not (E. Hayes-Larson et al., 2017; Eleanor Hayes-Larson et al., 2017; Zetola et al., 2012). All alphas fell within the 0.78-0.95 range indicating high levels of internal consistency with some risk of redundancy (See table 1).

Scales, adaptations, and measurement

The Patient and Community Perceptions of TB scales validated by Van Rie et al. were most commonly used to measure TB stigma. Eleven studies utilized the Van Rie instrument to measure TB stigma among people living with TB (Duko et al., 2015, 2019; E. Hayes-Larson et al., 2017; Eleanor Hayes-Larson et al., 2017; Mbuthia et al., 2020; Molla, Mekuriaw, et al., 2019; Molla, Mengesha, et al., 2019; Naidu et al., 2020; Nkambule et al., 2019; Zetola et al., 2012). The scales were originally developed and tested in Thailand for content and construct validity. Van Rie and colleagues performed exploratory and confirmatory factor analysis independently to assess item covariances. They used a four-point Likert scale to measure TB stigma which is split into two domains, Patient Perceptions of TB Stigma, designed for measuring felt and internalized TB stigma, and Community Perceptions of TB Stigma. The community level scale uses indicators that reflect the wider community’s responses to patients living with TB which may be referred to as perceived stigma (Van Rie et al., 2008). Together, the two domains consist of 23 total items.

Three of the studies used the full 23-item scale including patient and community perspectives. (Mbuthia et al., 2020; Naidu et al., 2020; Nkambule et al., 2019). Five studies used only the Patient Perceptions of TB Scale to evaluate patient level stigma with questions that assess the experiences and feelings of people living with TB (Duko et al., 2015, 2019; Molla, Mekuriaw, et al., 2019; Molla, Mengesha, et al., 2019; Zetola et al., 2012). Three studies used the Van Rie Community Perceptions of TB Scale, measuring perceived TB stigma among people living with TB (Bresenham et al., 2020; E. Hayes-Larson et al., 2017; Eleanor Hayes-Larson et al., 2017).

Two additional studies used an instrument developed by Soma and colleagues to measure perceived TB stigma in Ethiopia (Mohammedhussein, Alenko, et al., 2020; Mohammedhussein, Hajure, et al., 2020). The scale was originally developed and piloted in Bangladesh, India, Colombia and Malawi with 18-items(Somma et al., 2008). The instrument demonstrated acceptable internal reliability in the Malawian population, Cronbach’s alpha 0.63, and revealed the unique impact of HIV stigma on TB stigma that occurred in the Malawian context (Somma et al., 2008).

Seven of the 13 studies measured stigma as a continuous variable (Bresenham et al., 2020; E. Hayes-Larson et al., 2017; Eleanor Hayes-Larson et al., 2017; Mbuthia et al., 2020; Naidu et al., 2020; Nkambule et al., 2019; Zetola et al., 2012) the remaining studies used the mean stigma score within each data set as a cut point to dichotomize stigma into binary variable. Instrument threshold refers to the linearity of the relationship between an instrument measuring an independent variable, and an observable change in the outcome of interest, the dependent variable. Notably, Hayes -Larson and colleagues, were the only investigators to mention instrument thresholds as a consideration for analysis. They argued that since the original Van Rie scales did not specify instrument thresholds, the scale should be measured and analyzed as a continuous variable. Changes to the unit of measurement may affect the clinical applicability of the instrument and therefore may impact the instrument’s predictive and clinical validity.

Cultural and Linguistic Validation

Beaton and colleagues present step-by-step guidance for how to adapt instruments for cross cultural research. Their method includes 1) Initial translation with two independent translators, 2) synthesis of the two translations, 3) back translation, 4) formal expert review and analysis of semantic, idiomatic and experiential equivalence and finally 5) test of the prefinal version among a sample of the target population(Beaton et al., 2000).

Of the nine studies that used validated instruments, five used statistical data reduction techniques to evaluate appropriateness of the instrument in the country under study (Bresenham et al., 2020; E. Hayes-Larson et al., 2017; Eleanor Hayes-Larson et al., 2017; Mbuthia et al., 2020; Nkambule et al., 2019). Mbuthia and colleagues took great care to establish cultural and linguistic relevance in Kenya. Their research team conducted a mixed methods study to validate the Van Rie scale within a pastoralist community. The research team piloted the instrument, conducting an exploratory factor analysis while simultaneously conducting in-depth interviews and focus group discussions with a separate sample to explore the depth and breadth of the latent construct. They triangulated the qualitative and quantitative methods to evaluate the instrument for clarity, relevance and completeness(Mbuthia et al., 2020).

Hayes-Larson and colleagues used the previously validated Van Rie Community TB Scale and additionally used three novel items to measure internalized TB stigma in Lesotho. The three additional items were, ‘you are ashamed to have TB’, ‘people close to you would avoid you if they thought you had TB’ and ‘some people who have TB feel hurt because of how others react to knowing they have TB’. They performed an exploratory factor analysis on the total 14-items, but did not qualitatively explore the construct.

Bresenham et al. also conducted cultural analysis of the community perspectives of TB stigma developed by Van Rie. In a separate study, they performed cognitive testing and factor analysis within the South African population that led to scale adaptation. Due to poor performance in exploratory work, they removed three items from the scale and continued their research with eight of the original 11 items. Finally, Nkambule and colleagues also performed a confirmatory factor analysis with of the Van Rie scale within the Swazi population (Nkambule et al., 2019). However, they used the entire Van Rie scale, patient and community perspectives of TB stigma with acceptable fit TLI=94, CFI=0.88 and RMSEA=0.11 .

Four studies used the Van Rie scale in Ethiopia where it has not been culturally validated or linguistically evaluated in Amharic (Duko et al., 2015, 2019; Molla, Mekuriaw, et al., 2019; Molla, Mengesha, et al., 2019). Two studies also used the Soma et al. scales in Ethiopia without extensive cultural or linguistic validation, instead relying on two isiZulu speaking clinicians for translation and confirmation of face validity (Mohammedhussein, Alenko, et al., 2020; Mohammedhussein, Hajure, et al., 2020).

Appraisal of the Van Rie Community and Patient Perspectives Towards Tuberculosis scales

The studies in this review use the Van Rie Scales and the Soma et al. scales to evaluate TB related stigma. Here we will use the Quality Criteria for Health Status Questionnaire to review the Van Rie Scales and their psychometric properties in depth (Terwee et al., 2007). This questionnaire represents a standard criterion for evaluating instrument strength including 1) content validity 2) internal consistency 3) criterion validity 4) construct validity 5) reproducibility 6) responsiveness 7) floor and ceiling effects and 8) interpretability. We appraised the two Stigma Scales using these criteria in table 3. Overall, the Van Rie scales scored well in internal consistency, construct validity and floor and ceiling effects; moderately in content validity and reproducibility; and did not address criterion validity, responsiveness or interpretability (See table 3).

Table 3.

Appraisal of the Van Rie TB Stigma Scale (Van Rie et al., 2008)

Criteria	Evaluation
Content validity	Van Rie et al. performed an extensive review of the literature and formulated items based on in-depth interviews and focus group discussions with patients and key informants. However, their assessment of content validity lacked a formalized strategy for developing a validity and clarity index. Their content validity was further limited because they did not clearly state the aim of their instrument. Readers may infer that their goal was a discriminative instrument, but this is purely speculative.
Internal consistency	Internal consistency of both Community Perspectives towards Tuberculosis and Patient Perspectives towards Tuberculosis were measured using Cronbach’s alpha 0.90 and 0.83 respectively. The scores indicated excellent reliability without excessive redundancy.
Criterion validity	Van Rie et al. did not explore criterion validity as this was a novel instrument in 2008. There was no gold standard for comparison.
Construct validity	The authors tested construct validity through an a priori hypothesis that their instrument would be inversely correlated with social support. This hypothesis held, demonstrating a significant negative correlation between community TB perspectives and social support where p<0.05.
Reproducibility	The scales demonstrated moderate test re-test reliability for community TB perspectives (r=0.64 p<0.01) but did not demonstrate significant re-test reliability among patient TB perspectives (r=0.46 p<0.1). This may have been due to small sample size (n=15) or may indicate that level of stigma is static and changes with environmental input.
Responsiveness	During their validation, Van Rie did not state any hypothesis or test their instrument among groups “known” to have higher levels of TB stigma.
Floor and Ceiling Effects	There was no evidence of floor or ceiling effects. Patient and Community perceptions of TB stigma were normally distributed with fewer than 15% of respondents scoring either the highest or lowest levels of the instrument (mean= 27.9, SD = 7.5 and mean = 27.6, SD = 6.1 respectively).
Interpretability	Van Rie et al. did not include any metrics that would assess clinical relevancy like predictive validity to contextualize the instrument responses in clinical practice.

Discussion

While many clinical researchers are familiar with study design and manifest variables this review demonstrates that many others fail to scientifically justify latent variable measurement. Here, researchers do not apply the same rigor and precision to measurement of latent variables as they do calibration of research equipment or collection of biologic specimens. The widespread use of unvalidated questionnaires and proxy variables to measure TB related stigma seen in this review is concerning. Other researchers have taken notice of this trend; MacKenzie and colleagues also report prevalent use of inadequately validated instruments in their review of behavioral research measurement (MacKenzie et al., 2011). Rigorously validated instruments are a cornerstone of reproducible scientific research. Validation strengthens the quality and integrity of the findings and ensures that the indicators adequately capture the latent construct of interest. Given the prioritization by global health authorities to address TB stigma as an intervention to improve the care cascade, it is imperative that the scientific community address the findings of this review.

Some authors acknowledge a lack of detailed validation as a study weakness of their study. This leads us to ask why, if researchers recognize validation as an important component of rigorous scientific research, they bypass validation? Unvalidated instruments may incompletely and inaccurately represent TB stigma and may lead to biased findings. Conversely, validated measurement models are designed to minimize systematic error.

Although proxy variables are a fast and convenient alternative to long or detailed instruments, they invite systematic measurement error. Distilling a latent construct into a single item risks inadequately capturing all angles of an elusive or broadly defined construct. For example, TB status disclosure is an important indicator of perceived TB stigma but fails to capture the entire range or breadth of TB stigma. Additionally, proxy selection may reveal cultural bias where the selected variable represents the researcher’s cultural background, rather than the participant’s, inadvertently capturing only those participants who have experiences and perceptions similar to the research team. In this review, three articles used patient disclosure of stigmatizing experiences to broadly define stigma. Because stigma is culturally embedded, participants may fail to identify and recall experiences of stigma that have been normalized within their communities. In this example, both recall bias and cultural bias may systematically underestimate people who experience stigma.

There is growing acknowledgement and concern among stigma scholars about the impact of intersectional stigma on the mental health and infectious disease outcomes of people living with TB and HIV. None of the studies in this review mentioned the intersectional nature of stigma or the unique and often complicated experiences of people who lie at the intersection of multiple stigmatized identities. TB stigma and HIV stigma are inherently different constructs although they often affect the same people simultaneously. This was noted in the validation study by Soma and colleagues who saw that of their four international study sites, HIV stigma was closely entwined with TB stigma in Malawi specifically(Somma et al., 2008). This points to the variation the construct of TB stigma in diverse settings and also draws attention the intersectional nature of stigma.

Consideration for the distinct biomedical, religious and political aspects of social norms is central to stigma measurement. The most widely accepted TB stigma scales have not been culturally or linguistically validated in most African countries. Unlike physiologic or biological measurement, stigma is a social construct that varies depending up on the context; and resultingly, stigma varies broadly between communities and countries. One quarter of the world’s TB infections occur on the African continent (The World Health Organization, 2020). Nigeria and South Africa together represent seven percent of the world’s TB infections (The World Health Organization, 2020) but the instruments reviewed here are not culturally validated in either country. Overall, this review highlights the need for TB stigma instruments that are culturally and linguistically validated in high TB/HIV burden countries.

The steps for cross cultural measurement laid out by Beaton and colleagues are ideal for establishing linguistic sensitivity. However, this process only ensures that the items presented in the instrument are relevant and clear for the desired population. This process cannot determine whether the items fully or adequately represent all domains of the construct within a culture. For that reason, we argue that it is not sufficient to present a panel of technical experts (or stigmatized persons) with a preexisting instrument to establish cultural validity. Even brief qualitative exploration can help to identify important cultural concepts or experiences that are not an established part of a measure previously validated in another context. We are also hesitant to use local academics and clinicians alone to evaluate stigma instruments for content validity, as was done in some of the studies included in this review. Due to their education and institutional backing, clinician and academic experts may not use the same language and reference points as marginalized groups.

Due to word limits and publication conventions, many studies do not extensively explain their instruments or their nomological networks unless the instrument represents the variable or outcome of interest. Very few of the studies included in this review used TB stigma as the outcome of interest. As a result, we were unable to evaluate the studies in this review using consistent quality criteria and instead assessed only the original validation studies. In countries where instruments have not been culturally validated, publication of score distributions (including floor and ceiling effects), correlation metrics with locally established scales or instruments, and results that contribute to interpretability or responsiveness would help to evaluate the instrument’s utility and contextual suitability. For example, predictive validity or hypothesis testing of locally known groups would help to establish the clinical utility of the instrument and would demonstrate cultural validity.

Latent measurement models are carefully constructed to highlight variability and depth within a given construct. Using the mean stigma score as a cut point for a dichotomous variable is an effective way to maximize a smaller sample size and is adequate for formative research. However, dichotomizing stigma into a yes or no experience obscures the data, making it more difficult to identify and interpret meaningful associations and contributors to TB stigma. As a result, these dichotomous variables fail to capture the challenges and experiences of participants with lower levels of stigma. Without more granular data, researchers and clinicians may under-appreciate the impact that stigma has on access to care and treatment outcomes. Future studies should consider sample sizes large enough to power measurement of stigma as a continuous variable, to identify instrument thresholds, and to test the relationships between TB stigma instruments and TB care utilization and healthcare outcomes.

Previously, a lack of consensus about TB stigma measurement prevented appropriate measurement of criterion validity. This review shows overwhelmingly that the Van Rie TB stigma scales are the accepted gold standard for measuring TB stigma. The scales have been subject to theoretical and statistical validation techniques and are appropriate for use in people living with TB. Since Mbuthia and colleagues were able to meticulously interrogate cultural and linguistic validity of the Van Rie scales in Kenya, we are optimistic in future efforts to validate this instrument in other sub-Saharan countries. Future research efforts may utilize the Patient and Community Tuberculosis Perspectives to correlate criterion validity in future validation studies.

Strengths and Limitations of the Reviewed Studies

Within the articles reviewed here, the most significant of several study design limitations is a lack of appropriately validated instruments for measuring TB stigma. Although 28 studies met the inclusion criteria, psychometric synthesis was limited to only 13 studies that utilized validated instruments for a clinical or research setting.

Strengths and Limitations of this Review

This review excludes grey literature where field experts may witness and report on stigma measurement. Additionally, it is possible that we did not find every possible article on this topic due to variations in search terms or key words. Despite these limitations, this review also has several strengths. The review was based on a strategic and exhaustive search of peer reviewed literature assisted by a university informationist. All studies underwent independently verified critical appraisal to uphold scientific standards and study evaluations were guided by faculty with expertise in TB and HIV in South Africa.

Conclusion

This review offers a snapshot of recently used metrics to measure TB stigma. Using standardized criteria, we reviewed the instruments currently validated to measure TB stigma. Our appraisal emphasizes the need for reliable and valid instruments to measure TB and TB-HIV stigma in countries most affected by TB. All of the studies in this review, regardless of instrument, indicate that TB stigma is prevalent in global settings. Reduction of TB stigma is one important component of TB eradication efforts and emphasizes the need for further research into TB and intersectional stigma.

Table 4.

Appraisal of the Berger HIV Stigma Scale (Berger et al., 2001; Bunn et al., 2007)

Criteria	Evaluation
Content validity	Berger et al. tested content validity in the HIV Stigma scale using content experts to rate relevance and clarity. All of the 40 items were deemed relevant and clear. Because the Bunn study did not introduce new items to the scale there was no need to re-evaluate content validity.
Internal consistency	In the original validation study by Berger et al., the scales showed high internal reliability with Cronbach’s alpha 0.96 for the entire 40 item instrument. All subscales demonstrated reliability of 0.90 or greater. In the shortened instrument validation by Bunn et al., internal consistency remained high with Cronbach’s alpha above 0.90 despite the reduced number of items in the instrument.
Criterion validity	Neither Berger et al., or Bunn et al., assessed the criterion validity of the HIV Stigma Scale. In 2001 when Berger et al. validated their instrument, several other well-known HIV stigma scales such as the Kalichman’s AIDS Related Stigma Scale and Holzemer’s HIV/AIDS Stigma Instrument had not yet been created and validated.
Construct validity	In the validation of the shortened instrument, Bunn et al., tested construct validity comparing scores on the 23-item HIV Stigma Scale with the Stigma Consciousness Questionnaire, Discrimination scale, Fear of Discovery Scale and Rosenberg Self-Esteem Scale. As hypothesized, perceived stigma was negatively associated with self-esteem (−0.43), and correlated strongly with the Stigma Consciousness Questionnaire (0.62), Discrimination Scale (0.64), and Fear of Discovery Scale (0.71). All correlations were statistically significant at p<0.05 and were highly similar to the correlation coefficients in the original Berger et al. study.
Reproducibility	In the original validation study by Berger et al., a subset of respondents from the original sample (189/318) completed a second questionnaire via mail to assess test-retest reliability. The overall scale demonstrated test-retest correlation of 0.92 and all subscales revealed correlations >0.87 indicating excellent reproducibility. Reproducibility was not re-tested in the Bunn validation study.
Responsiveness	Neither validation study stated a hypothesis among groups “known” to have higher levels of HIV stigma.
Floor and Ceiling Effects	Neither validation study disclosed score distributions for the subscales or the total scales. Therefore, we are unable to evaluate floor and ceiling effects.
Interpretability	Neither Berger et al. or Bunn et al., included any metrics that would assess clinical relevance like predictive validity to contextualize the instrument responses in clinical practice.