Value of Cognitive Interviewing in the Development of the Weight Stigma in Healthcare Inventory

Abstract

Background:

Despite the detrimental effects of weight stigma in healthcare, there is no widely validated measure comprehensively examining such experiences.

Objective:

We aimed to develop and pilot test an inventory to measure patient experiences of weight stigma in healthcare, and to ensure our items were easily understood.

Patient Involvement:

During our iterative design process, patients assessed whether our inventory items were easy to understand and we included an open-ended comments question.

Methods:

We compiled items from pre-existing tools assessing experiences of weight stigma in healthcare, and developed our own novel items. We conducted field pre-testing with a convenience sample of 48 patients at a Midwest academic internal medicine clinic. We utilized an iterative design process whereby respondents provided feedback on our inventory, we analyzed the data and made revisions, and then repeated the cycle.

Results:

Respondents found some of the language in our items confusing; expressed reluctance to speculate on the motivations of healthcare providers; had difficulty with “double-barreled” questions; found some questions vague; and expressed the desire to have weight addressed in clinical encounters neither too much nor too infrequently. We altered items appropriately, and in subsequent rounds of data collection they were easier to understand.

Discussion:

Patients found many common weight stigma survey items and some of our novel items confusing. Our modified inventory reduces patient confusion and enhances data quality.

Practical Value:

Our study demonstrates the value of cognitive interviewing. Furthermore, the WSHCI will be a useful tool for clinicians and research teams seeking to measure weight stigma in healthcare but first needs to be validated in a larger sample.

1. Introduction

Weight stigma is a societal devaluation of people because of excess body weight, which can lead to weight-based stereotypes, prejudice, and/or overt forms of discrimination.¹ Individuals with obesity encounter weight stigma in many contexts, including healthcare settings.² Weight stigma has increased over time, with a prevalence of 12% in 2004–2006 within a large U.S. national sample.³

In one study, it was found physicians spend less time with patients with obesity and view them as “awkward, unattractive, ugly and non-compliant.”⁴ In another study, of more than 20 possible sources of weight stigma, patients ranked physicians as the second most common source.⁵ Anywhere from 40–70% of patients seeking treatment for obesity experience weight stigma in healthcare.^5–8 Bias against patients with obesity contributes to avoidance of medical professionals, delays in care,⁴ depression and psychological distress.^9,10 Weight stigma is associated with worse weight loss outcomes,¹¹ binge eating,^12,13 and reduced physical activity.¹⁴

Many studies examine attitudes among healthcare providers, but fewer studies have examined patients’ experiences of weight stigma in healthcare.^15,16 The Stigmatizing Situations Inventory (SSI) was developed in 1999 to evaluate weight stigma in various contexts, and includes six questions about weight stigma in healthcare.¹⁷ This tool was developed through a series of focus groups involving men and women, patients and non-patients. Several researchers in healthcare settings have used slightly modified versions of these questions.

The SSI and other weight stigma questionnaires have undergone limited field testing especially among men.^16,18 Ferrante’s Stigma Situations in Healthcare instrument (SSHC) was adapted from the SSI and was used in a population of women with obesity seeking primary care treatment.¹⁸ Raves developed the Healthcare Weight-Related Stigma (HWCS) tool by adapting questions from the SSI and the Interpersonal Sources of Weight Stigma tool developed by Puhl and Brownell.^5,19 The HWCS was used in a population of bariatric surgery patients (77% women). Wadden developed the Health Care Questionnaire, which included 10 items assessing weight stigma in healthcare, and sampled 259 women seeking treatment for weight loss.⁷ These tools, while useful, to our team’s knowledge did not undergo validation through cognitive interviewing. Furthermore, they did not include sufficient questions related to physical examination, medical equipment, and the built medical environment. These areas of inquiry are vital to better assess which aspects of clinical practice are most problematic.

We aimed to develop a tool to measure weight stigma in healthcare, the Weight Stigma in Healthcare Inventory (WSHCI). The purposes of this study are to determine whether the WSHCI items we developed are easily understood as intended by patients, and whether changes are needed to enhance comprehension and ease of use. We selected the use of cognitive interviewing techniques based on survey methodology literature, indicating that survey questions are often not understood as they were originally intended.²⁰ The focus of this paper is on the value of cognitive interviewing, and how this method can enhance survey design and data quality.

2. Methods

This study was approved by the University of Iowa Institutional Review Board.

2.1. Measures

Measures included the WSHCI, demographics, subjective health status, and an item on preferred weight terminology. We also included seven items we crafted on healthcare avoidance, as this may be a key mechanism underlying worse health outcomes for those experiencing weight stigma.²¹

2.2. Procedures

We compiled items from several pre-existing tools assessing experiences of weight stigma in healthcare, including a modified version of the SSI, the Interpersonal Sources of Weight Stigma (ISWS), and the 10 weight stigma items in the Health Care Questionnaire.^5,7,22 We selected questions from these measures through a process of expert review. We modified the items to refer to our healthcare system and used the term ‘healthcare providers’ to include non-physician staff. Eleven questions were modified from the original source to ask about positive rather than negative experiences to identify respondents who always give the same answer (straight-lining) which may indicate poor quality responses.²⁰ We also created 13 novel items for areas which were not adequately addressed in previous tools: one item about general experiences of weight stigma and twelve items about the physical exam, medical equipment and the clinical built environment. The final inventory contained 24 items divided into four sections on general weight stigma experiences, clinical communication, physical examination, and the physical clinical environment (Supplement 1).

We utilized an iterative design process whereby respondents provided feedback on our inventory, we analyzed the data and made revisions, and then repeated the cycle. Cognitive interviews were conducted using the methods described by Fowler and Fink et al.^23,24 We conducted field pre-testing with a paper version of our inventory with a convenience sample of 48 patients aged ≥18 years and a BMI ≥ 30 attending a general internal medicine clinic in a Midwest academic healthcare system. Patients completed demographic items, think-aloud, and probing procedures with a subset of 10–11 questions (approximately one-third of the entire inventory) to assess for question clarity (Supplement 1).²⁰ The number of questions was limited to minimize respondent time burden. Patients were consented for participation after a primary care visit and escorted to a private room for the interview. Participant responses were recorded and then transcribed by KMR.

After completing 12 interviews per question subset (36 interviews total), the interviews were inductively coded by KMR for understanding of the questions and common themes.²⁵ Data and codes were discussed until consensus was reached (KMR, HHL, AMS). The inventory was revised based on the qualitative coding analysis. An additional 4 patients per question subset (12 interviews total) completed the same cognitive interview process as before with the revised items. We subsequently made additional minor revisions to our inventory.

2.3. Analytic Strategy

The first author read all the transcripts to get a sense of the data. Item-level summaries were created based on aggregated data across interviews. The summaries documented the frequency of problems experienced by patients, and the nature of the problem (e.g. comprehension, memory retrieval). Themes appearing throughout the interviews were inductively coded and reviewed as a research team until agreement was reached. Following each round of interviews, the data was reviewed by our multidisciplinary team, and items were revised for subsequent testing.

3. Results

Our sample was predominantly non-Hispanic White (95.8%), female (68.8%) and with mean BMI 36.9 (SD 7.0 kg/m²). Most respondents had a 4-year college degree or higher education (62.5%) and the most common income category was $25,000 - $74,999 per year (Table 1).

Table 1.

Patient characteristics (N = 48).

Characteristic	Mean (SD) or N (%)
Age (years)	63.7 (SD 13.1)
Race/Ethnicity
White, non-Hispanic	46 (95.8%)
Black, non-Hispanic	1 (2.1%)
Hispanic	0 (0%)
Other	1 (2.1%)
Gender
Female	33 (68.8%)
Male	15 (31.2%)
BMI (kg/m^2)	36.9 (SD 7.0)
Education
<High school	2 (4.2%)
High school degree	6 (12.5%)
Some college	4 (8.3%)
2-year college degree or trade school	6 (12.5%)
4-year college degree	18 (37.5%)
Graduate work	12 (25.0%)
Income
< $25,000	14 (10.3%)
$25,000- $$74,999	60 (44.1%)
$75,000- $124,999	38 (27.9%)
$125,000- $174,999	18 (13.2%)
≥$175,000	6 (4.4%)
Rural/Urban Status (per RUCA code)
Urban	29 (67.4%)
Suburban	6 (14.0%)
Rural	8 (18.6%)
Household Size
1 member	17 (36.2%)
2 members	19 (40.4%)
3 members	5 (10.6%)
4 members	2 (4.3%)
5 members	2 (4.3%)
6 members	0 (0%)
7 members	1 (2.1%)
8 members	1 (2.1%)
Self-Reported Health Status
Poor	2 (4.2%)
Fair	10 (20.8%)
Good	24 (50%)
Very good	10 (20.8%)
Excellent	2 (4.2%)
Weight Loss Surgery
No	41 (85.4%)
Yes	3 (6.3%)
Considering	4 (8.3%)

Representative quotes from our cognitive interviews are given in Table 2, along with the original and revised questions. The cognitive interviews revealed that participants did not understand many of the questions as originally written.

Table 2.

Selected examples of problematic questions, with comments and revisions.

Question	Understood	Comments	Revised Question(s)	Understood	Explanation for the Modification
Q3: I felt that UIHC health care providers didn’t treat overweight people as nicely as they do people with lower weight.	100%	How do they know how they treat other people? That happens in an examining room. How do you know much about either fat or thin people are getting treated?	Q3: Health care providers were less nice to me than other people because of my weight. Q5: I observed UIHC employees treating overweight patients rudely by staring, using negative body language, or making negative comments about their weight.	100% (both)	Draws more on direct experiences. Respondents understood question but had difficulty answering.
Q5: I was treated as a second-class citizen at UIHC because of my weight.	86%	Down here where it says treated as a ‘second class citizen,’ I don’t really know what a second-class citizen is.	Q4: I was treated as a lesser person due to being overweight.	92%	Removed confusing “second-class” terminology.
Q6: UIHC health care providers recommended a diet even if I did not come in to discuss weight loss.	67%	What I heard was that providers cared about my health…scolded. They’re always looking out for your best interest…I don’t think that’s a bad thing.	Q7: Health care providers talked to me about weight loss diets too much. Q8: Health care providers avoided the topic of weight loss diets even though I wanted the topic addressed.	100% (both)	Pair of questions now addresses talking about weight too much and avoiding the topic.
Q7: A health care provider at UIHC blamed unrelated physical problems on my weight.	71%	They’re trying to tell me my hip problem is related to my weight…but if I don’t want to hear it, I’m going to rationalize that they’re blaming unrelated things on my weight. The physician probably knows about more relationships than the patient. So the patient may call it unrelated and the physician’s observation is probably true.	Q9: Health care providers blamed my health problems on my weight more than they should have.	82%	Does not ask respondent to assess whether a health problem is related to weight.
Q10: UIHC health care providers have said critical or insulting things to me about my weight.	67%	[Meaning of “critical”?] That it was more urgent. Insulting means you go at it repeatedly…Critical [means] critique.	Q12 Health care providers said insensitive things to me about my weight.	100%	No longer double-barrel, removes “critical” and “insulting.”
Q15: Health care providers at UIHC provided appropriate and respectful physical examinations.	83%	I would be under the assumption that would be more about sexual harassment or sexual assault.	Q18: Health care providers performed respectful physical exams.	NT	No longer double-barrel, removes “appropriate” which carried a sexual harassment connotation.
Q23: Signs and public messaging at UIHC could be offensive to people of my weight.	50%	I didn’t know what it would be referring to. I haven’t ever had any public messaging. To me that could be social media…That’s confusing to me…	Q25: UIHC’s public signs, website, advertising, and announcements addressed weight in a way that was offensive.	NT	Removed “public messaging” and made more specific.

NT: not tested

Respondents found some of the language in our items confusing. For example, one item asked about being treated as a “second-class citizen,” but respondents were not familiar with this terminology. Another of our questions, derived from the Health Care Questionnaire, asked about “critical” comments. Respondents interpreted the word “critical” as referring to urgency or “critique,” whereas we intended to ask about criticism which carries a stronger negative connotation.

Participants frequently expressed reluctance to speculate on healthcare providers’ beliefs, attitudes, and treatment of patients with obesity. For example, in the ‘Overall Experiences’ section, we originally asked about whether healthcare providers treated overweight people as nicely as others. However, several respondents stated they had not directly observed healthcare providers interacting with other patients so they could not answer this question. Respondents also indicated it can be difficult to know the motivations of healthcare providers, e.g., even if one is treated badly, this could be due to many factors.

Participants and members of our research team realized that several of our questions were “double-barreled” questions. These are questions in which two questions are actually bundled together, making it difficult to answer. For example, we originally asked about “appropriate and respectful” physical examinations. Respondents might feel differently about whether what was done during exams was appropriate vs. whether the exam was done in a respectful manner.

The medical authority of physicians was another theme. For example, a question asked about healthcare providers blaming unrelated physical problems on weight. Several respondents indicated that healthcare providers have more medical knowledge than patients and so a patient might be incorrect in thinking an unrelated problem was blamed on weight.

Respondents also expressed concerns that they might not recognize weight as a health problem, while physicians might appropriately identify weight management as an important topic to address. In the communication section, we asked whether a healthcare provider recommended a weight-loss diet even though that was not the purpose of the visit. Although this has been identified as a stigmatizing behavior, several respondents interpreted this as a caring behavior through which the healthcare provider showed concern for the patient’s well-being.

Respondents found our questions related to transportation, signage, and public messaging too vague and requested greater specificity. Respondents also wanted a “not applicable” option for several of our items; preferred a frequency scale (rather than agree/disagree); and preferred a question matrix format (data not shown).

4. Discussion and Conclusion

4.1. Discussion

Despite the detrimental effects of weight stigma in healthcare, there is no widely validated scale that comprehensively examines such negative experiences. Moreover, many of the existing tools examining weight stigma in healthcare have not undergone validation with cognitive interviews. Our use of cognitive interviewing showed that many of the items on these scales are in fact problematic. We used the interviewees’ responses to improve upon these measures. Additionally, the existing scales on which we based our questionnaire did not fully address weight stigma related to the physical examination, medical equipment, or the clinical environment. We added items related to these areas and tested them using cognitive interviewing. Together, these questions can be used to more comprehensively measure patients’ experiences of weight stigma in healthcare.

The cognitive interviewing process identified some problematic questions from existing surveys. Several items from previously developed tools asked respondents to speculate on how others were treated or on the motivations of physicians. As our respondents and others have found, weight stigma can be an ambiguous experience and the motivations of others are not always clear.²⁶ We reduced this ambiguity as best possible by modifying items to only ask about behaviors patients personally observed or experienced.

The SSI and the Health Care Questionnaire both ask about unsolicited dieting advice, but several of our respondents interpreted this as a caring behavior. However, we intended to ask about unwanted dieting advice. Given these differing interpretations, we added a version of this question that clearly carried a negative connotation. We also added an item asking about healthcare providers avoiding the topic of weight loss, as some respondents construed omitting the topic as a lack of caring. We treated these two questions as a pair, with one question referring to too much weight-related communication and one question referring to avoidance of the topic. We scored this pair of questions as one by taking the higher of the two scores, corresponding to more weight stigma. As Nagpal notes in the context of pregnancy, many women reported that they wanted their weight addressed but found that healthcare providers seemed too uncomfortable to broach the topic.²⁷ Just like excessive emphasis on weight, this unwillingness to address weight as a health concern was experienced as a form of stigma. Previous surveys have not addressed the topic of avoidance of weight-related discussions as a form of stigma.^17,18

The SSI asks about doctors blaming “unrelated physical problems” on weight, but our respondents did not feel qualified to make this medical judgement. We changed the question and instead asked about whether health problems were blamed on weight “more than they should have” been. This framing still requires an element of interpretation, but it allows for a common scenario, that of over-attribution of weight as a contributing factor. In these instances, a patient believes that healthcare providers do not adequately consider other possible contributory factors in disease and instead disproportionately blame health problems on weight.¹⁷

Those interviewed also identified double-barreled and questions with multiple interpretations. For example, the Health Care Questionnaire asks about doctors saying “critical or insulting things” about weight. This question was a double-barreled question and respondents did not interpret the word “critical” as we had intended. We changed the question to refer to providers saying “insensitive things” about weight. Thus, the question only asked about one aspect of communication and used words that were clearer to the respondent.

Our inventory added several novel items to explore under-addressed areas of weight stigma in healthcare, especially with regard to the physical examination and the clinical environment. For example, we asked about feeling pressure to be weighed and added a question about being weighed in a manner that respected privacy. We also asked questions about whether physical examinations were appropriate and respectful, topics not addressed in previous surveys. Like other questionnaires, we asked about furniture and medical equipment. We also asked about the accessibility of buildings, provision of assistance with transportation (such as wheelchairs), and the appropriateness of advertising and websites. These are crucial aspects of the healthcare experience not previously addressed.^7,16–18

Cognitive interviewing also identified the need for changes in our new questions. For some of our novel items, respondents wanted greater specificity. We included details and replaced the phrase “public messaging” with “public signs, website, advertising, and announcements.” In one of our novel items, respondents found the phrasing “discriminated against” too harsh, as the term can sometimes refer to extraordinarily or unbelievably bad treatment.²² In response, we removed the word “discriminated.” However, we still wanted to capture instances of very severe discrimination so we added an item about being “treated extremely badly” due to weight. Thus our new questions better reflected our intended meaning. The WSHCI after modifications based on cognitive interviewing is shown in Supplement 2.

While cognitive interviewing is commonly used in survey development,²⁸ to our knowledge this is the first report on its use to develop a tool to evaluate experiences of weight stigma. Strengths of our study include the participation of men and inclusion of individuals with a wide range of BMIs. Our study uncovered some key limitations in the commonly-used SSI and Health Care Questionnaire.^7,17 Additionally, we included more questions on physical examination and the clinical environment, and an outcomes measure assessing healthcare avoidance. These differences originate from our group’s intention to use the WSHCI to target specific aspects of healthcare for reduction of weight stigma.

Limitations of our study include the use of a convenience sample for cognitive interviewing, which predominantly consisted of well-educated, non-Hispanic White participants from the Midwest. We intend to further validate our inventory in a more diverse sample.

4.2. Conclusions

The process of cognitive interviewing was tremendously valuable in the development of the WSHCI. It allowed us to identify and modify confusing questions, and to ensure that respondents understood our questions. We anticipate this will reduce misinterpretation of the items by respondents, improving data quality. Furthermore, through the process we found additional topics to include on the inventory to improve its scope. We anticipate that the WSHCI will prove useful in identifying weight stigma in healthcare. We will also use the tool to correlate wight stigma in healthcare with one of its most pernicious effects: healthcare avoidance. We plan to further validate our inventory in a larger, diverse sample.

4.3. Practice Implications

We will use this tool to establish the prevalence of weight stigma in healthcare within our institution and others, to establish the domains in which weight stigma occurs, and to develop and evaluate targeted interventions to reduce weight stigma in healthcare. Researchers should preferentially seek out instruments validated through cognitive interviewing, and strongly consider the use of cognitive interviewing when developing their own research tools.