Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2022 Apr 1.
Published in final edited form as: Int J Eat Disord. 2021 Feb 27;54(4):660–667. doi: 10.1002/eat.23479

Registered report: Initial development and validation of the eating disorders safety behavior scale

Erin E Reilly 1, Brittany Bohrer 2, Daniel Sullivan 1, Jamal H Essayil 3, Nicholas R Farrell 4, Tiffany A Brown 2, Sasha Gorrell 5, Lisa M Anderson 6, Marita Cooper 7, Colleen C Schreyer 7, Olenka Olesnycky 1, Olivia Peros 1, Katherine Schaumberg 8
PMCID: PMC8044048  NIHMSID: NIHMS1678477  PMID: 33638564

Abstract

Anxiety and eating disorders (EDs) often co-occur, prompting calls to explore anxiety-related maintenance processes in ED samples. Safety behaviors, which function to prevent a feared outcome from occurring or to reduce anxiety associated with a feared stimulus, are observed across anxiety disorders and, along with overt avoidance behaviors, are an important target in treatment. Data suggest that individuals with EDs also engage in safety behaviors. However, no existing assessments provide a comprehensive measure of eating-disorder-specific overt avoidance and safety behaviors. The goal of this Stage 1 Registered Report is to develop a comprehensive self-report measure of ED-specific safety behaviors. In Study 1, we will recruit 50 women with EDs to complete the scale and provide feedback on the response scale. Feedback from these participants will be used to refine the measure. In Study 2, we will evaluate the psychometric properties of the measure in a large sample of women with EDs (n dependent on the size of measurement) and a community sample without current or a history of ED symptoms. We will explore the measure factor structure, known-groups validity by comparing scores from women with EDs to healthy controls, internal consistency, and convergent and divergent validity with other psychological instruments.

Keywords: anxiety, eating disorders, measurement development, safety behaviors

1 |. INTRODUCTION

Existing theoretical models have suggested the possibility of shared mechanisms across anxiety and eating pathology (Pallister & Waller, 2008; Schaumberg et al., 2021). Safety behaviors, or behaviors that function to prevent a feared outcome or to manage anxiety (Salkovskis, 1991), are commonly thought to maintain anxiety-based symptoms over time alongside overt avoidance behaviors (Helbig-Lang & Petermann, 2010; Wells et al., 1995). Research also suggests that individuals with eating disorders (EDs) may engage in frequent safety behaviors (Pallister & Waller, 2008; Waller & Marcoulides, 2013), although this work remains limited. In the current set of studies, we aim to develop a comprehensive self-report measure of overt avoidance and safety behaviors in clinical samples of patients with EDs to facilitate the larger goals of promoting research on this topic to improve clinical care.

As noted earlier, clinical and empirical consensus supports significant overlap in the presenting symptoms and mechanisms of EDs and anxiety. First, up to 70% of individuals with EDs report meeting criteria for an anxiety disorder (Hudson, Hiripi, Pope, & Kessler, 2007; Swanson, Crow, Grange, Swendsen, & Merikangas, 2011). Second, core ED symptoms (e.g., fear of fatness; restrictive eating) share characteristics with anxiety disorders, such as avoidance, worry/repetitive thought, and behavioral inhibition (Pallister & Waller, 2008). Finally, while mechanistic research in EDs remains limited, reviews have highlighted the potential for shared maintenance processes across both EDs and anxiety, such as avoidance behaviors and problems with learning (Pallister & Waller, 2008; Schaumberg et al., 2021). There have been calls to revisit the use of anxiety-based treatments for EDs (Butler & Heimberg, 2020; Murray et al., 2016; Reilly, Anderson, Gorrell, Schaumberg, & Anderson, 2017). However, research supporting shared mechanisms of EDs and anxiety remains limited.

Safety behaviors represent a core feature of anxiety disorders and are hypothesized to maintain anxiety symptoms (Helbig-Lang & Petermann, 2010). Safety behaviors are posited to interfere with extinction learning in treatment (Blakey & Abramowitz, 2016; Salkovskis, 1991), and therefore represent promising treatment targets. Within the anxiety disorders literature, these forms of avoidance can be more overt in nature or can be subtle and occur within anxiety provoking situations (Salkovskis, 1991; Wells et al., 1995). Theoretical work suggests that avoidance-based ED symptoms may function as safety behaviors (Pallister & Waller, 2008). However, while anecdotal evidence suggests that some ED symptoms function as safety behaviors (i.e., serving to prevent feared consequences from occurring or to regulate eating-related anxiety), no existing measures exist to assess the range of possible safety behaviors in EDs.

One challenge to assessing safety behaviors transdiagnostically arises from the fact that specific safety behaviors can vary widely within and across specific disorders. However, the function of safety behaviors is consistent transdiagnostically, as they serve to prevent or mitigate feared outcomes. The identification and assessment of safety behavior engagement is vital to treatment planning and intervention efficacy, as safety behaviors maintain anxiety through preventing extinction learning (Blakey & Abramowitz, 2016; Salkovskis, 1991). If an individual approaches a feared stimulus, engages in a safety behavior, and their feared outcome does not occur, they may attribute the disconfirmation to the safety behavior, rather than learning that their expectancy was violated (Wells et al., 1995). Furthermore, using safety behaviors to manage anxiety represents affective avoidance that is likely negatively reinforced, promoting further avoidance of negative feelings over time (Wells et al., 1995).

Pallister and Waller (2008) highlight safety behaviors as one of the key shared mechanisms between eating and anxiety disorders. A range of ED behaviors (e.g., compulsive self-weighing, body checking, calorie counting), as well as cognitions (e.g., engaging in distraction, repeating mantras, making mental comparisons), may serve as safety behaviors. Specifically, these behaviors may function to prevent feared consequences (e.g., gaining weight, negative self-evaluation due to weight) or help with managing eating or weight-related anxiety. Importantly, while some of these behaviors—such as dietary restriction—are assessed and addressed unanimously across existing ED treatments (Fairburn, 2008; Lock et al., 2010), others may not be comprehensively assessed in common clinical practice. Even when assessed, safety behaviors may go unaddressed in treatment for various reasons, including specific patient and clinician factors (Becker, Farrell, & Waller, 2019; Waller, 2009; Waller, Stringer, & Meyer, 2012). For example, given data suggesting that clinician self-reported anxiety predicts lower adherence to cognitive-behavioral therapy (CBT) for EDs (Mulkens, de Vos, de Graaff, & Waller, 2018; Turner, Tatham, Lant, Mountford, & Waller, 2014; Waller et al., 2012), it may be the case that elimination of safety behaviors (a core component of CBT) may be less likely among those individuals with elevated anxiety. Furthermore, other treatment-related factors, such as the stage of treatment, severity of illness, and current focus of treatment, may influence the extent to and order in which safety behaviors are addressed. For example, some researchers in anxiety disorders have argued that more gradual focus on decreasing safety behaviors in treatment may decrease treatment non-adherence and dropout (Blakey & Abramowitz, 2016). Altogether, accurately assessing safety behaviors in treatment for EDs could clarify barriers to treatment efficacy and suggest targets for improving outcomes.

Effectively characterizing and treating avoidance and safety behaviors hinges on the availability of valid and reliable measurements. There is currently only one self-report assessment that explicitly evaluates ED-specific safety behaviors. Specifically, Waller and Marcoulides (2013) published a psychometric investigation of the Brief Safety Behaviors Scale (BSBS), which gauges a number of transdiagnostic safety behaviors, in an ED sample. Although data tentatively supported the measure's reliability and validity, the measure predominantly captures safety behaviors observed across anxiety disorders, with few items addressing ED-specific safety behaviors (Waller & Marcoulides, 2013). Although several other measurements, including the Yale–Brown–Cornell Eating Disorders Scale (Sunday, Halmi, & Einhorn, 1995), assess behaviors and cognitions that can be conceptualized as safety behaviors, none of these measures specifically focus on ED-related safety behaviors.

The overarching goal of the current set of studies is to develop, and provide initial validation for, an ED-specific overt avoidance and safety behaviors measure. Study 1 aims to receive feedback on an initial item pool from a sample of women with current EDs. Study 2 aims to explore the initial factor structure, reliability, and convergent and divergent validity of the questionnaire developed in Study 1 in a clinical sample of women with EDs and compare the properties of the measure between patients with EDs and healthy control (HC) women.

2 |. METHODS

2.1 |. Generation of initial survey items

A preliminary list of safety behaviors was generated by sourcing clinical insight from nine professionals who have expertise in treating EDs. All clinicians were asked to compile a list of safety behaviors observed in their work with ED patients; lists were generated independently and submitted anonymously. The lead author compiled items and eliminated duplicates. The initial list contains 108 safety behaviors (see Supplement).

2.2 |. Study 1: Pilot study

2.2.1 |. Participants

Women ≥18 years old who self-identify as currently having an ED, including anorexia nervosa (AN), bulimia nervosa (BN), binge-eating disorder (BED), and other specified feeding or eating disorder (OSFED), will be eligible to participate in the study. We have chosen to limit our recruitment to only women due to the fact that the majority of existing literature informing our study was conducted in females; furthermore, given significant work suggesting variability in ED presentation across male and females and low representation of males in treatment-seeking samples (Murray et al., 2017), we would not be sufficiently powered to explore gender invariance in the measurement within the scope of the current manuscript. We will confirm an ED diagnosis using the Eating Disorder Diagnostic Scale (EDDS; see measurements), operationalizing a probable ED diagnosis as scores ≥16.5 on the EDDS. Participants will be recruited through a large academic medical center in the United States and via online forums (see Procedures). There will be no exclusion criteria related to diagnostic comorbidities in this portion of the study.

2.2.2 |. Measures

Feedback on items

Participants will be asked to rate the frequency of engaging in the 108 candidate items, when in an anxiety-provoking situation either to prevent a feared outcome or cope with anxiety, using the following scale: 0 = never, 1 = rarely, 2 = sometimes, 3 = usually, and 4 = always. Prior to scale completion, we will provide patients with a definition of safety behaviors. Because of the length of the scale and to combat the effects of fatigue, we will present items in a randomized order across participants.

Additional feedback items

We will also collect responses to questions that gauge the need to add items and the general relevance of the measure to patients, including “Are there any safety behaviors that you engage in related to your shape, weight, or eating that we did not include?” and “Please provide any thoughts that you have regarding the overall relevance of the items to you.”

The next series of questions will focus on feedback on the response scale. We initially selected the response scale based on its use in similar assessments of safety behaviors (e.g., Social Behaviour Questionnaire; Clark et al., 1995). In efforts to explore whether a specific count/frequency-based measurement may be more appropriate for assessing the construct of interest, we will ask participants to rate the extent to which they found the response scale easy to understand using a Likert-type scale ranging from 1 (Strongly Disagree) to 5 (Strongly Agree), and easy to apply to their own behaviors using a Likert-type scale ranging from 1 (Strongly Disagree) to 5 (Strongly Agree). We will also include open-ended responses for participants to provide additional open-ended feedback. Participants will then be asked questions regarding the frequency that they engage in specific safety behaviors, and their ability to recall the number of behaviors engaged in, across differing timeframes (e.g., frequency in past 24 hours vs. 7 days).

Fidelity checks

We will include two fidelity items (e.g., “Please select always.”) in the first two sets of questions in the survey. Participants who do not pass either check will be immediately redirected to the end of the survey.

Eating Disorder Diagnostic Scale ( Stice, Telch, & Rizvi, 2000)

The EDDS is a 22-item self-report questionnaire that assesses ED pathology over the past 3 months. The psychometric properties (Krabbenborg et al., 2012; Stice et al., 2000; Stice, Fisher, & Martinez, 2004) indicate adequate test–retest reliability, agreement with a structured diagnostic interview for psychopathology, internal consistency, and convergent validity across a range of samples (Krabbenborg et al., 2012; Stice et al., 2004). Past work has indicated that a score ≥ 16.5 reliably distinguished individuals with EDs from control participants (Krabbenborg et al., 2012), and the measure more generally has demonstrated strong agreement with the Eating Disorder Examination (Fairburn, Cooper, & O'Connor, 2008; Stice et al., 2000, 2004).

Depression, Anxiety, and Stress Scale-21 (Lovibond & Lovibond, 1995)

The Depression, Anxiety, and Stress Scale-21 (DASS-21) is a 21-item version of the original DASS-42, designed to measure and discriminate anxious and depressive psychopathology over the past 7 days. The DASS-21 evidences strong internal consistency and concurrent validity (Henry & Crawford, 2005). In Study 1, we will use the DASS-21 to characterize the current level of anxious and depressive symptoms in our sample.

Demographics

Participants will report on their self-identified gender, race and ethnicity, age, current ED diagnosis (and who made this diagnosis), age of ED onset, current and past treatment received (if applicable), current weight and height, and lowest and highest past weights since puberty.

2.2.3 |. Procedures

All methods, procedures, and statistical analyses are preregistered at https://osf.io/tukah/. Participants for the study will be recruited through two methods: (a) we will post information about the study on ED-specific forums (e.g., Reddit) and ED organization Listservs (e.g., Academy for Eating Disorders); (b) we will recruit participants currently enrolled in the academic medical center ED program of the fourth author (JE). Study information and recruitment materials will be consistent across both methods, including a standardized flyer and/or consent form with information about the study. In addition, we will provide a script for providers and research staff at the academic medical center to use when presenting information about the study to patients. The data supervisor for the academic medical center site (JE) will provide this information and script to providers and research staff at the clinic, who can disseminate to patients outside of session time. Because we plan to recruit a heterogeneous sample of women with EDs, we do not plan to restrict the recruitment of clinic patients to any specific time frame or phase of treatment (e.g., within the first month of treatment) or level of care (e.g., outpatient versus partial hospitalization). All recruitment materials will be approved and disseminated as instructed by our institutional review board. Subjects who participate in the study will be entered to win a $50 gift card.

2.2.4 |. Statistical analysis plan

Sample size determination

We plan to recruit 50 participants. This choice was made attempting to adopt the most conservative estimates in the literature for appropriate sample sizes for pilot studies in measurement development while accounting for missing and/or incomplete data. Specifically, existing recommendations in the literature range from ~24 to 40 participants for a pilot study in measure development (Hertzog, 2008; Johanson & Brooks, 2010).

Data screening

Prior to analysis, we will discard the data of participants who take less than 5 min to complete the survey and those who do not pass either fidelity check.

Elimination of irrelevant items and addition of items

For eliminating and adding items, we will take a hybrid quantitative and theoretical approach. We will flag items that are under-endorsed (mean frequency score < 2.0) and decide to eliminate these items based on consensus of authors (≥75% consensus that the item should be eliminated regarding the theoretical and clinical relevance of each item. We will also consider eliminating items with either high or low inter-item correlations (0.8 < r < 0.3) (Clark & Watson, 1995; Netemeyer, Bearden, & Sharma, 2003). For adding items, we will code the frequency of specific responses to the open-ended questions probing safety behaviors that were not included in the list. Author consensus (≥75%) regarding the relevance of each item will be used to select additional items.

Consideration of feedback on response scale

Based on participant feedback, we will consider the appropriateness of the response scale of the measurement. We will evaluate mean responses on the item gauging interpretability of the scale, aiming for a mean score between 3.5 and 5.0, indicating an average rating in the Agree/Strongly Agree range that outcome scale is easy to complete. In the event that the mean interpretability scale of the response scale item is <3.5, we will consider feedback provided in open-ended questions wherein participants indicate what is confusing about the scale, and/or consider shifting to an adapted response scale wherein participants report on the frequency of behaviors.

2.3 |. Study 2: Initial validation study

The goals of Study 2 are to (a) explore the initial factor structure, internal consistency, and convergent and divergent validity of the measure in a clinical sample, (b) evaluate initial associations between scores on the measure and ED symptoms, and (c) compare scores on the measure between patients with EDs to a sample of women without history of or current EDs. Following collection of pilot data, we will complete the analytic plan and alter the measure as outlined above. Next, we will proceed to initial data collection for the revised measure.

2.3.1 |. Participants

Participants will be ≥18 years old: (a) women self-identifying as currently having an ED, or (b) women identifying as having no current or history of an ED (healthy controls; HCs). As in Study 1, women in the group with a current ED must endorse an EDDS score ≥ 16.5. All participants are required to identify as women. As noted earlier, we have chosen to limit our recruitment to women for this initial study based on the samples in past work that informed our study rationale, as well as concerns related to lack of power to explore gender invariance in the assessment.

2.3.2 |. HC exclusion

To provide a homogeneous comparison to our ED sample, we will exclude HC women with a history of an ED. We will state this exclusion criteria at the outset of the survey and ask participants to complete the EDDS as the first measurement in the study; those who score < 16.5 will be directed to a question of whether they have ever had an ED and/or clinically significant ED symptoms. HCs who score e ≥ 16.5 on the EDDS or endorse a history of an ED will be directed to the end of the survey.

2.3.3 |. Measures

Demographics

Demographics will be collected in the same manner as in Study 1. Subjects will self-report gender, race and ethnicity, age, current ED diagnosis (and who made this diagnosis, if applicable), age of ED onset, current and past treatment (if applicable), current weight and height, and lowest and highest past weights since puberty.

Eating Disorders Safety Behaviors Scale

Participants will complete the revised Eating Disorders Safety Behaviors Scale.

Eating Disorder Diagnostic Scale (Stice et al., 2000)

The EDDS will be used as both a screener and a continuous measure of ED symptomatology over the past 3 months. As noted earlier, the measure has strongly documented psychometric properties and has demonstrated both strong negative and positive predictive values with well-validated structured interviews of ED pathology (Krabbenborg et al., 2012; Stice et al., 2000, 2004).

Depression, Anxiety, and Stress Scale-21 (Lovibond & Lovibond, 1995)

Described earlier, the DASS-21 will be used as a continuous measure of depression and anxiety symptoms. The DASS-21 provides a measurement of symptoms over the past 7 days and possesses good reliability and validity (Henry & Crawford, 2005).

Brief Safety Behaviors Scale (Waller & Marcoulides, 2013)

The Brief Safety Behaviors Scale (BSBS) is a 30-item assessment of safety behaviors focusing primarily on generalized safety behaviors observed in anxiety disorders and obsessive–compulsive disorder (e.g., self-monitoring heartbeat; checking that the gas is turned off before leaving the home). Waller and Marcoulides (2013) have previously validated the BSBS in an ED sample; we will use this measure to gauge convergent validity.

Anxiety Sensitivity Index-3 (Taylor et al., 2007)

The Anxiety Sensitivity Index (ASI-3) is an 18-item measurement of anxiety sensitivity (i.e., tendency to experience physical or cognitive symptoms of anxiety as dangerous or threatening). The measure has strong psychometric properties (Taylor et al., 2007). Anxiety sensitivity is elevated across a range of anxiety disorders and is generally positively related to ED symptoms (Davey & Chapman, 2009; Fulton et al., 2012; Olatunji & Wolitzky-Taylor, 2009). Due to theoretical and empirical links between anxiety sensitivity and safety behaviors, such that individuals with elevations in anxiety sensitivity are linked to more frequent engagement in safety behaviors (Olatunji & Wolitzky-Taylor, 2009), the ASI-3 will provide a measure of convergent validity. Furthermore, past work has also suggested elevations in anxiety sensitivity in ED samples (Anestis, Holm-Denoma, Gordon, Schmidt, & Joiner, 2008). We anticipate a small-to-medium correlation between the ASI-3 and our measure of safety behaviors.

Intolerance of Uncertainty Scale, Short Form (Carleton, Norton, & Asmundson, 2007)

The Intolerance of Uncertainty Scale, Short Form (IUS) is a 12-item measurement of intolerance of uncertainty, a personality characteristic that refers to negative cognitive, affective, and behavioral responses to uncertainty. IU is linked to excessive avoidance behaviors in anxiety (Flores, López, Vervliet, & Cobos, 2018) and is also elevated in EDs (Brown et al., 2017). Because IU is thought to predispose individuals to engage in behaviors to manage feelings of anxiety, it is theoretically relevant to safety behaviors, and we will use the IUS-12 as a measure of convergent validity. Similar to the ASI-3, we anticipate small-to-medium correlations between the IUS-12 and our measure of safety behaviors.

Behavioral Activation Scale (Carver & White, 1994)

The Behavioral Activation Scale (BAS) is a 13-item self-report measure with three subscales gauging reward responses. Although emerging research suggests that reward-based processing may be altered in anxiety (Richey et al., 2014; Taylor, Lyubomirsky, & Stein, 2017), there is no strong theoretical link between reward processing and safety behavior use. Therefore, we will use the BAS as a measure of discriminant validity.

Fidelity checks

As in Study 1, we will include two fidelity items (e.g., “Please select always.”) in the first two sets of questions in the survey. Participants who do not pass either check will be redirected to the end of the survey.

2.3.4 |. Procedures

All methods, procedures, and statistical analyses are preregistered at https://osf.io/tukah/.

2.3.5 |. ED recruitment

Participants with EDs will be recruited through two methods. First, we will post recruitment information about the study on ED-specific forums (e.g., Reddit) and on Listservs for ED organizations (e.g., the Academy for Eating Disorders). A final list of all Listservs and forums used for recruitment along with the standardized posting will be shared on the study's OSF page. In these posts, the description of the study, inclusion and exclusion criteria, and the link to the survey will be included. The participants will first be directed to the EDDS to determine initial eligibility for the study, and those who meet eligibility criteria will be directed to the remainder of study procedures.

The second method for recruitment will take place at several of the clinics of the authors of this paper (authors JE, KS, NRF, TAB, MC/CS, SG, and LMA), which encompass outpatient, intensive outpatient, and partial hospitalization programs. The data representative for each site (the author[s] affiliated with each site) will be provided a set of recruitment materials, which include study flyers to post in the clinic space and/or include in email, and a script to follow when presenting the flyers to patients. Given variability in methods of treatment delivery, data representatives can choose whether to present this information to subjects in person or via email; however, we will instruct providers to not present this information to subjects during a therapy session and instead present outside of session time. Across all recruitment at treatment sites, the script will highlight the anonymity of the data and reiterate that participation in the study will have no bearing on the individual's treatment.

2.3.6 |. HC recruitment

We plan to recruit HC subjects via social media postings on Facebook and Twitter. The study team will share study information on their pages using a standardized script, which will be posted on OSF. If we experience difficulty with recruitment (defined as <50 participants in 3 months), we will recruit HCs through the Hofstra University research pool.

2.3.7 |. Compensation

Participants who complete the study will be entered to win one of three $300 gift cards.

2.3.8 |. Statistical analysis plan

Sample size determination

We plan to recruit a minimum of 250 participants, based on past work suggesting that correlation coefficients stabilize and increase in accuracy around this sample size (Maxwell, Kelley, & Rausch, 2008). However, our final sample size will be dictated by the number of items included in the pilot measurement, such that we will recruit 10 participants with EDs per each item in the version of the measure being tested (e.g., if there are 40 items in the revised measurement, we will recruit 400 participants). Of note, methods for sample size determination in factor analysis based on “rules of thumb” have been critiqued in the literature, as the qualities of the data and measure design influence the sample size needs of the study (Maccallum et al., 1999). Maccallum et al. (1999) highlight that required sample sizes are contingent on overdetermination (i.e., the extent to which there are multiple indicators of each factor) and communality (i.e., the extent to which items correlate with one another). Given the number of items that we introduce in the original measurement, alongside past measurements of anxiety-based safety behaviors often suggesting two to three theoretical-driven factors with factor loadings above 0.6 (Cuming et al., 2009; Plasencia, Alden, & Taylor, 2011), we anticipate that there will be adequate overdetermination and communality to justify a sample size 250+.

Because we do not plan to explore the factor structure in HCs and instead simply plan to conduct mean comparisons across the ED and HC groups in safety behavior scores, we will plan to recruit 150 HCs, as power analyses in G*Power (Erdfelder, Faul, & Buchner, 1996) indicated that an appropriate sample size for mean comparisons across unequal groups would be ~143, presuming a 3:1 allocation ratio of the sample, power set at 0.95, one-tailed test, alpha set at 0.05, and a small-to-medium effect size (0.32) based on prior research comparing ED and HC subjects on trait anxiety constructs (Pollice, Kaye, Greeno, & Weltzin, 1997).

Exploratory factor analysis

Within the ED group, we will explore inter-item correlations across all items retained from Study 1 to determine whether items provide appropriate representation of the construct of interest. We will conduct an exploratory factor analysis to reduce the multidimensionality of the data using the principal component factor extraction method. Consistent with existing recommendations (Brown, 2015) and with the understanding that we may alter our approach pending the structure and specifics of our data, we will determine the appropriateness of factor analysis for our data using accepted cutoffs (Tabachnick & Fidell, 2013) for the Kaiser–Meyer–Olkin Measure of Sampling Adequacy, Bartletťs test of sphericity, and Mardia's tests of multivariate normality. The number of factors extracted will be determined primarily using Horn's parallel analysis (Horn, 1965). Secondarily, we will also apply the Kaiser–Guttman criterion (Guttman, 1954; Kaiser, 1960) and inspect a scree plot (Cattell, 1966). These benchmarks will be weighed alongside theoretical considerations from the researchers. The retained factors will be rotated using an orthogonal (varimax) and an oblique (promax) rotation with Kaiser normalizations to determine the clearest factor loadings. We will eliminate items that demonstrate poor performance (i.e., factor loadings<0.32; low inter-item correlations; cross-loadings; uniqueness<0.5) and remove factors with <3 items. Internal consistency for the final factors will be evaluated using Cronbach's a. Missing data will be handled with multiple imputation.

Known-groups validity

We will sum the mean score on the scale for each group and use analysis of variance (ANOVA) to explore potential group differences.

Convergent and divergent validity

Within both the clinical and HC groups, we will explore associations between total scores on the measurement, which we will compute using a total score for all final items, and clinical variables listed earlier using bivariate correlations. Anxiety sensitivity, intolerance of uncertainty, and general safety behaviors will provide measures of convergent validity. The BAS will serve as a measure of divergent validity.

2.3.9 |. Reliability

We will explore reliability of the measurement using internal consistency. We will calculate Cronbach's α for items on each subscale as indicated by the exploratory factor analysis. We will also report internal consistency for the full measurement.

3 |. CONCLUSION

Safety behaviors are a well-characterized feature of anxiety disorders and are considered critical to assess and target in existing treatments. Due to overlap between anxiety and ED pathology, alongside initial data supporting the presence of safety behaviors in ED samples, further characterizing safety behaviors in EDs offers one avenue for improving the field's understanding of the maintenance and treatment of ED symptoms. Development of an assessment to gauge ED-specific safety behaviors will facilitate this future research and potentially offer a clinically relevant tool for providers treating patients with EDs.

Supplementary Material

Appendix A

Acknowledgments

Funding information

National Institute of Mental Health, Grant/Award Numbers: K01MH123914, K23MH123910, T32MH0118261–33

Footnotes

CONFLICT OF INTEREST

The authors declare no conflict of interest.

DATA AVAILABILITY STATEMENT

Data sharing not applicable to this article as no datasets were generated or analyzed during the current study. When the study is conducted, the data that support the findings of this study are available from the corresponding author upon reasonable request.

SUPPORTING INFORMATION

Additional supporting information may be found online in the Supporting Information section at the end of this article.

REFERENCES

  1. Anestis MD, Holm-Denoma JM, Gordon KH, Schmidt NB, & Joiner TE (2008). The role of anxiety sensitivity in eating pathology. Cognitive Therapy and Research, 32(3), 370–385. 10.1007/s10608-006-9085-y [DOI] [Google Scholar]
  2. Becker CB, Farrell NR, & Waller G (2019). Exposure therapy for eating disorders. Oxford, England: Oxford University Press. [Google Scholar]
  3. Blakey SM, & Abramowitz JS (2016). The effects of safety behaviors during exposure therapy for anxiety: Critical analysis from an inhibitory learning perspective. Clinical Psychology Review, 49,1–15. 10.1016/j.cpr.2016.07.002 [DOI] [PubMed] [Google Scholar]
  4. Brown M, Robinson L, Campione GC, Wuensch K, Hildebrandt T, & Micali N (2017). Intolerance of uncertainty in eating disorders: A systematic review and meta-analysis. European Eating Disorders Review, 25(5), 329–343. 10.1002/erv.2523 [DOI] [PubMed] [Google Scholar]
  5. Brown TA (2015). Confirmatory factor analysis for applied research (2nd ed.). New York, NY: The Guilford Press. [Google Scholar]
  6. Butler RM, & Heimberg RG (2020). Exposure therapy for eating disorders: A systematic review. Clinical Psychology Review, 78, 101851. 10.1016/j.cpr.2020.101851 [DOI] [PubMed] [Google Scholar]
  7. Carleton RN, Norton MAPJ, & Asmundson GJG (2007). Fearing the unknown: A short version of the Intolerance of Uncertainty Scale. Journal of Anxiety Disorders, 21(1), 105–117. 10.1016/j.janxdis.2006.03.014 [DOI] [PubMed] [Google Scholar]
  8. Carver CS, & White TL (1994). Behavioral inhibition, behavioral activation, and affective responses to impending reward and punishment: The BIS/BAS scales. Journal of Personality and Social Psychology, 67(2), 319–333. 10.1037/0022-3514.67.2.319 [DOI] [Google Scholar]
  9. Cattell RB (1966). The scree test for the number of factors. Multivariate Behavioral Research, 1(2), 245–276. 10.1207/s15327906mbr0102_10 [DOI] [PubMed] [Google Scholar]
  10. Clark DM, Butler G, Fennell M, Hackmann A, McManus F, & Wells A (1995). Social behaviour questionnaire. Unpublished manuscript.
  11. Clark LA, & Watson D (1995). Constructing validity: Basic issues in objective scale development. Psychological Assessment, 7(3), 309–319. 10.1037/1040-3590.7.3.309 [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Cuming S, Rapee RM, Kemp N, Abbott MJ, Peters L, & Gaston JE (2009). A self-report measure of subtle avoidance and safety behaviors relevant to social anxiety: Development and psychometric properties. Journal of Anxiety Disorders, 23(7), 879–883. 10.1016/j.janxdis.2009.05.002 [DOI] [PubMed] [Google Scholar]
  13. Davey GCL, & Chapman L (2009). Disgust and eating disorder symptomatology in a non-clinical population: The role of trait anxiety and anxiety sensitivity. Clinical Psychology & Psychotherapy, 16(4), 268–275. 10.1002/cpp.623 [DOI] [PubMed] [Google Scholar]
  14. Erdfelder E, Faul F, & Buchner A (1996). GPOWER: A general power analysis program. Behavior Research Methods, Instruments & Computers, 28(1), 1–11. 10.3758/BF03203630 [DOI] [Google Scholar]
  15. Fairburn CG (2008). Cognitive behavior therapy and eating disorders. New York, NY: Guilford Press. [Google Scholar]
  16. Fairburn CG, Cooper Z, & O'Connor M (2008). 'Eating Disorder Examination'. In Fairburn CG (Ed.), Cognitive Behaviour Therapy and Eating Disorders. New York, NY: Guilford Press. 10.1037/t03975-000 [DOI] [Google Scholar]
  17. Flores A, López FJ, Vervliet B, & Cobos PL (2018). Intolerance of uncertainty as a vulnerability factor for excessive and inflexible avoidance behavior. Behaviour Research and Therapy, 104,34–43. 10.1016/j.brat.2018.02.008 [DOI] [PubMed] [Google Scholar]
  18. Fulton JJ, Lavender JM, Tull MT, Klein AS, Muehlenkamp JJ, & Gratz KL (2012). The relationship between anxiety sensitivity and disordered eating: The mediating role of experiential avoidance. Eating Behaviors, 13(2), 166–169. 10.1016/j.eatbeh.2011.12.003 [DOI] [PubMed] [Google Scholar]
  19. Guttman L (1954). Some necessary conditions for common-factor analysis. Psychometrika, 19(2), 149–161. 10.1007/BF02289162 [DOI] [Google Scholar]
  20. Helbig-Lang S, & Petermann F (2010). Tolerate or eliminate? A systematic review on the effects of safety behavior across anxiety disorders. Clinical Psychology: Science and Practice, 17(3), 218–233. https://doi. org/10.1111/j.1468–2850.2010.01213.x [Google Scholar]
  21. Henry JD, & Crawford JR (2005). The short-form version of the depression anxiety stress scales (DASS-21): Construct validity and normative data in a large non-clinical sample. British Journal of Clinical Psychology, 44(2), 227–239. 10.1348/014466505X29657 [DOI] [PubMed] [Google Scholar]
  22. Hertzog MA (2008). Considerations in determining sample size for pilot studies. Research in Nursing & Health, 31(2), 180–191. 10.1002/nur.20247 [DOI] [PubMed] [Google Scholar]
  23. Horn JL (1965). A rationale and test for the number of factors in factor analysis. Psychometrika, 30(2), 179–185. 10.1007/BF02289447 [DOI] [PubMed] [Google Scholar]
  24. Hudson JI, Hiripi E, Pope HG, & Kessler RC (2007). The prevalence and correlates of eating disorders in the National Comorbidity Survey Replication. Biological Psychiatry, 61(3), 348–358. 10.1016/j.biopsych.2006.03.040 [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Johanson GA, & Brooks GP (2010). Initial scale development: Sample size for pilot studies. Educational and Psychological Measurement, 70(3), 394–400. 10.1177/0013164409355692 [DOI] [Google Scholar]
  26. Kaiser HF (1960). The application of electronic computers to factor analysis. Educational and Psychological Measurement, 20(1), 141–151. 10.1177/001316446002000116 [DOI] [Google Scholar]
  27. Krabbenborg MAM, Danner UN, Larsen JK, van der Veer N, van Elburg AA, de Ridder DTD, … Engels RCME. (2012). The eating disorder diagnostic scale: Psychometric features within a clinical population and a cut-off point to differentiate clinical patients from healthy controls. European Eating Disorders Review, 20(4), 315–320. 10.1002/erv.1144 [DOI] [PubMed] [Google Scholar]
  28. Lock J, Grange DL, Agras WS, Moye A, Bryson SW, & Jo B (2010). Randomized clinical trial comparing family-based treatment with adolescent-focused individual therapy for adolescents with anorexia nervosa. Archives of General Psychiatry, 67(10), 1025–1032. 10.1001/archgenpsychiatry.2010.128 [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Lovibond PF, & Lovibond SH (1995). The structure of negative emotional states: Comparison of the depression anxiety stress scales (DASS) with the Beck depression and anxiety inventories. Behaviour Research and Therapy, 33(3), 335–343. 10.1016/0005-7967(94)00075-U [DOI] [PubMed] [Google Scholar]
  30. Maccallum RC, Widaman KF, Zhang S, Hong S, & Sehee. (1999). Sample size in factor analysis. Psychological Methods, 4,84–99. [Google Scholar]
  31. Maxwell SE, Kelley K, & Rausch JR (2008). Sample size planning for statistical power and accuracy in parameter estimation. Annual Review of Psychology, 59(1), 537–563. 10.1146/annurev.psych.59.103006.093735 [DOI] [PubMed] [Google Scholar]
  32. Mulkens S, de Vos C, de Graaff A, & Waller G (2018). To deliver or not to deliver cognitive behavioral therapy for eating disorders: Replication and extension of our understanding of why therapists fail to do what they should do. Behaviour Research and Therapy, 106, 57–63. 10.1016/j.brat.2018.05.004 [DOI] [PubMed] [Google Scholar]
  33. Murray SB, Nagata JM, Griffiths S, Calzo JP, Brown TA, Mitchison D, … Mond JM. (2017). The enigma of male eating disorders: A critical review and synthesis. Clinical Psychology Review, 57, 1–11. 10.1016/j.cpr.2017.08.001 [DOI] [PubMed] [Google Scholar]
  34. Murray SB, Treanor M, Liao B, Loeb KL, Griffiths S, & Le Grange D (2016). Extinction theory & anorexia nervosa: Deepening therapeutic mechanisms. Behaviour Research and Therapy, 87, 1–10. 10.1016/j.brat.2016.08.017 [DOI] [PubMed] [Google Scholar]
  35. Netemeyer R, Bearden W, & Sharma S (2003). Scaling procedures. Thousand Oaks, CA: Sage. 10.4135/9781412985772 [DOI] [Google Scholar]
  36. Olatunji BO, & Wolitzky-Taylor KB (2009). Anxiety sensitivity and the anxiety disorders: A meta-analytic review and synthesis. Psychological Bulletin, 135(6), 974–999. 10.1037/a0017428 [DOI] [PubMed] [Google Scholar]
  37. Pallister E, & Waller G (2008). Anxiety in the eating disorders: Understanding the overlap. Clinical Psychology Review, 28(3), 366–386. 10.1016/j.cpr.2007.07.001 [DOI] [PubMed] [Google Scholar]
  38. Plasencia ML, Alden LE, & Taylor CT (2011). Differential effects of safety behaviour subtypes in social anxiety disorder. Behaviour Research and Therapy, 49(10), 665–675. 10.1016/j.brat.2011.07.005 [DOI] [PubMed] [Google Scholar]
  39. Pollice C, Kaye WH, Greeno CG, & Weltzin TE (1997). Relationship of depression, anxiety, and obsessionality to state of illness in anorexia nervosa. International Journal of Eating Disorders, 21(4), 367–376. [DOI] [PubMed] [Google Scholar]
  40. Reilly EE, Anderson LM, Gorrell S, Schaumberg K, & Anderson DA (2017). Expanding exposure-based interventions for eating disorders. The International Journal of Eating Disorders, 50(10), 1137–1141. 10.1002/eat.22761 [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Richey JA, Rittenberg A, Hughes L, Damiano CR, Sabatino A, Miller S, … Dichter GS. (2014). Common and distinct neural features of social and non-social reward processing in autism and social anxiety disorder. Social Cognitive and Affective Neuroscience, 9(3), 367–377. 10.1093/scan/nss146 [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Salkovskis PM (1991). The importance of behaviour in the maintenance of anxiety and panic: A cognitive account. Behavioural and Cognitive Psychotherapy, 19(1), 6–19. 10.1017/S0141347300011472 [DOI] [Google Scholar]
  43. Schaumberg K, Reilly EE, Gorrell S, Levinson CA, Farrell NR, Brown TA, … Anderson LM. (2021). Conceptualizing eating disorder psychopathology using an anxiety disorders framework: Evidence and implications for exposure-based clinical research. Clinical Psychology Review, 83, 101952. 10.1016/j.cpr.2020.101952 [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Stice E, Fisher M, & Martinez E (2004). Eating disorder diagnostic scale: Additional evidence of reliability and validity. Psychological Assessment, 16(1), 60–71. 10.1037/1040-3590.16.1.60 [DOI] [PubMed] [Google Scholar]
  45. Stice E, Telch CF, & Rizvi SL (2000). Development and validation of the eating disorder diagnostic scale: A brief self-report measure of anorexia, bulimia, and binge-eating disorder. Psychological Assessment, 12(2), 123–131. 10.1037/1040-3590.12.2.123 [DOI] [PubMed] [Google Scholar]
  46. Sunday SR, Halmi KA, & Einhorn A (1995). The Yale-Brown-Cornell eating disorder scale: A new scale to assess eating disorder symptomatology. International Journal of Eating Disorders, 18(3), 237–245. [DOI] [PubMed] [Google Scholar]
  47. Swanson SA, Crow SJ, Grange DL, Swendsen J, & Merikangas KR (2011). Prevalence and correlates of eating disorders in adolescents: Results from the National Comorbidity Survey Replication Adolescent Supplement. Archives of General Psychiatry, 68(7), 714–723. 10.1001/archgenpsychiatry.2011.22 [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Tabachnick BG, & Fidell LS (2013). Using multivariate statistics. New York, NY: Pearson Education. [Google Scholar]
  49. Taylor CT, Lyubomirsky S, & Stein MB (2017). Upregulating the positive affect system in anxiety and depression: Outcomes of a positive activity intervention. Depression and Anxiety, 34(3), 267–280. 10.1002/da.22593 [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Taylor S, Zvolensky MJ, Cox BJ, Deacon B, Heimberg RG, Ledley DR, … Cardenas SJ. (2007). Robust dimensions of anxiety sensitivity: Development and initial validation of the anxiety sensitivity Index-3. Psychological Assessment, 19(2), 176–188. 10.1037/1040-3590.19.2.176 [DOI] [PubMed] [Google Scholar]
  51. Turner H, Tatham M, Lant M, Mountford VA, & Waller G (2014). Clinicians' concerns about delivering cognitive-behavioural therapy for eating disorders. Behaviour Research and Therapy, 57,38–42. 10.1016/j.brat.2014.04.003 [DOI] [PubMed] [Google Scholar]
  52. Waller G (2009). Evidence-based treatment and therapist drift. Behaviour Research and Therapy, 47(2), 119–127. 10.1016/j.brat.2008.10.018 [DOI] [PubMed] [Google Scholar]
  53. Waller G, & Marcoulides OK (2013). Safety Behaviours in eating disorders: Factor structure and clinical validation of the brief safety Behaviours scale. European Eating Disorders Review, 21(3), 257–261. 10.1002/erv.2208 [DOI] [PubMed] [Google Scholar]
  54. Waller G, Stringer H, & Meyer C (2012). What cognitive behavioral techniques do therapists report using when delivering cognitive behavioral therapy for the eating disorders? Journal of Consulting and Clinical Psychology, 80(1), 171–175. 10.1037/a0026559 [DOI] [PubMed] [Google Scholar]
  55. Wells A, Clark DM, Salkovskis P, Ludgate J, Hackmann A, & Gelder M (1995). Social phobia: The role of in-situation safety behaviors in maintaining anxiety and negative beliefs. Behavior Therapy, 26 (1), 153–161. 10.1016/S0005-7894(05)80088-7 [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Appendix A

RESOURCES