Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2016 Dec 1.
Published in final edited form as: J Behav Ther Exp Psychiatry. 2014 Dec 30;49(0 0):53–60. doi: 10.1016/j.jbtep.2014.12.004

Training less threatening interpretations over the Internet: Does the number of missing letters matter?

Shari A Steinman 1, Bethany A Teachman 1
PMCID: PMC4486638  NIHMSID: NIHMS652225  PMID: 25579609

Abstract

Background and objectives

Cognitive Bias Modification to reduce threat interpretations (CBM-I) trains individuals to resolve ambiguous scenarios via completion of word fragments that assign benign meanings to scenarios. The current study tested: 1) whether Internet-based CBM-I can shift interpretations to be more positive/less negative, and 2) whether varying the number of letters missing in the word fragments (assumed to increase task difficulty) moderates CBM-I’s effects.

Methods

Participants (N=350) completed a brief online version of CBM-I, followed by assessments of interpretation bias, fear of negative evaluation, and anticipatory anxiety. Participants were randomly assigned to 1 of 5 conditions: control (half of scenarios ended positively, half negatively), or 4 positive conditions (all scenarios ended positively, but word fragments varied on number of letters missing, from 0 to 3).

Results

Relative to the control condition, all positive conditions led to more positive/less negative interpretations. When analyses were re-run with only a highly socially anxious subset of the sample (n=100), conditions in which the final word of scenarios was missing 0, 1, or 2 letters led to more positive/less negative interpretations compared to the control condition, but the condition missing 3 letters did not differ from the control condition. There were no differences between conditions on other outcome measures.

Limitations

Training was brief, and an unselected sample was used.

Conclusions

Results suggest a brief Internet-based CBM-I paradigm can shift interpretation bias, but not necessarily other anxiety-relevant outcomes. Making the task too difficult may blunt effects for highly socially anxious individuals.

Keywords: Cognitive Bias Modification, Interpretation, Social Anxiety, e-Health

Cognitive models of anxiety suggest that anxious individuals interpret ambiguous information in a negative or threatening way (Clark & Beck, 2010). This negative interpretation bias is theorized to maintain, and potentially cause, anxiety disorders. To test this theorized connection between interpretation bias and anxiety, researchers have developed computerized paradigms to directly shift the way individuals interpret ambiguous information, called Cognitive Bias Modification for Interpretations (CBM-I; Mathews & Mackintosh, 2000). CBM-I for anxiety trains participants to interpret ambiguous information in a less threatening way through conditioning paradigms. For example, a common CBM-I paradigm repeatedly presents individuals with ambiguous scenarios, which end in word fragments that, when completed, disambiguate the scenarios in a benign way (Mathews & Mackintosh, 2000). Training is expected to increase positive and reduce negative interpretations, and reduce anxiety symptoms.

Several lab-based studies have found that CBM-I can successfully modify interpretations to be more positive and less negative across diverse anxious samples (see meta-analyses by Hallion & Ruscio, 2011, and Menne-Lothmann et al., 2014). Moreover, a growing number of studies have demonstrated that shifting interpretations via CBM-I leads to a subsequent shift in anxiety levels, providing support for the causal claim in cognitive models of anxiety (see MacLeod & Mathews, 2012). However, not all CBM-I studies have produced promising results. Some have not changed bias, and others have produced shifts in unexpected directions (e.g., Fox, Mackintosh, & Holmes, 2014; see Hallion & Ruscio, 2011). Further, there is evidence of publication bias, such that nonsignificant findings are often not published (Hallion & Ruscio). Results have been particularly mixed in the few studies that have attempted to shift interpretation bias over the Internet. For example, in a study comparing various online treatments for depression, seven brief sessions of Internet-based CBM-I significantly increased positive interpretation bias and reduced depressive symptoms (Williams, Blackwell, Mackenzie, Holmes, & Andrews, 2013). On the contrary, another Internet-based study found that eight brief CBM-I sessions for anxiety shifted interpretations to be more positive and less negative (relative to a control condition); however, both the control and CBM-I conditions led to similar changes in anxiety and depression symptoms, and in subjective distress (Salemink, Kindt, Rienties, & van den Hout, 2014). Taken together, these results suggest that while CBM-I may one day be an efficacious treatment for psychopathology, it is important first to improve the reliability of CBM-I effects, and determine if the Internet is an appropriate way to disseminate CBM-I.

One possible way to increase the strength of CBM-I can be drawn from the memory and learning literature. According to desirable difficulty theories (e.g., Bjork, 1994, 1999; Pyc & Rawson, 2009), memory is improved when the learning process is relatively difficult, so that participants are challenged, but only to a point where they can still succeed. Consequently, it is possible that increasing the difficulty of the learning process in CBM-I tasks may lead to stronger training effects on bias and subsequent anxiety (see Hertel & Mathews, 2011). For instance, needing to actively generate benign resolutions when information is ambiguously threatening, versus passively reading a benign resolution, may be an example of a desirable difficulty. In fact, evidence demonstrates that CBM-I is more likely to affect subsequent mood and anxiety in response to a stressor when participants are asked to generate the emotional meanings of scenarios (i.e., complete word fragments), as opposed to when participants complete an easier version of CBM-I, in which they passively read the scenarios (i.e., there are no word fragments; e.g., Hoppitt, Mathews, Yiend, & Mackintosh, 2010; Mathews & Mackintosh, 2000). However, a recent study suggests that modifying CBM-I to be much more active, such that participants generated their own positive interpretations of ambiguous scenarios via a microphone, did not improve mood as effectively as traditional CBM-I (Rohrbacher et al., 2014), suggesting this issue requires further testing. In the current study, we test whether relatively more active training should be more effective than more passive training at changing interpretation bias and reducing anxiety, and extend prior work by varying the difficulty of completing the CBM-I task.

The current study evaluates whether a single Internet-based session of CBM-I that targets social anxiety-relevant interpretations can shift interpretations, fear of negative evaluation, and anticipatory social anxiety in a large, unselected sample. Participants were randomly assigned to 1 of 5 conditions: a control condition that is not designed to train positive interpretations (50% of scenarios end positively, 50% end negatively), a positive condition that does not involve active generation of emotional meaning of scenarios (all scenarios end positively, and scenarios do not include word fragments), or three positive conditions that vary the number of letters missing in the word fragments that resolve the emotional meaning of the scenarios, from 1 to 3). Following CBM-I, participants completed measures of interpretation bias, fear of negative evaluation, as well as anticipatory anxiety. We hypothesized that all positive training conditions would lead to more positive/less negative interpretations, relative to the control condition. Further, we predicted that all positive conditions that include a word fragment would lead to less fear of negative evaluation and anticipatory anxiety, based on prior findings that active resolution of word fragments (versus passive reading) led to greater changes in mood (Hoppitt et al., 2010; Mathews & Mackintosh, 2000). Finally, we predicted that increasing the amount of active generation needed to complete fragments, by increasing the number of letters missing from fragments, would lead to stronger CBM-I effects on all outcome measures.

Finally, the study included baseline measures of social anxiety symptom severity and interpretation bias to explore whether these individual differences would moderate who benefits the most from CBM-I. One possibility was that training effects would be strongest for people with a high level of baseline interpretation bias or symptom severity, given more opportunity to see training effects (i.e., room for improvement). Alternatively, training effects might be strongest for people with low baseline bias or symptom severity. Less severe symptoms may be less engrained and more malleable. Also, these individuals have already shown some aptitude for making relatively healthy interpretations, so their interpretation bias might be more amenable to change with a brief training program (i.e., capitalizing on a strength versus addressing a deficit). Given mixed prior empirical findings (e.g., Micco, Henin, & Hirshfeld-Becker, 2013, and Salemink & Wiers, 2011, found moderation by baseline interpretation bias, but Steinman, 2010, did not), these analyses are exploratory. We also assessed training effects in just the highly socially anxious subset of our sample in line with more traditional tests of the clinical utility of training.

Methods

Participants

Three hundred and fifty participants (64.9% female) were recruited over the Internet, via Amazon.com’s Mechanical Turtk (mTurk), in exchange for $0.401. Participants reported citizenship from 16 countries, with the majority of participants (92.9%) reporting U.S. citizenship. Participants’ ages ranged from 18 to 64 (M = 35.44, SD = 12.28). Ethnicity was reported as: 6.9% Hispanic or Latino, 86.6% not Hispanic or Latino, and 6.6% unknown or not reported, and race was reported as: 77.7% White, 9.4% Black, 4.9% Asian, 4.0% as more than one race, and 4.0% as other or unknown.

Materials2

Baseline Social Anxiety Symptoms

The Social Interaction and Anxiety Scale (SIAS; Mattick & Clarke, 1998) is a 20-item questionnaire that assesses reactions to a variety of social situations. In the SIAS, participants rate how characteristic 20 statements are of them (e.g., “I have difficulty talking with other people”) on a Likert scale. The SIAS has good psychometric properties (Rodebaugh, Woods, Heimberg, Liebowitz, & Schneier, 2006). The SIAS was administered at the beginning of the study to evaluate baseline social anxiety, both to check that the CBM-I conditions did not differ at baseline and to check whether severity of social anxiety symptoms moderated training effects. In the current sample, Cronbach’s alpha was .94, suggesting excellent reliability.

Cognitive Bias Modification for Interpretations (CBM-I) Task

Participants were asked to read and imagine themselves in a series of 36 scenarios (adapted from Mathews & Mackintosh, 2000). Each scenario was related to a social situation, and was designed to remain ambiguous until the final word of the scenario. The final word of the scenario resolved the ambiguity in either a positive or negative way, depending on the condition. For the current study, we defined “positive” as valenced positively (e.g., something good, positive, or happy occurs) or inconsistent with social anxiety (e.g., people are not judging you negatively, or others like you). We defined “negative” as valenced negatively (e.g., something negative or sad occurs) or consistent with social anxiety (e.g., people are judging you negatively, or others do not like you).

The 36 scenarios in each of the 5 conditions were identical, except for the final word in each scenario. For example, participants would read, “Your boss asks to see you following the recent submission of a paper you wrote. He tells you that he wants to talk to you because your work was …” In the control condition (labeled CBM-Control), half of the scenarios ended with a word fragment that, when completed, resolved the ambiguity in a negative way (e.g., “unc_ear” in the example above), and half of the scenarios ended with a word fragment that resolved the ambiguity in a positive way (e.g., “exc_ptional” in the example above). In the CBM-Control condition, the final word of each scenario was always missing one letter, which the participant had to correctly type to move on to the next scenario (e.g., “l” to complete the word “unclear,” or “e,” to complete the word “exceptional.”).

In the other four conditions, all scenarios ended with a word that resolved the ambiguity in a positive way (i.e., “exc_ptional” in the example above). However, to vary the amount of active generation needed to complete each scenario, the number of letters missing from the final word of each scenario varied, depending on the condition. In the CBM-0 condition, the final word of each scenario was not missing any letters. In the CBM-1, CBM-2, and CBM-3 conditions, the final word of each scenario was missing one, two, or three letters, respectively, that participants had to type correctly to move on to the next scenario.

The specific letters missing within the word fragments in the CBM-I task were selected by research assistants, and generally included similar letters to those missing in Mathews and Mackintosh’s (2000) original word fragments. Note that if a letter was missing in the CBM-1 condition, the same letter was also missing in the CBM-2 condition. Similarly, the two letters missing in the CBM-2 condition were missing in the CBM-3 condition.

An independent sample (N = 16) pre-rated the valence of all scenarios in the CBM-I task and all disambiguated interpretations in the Recognition Rating task (described below), on a scale of −5 (very negative) to +5 (very positive). All positive scenarios in the CBM-I task and all positive disambiguated interpretations in the Recognition Rating task had a mean rating above 0, and all negative scenarios in the CBM-I task and all negative disambiguated interpretations in the Recognition Rating task had a mean rating below 0. Further, one-sample t-tests demonstrated that the scenarios and disambiguated interpretations significantly differed from zero in the expected directions. Specifically, the mean positive scenario and positive disambiguated interpretation ratings were significantly greater than zero, and the mean negative scenario and negative disambiguated interpretation ratings were significantly less than zero (all p < .001). Given that our CBM-I task scenarios were modified from Mathews and Mackintosh’s (2000) original training materials, we did not match the target positive and negative word fragments on emotional intensity or psycholinguistic variables (e.g., number of letters in the words), though ensuring the conditions matched on these variables could be a useful check for future research.

Following each scenario, participants were asked a comprehension question (with a “yes” or “no” answer) to ensure that they had read the scenario, and to reinforce the positive or negative interpretation of the scenario. For the example above, the matching comprehension questions was “Are you going to get a bad review from your boss?” Participants were not allowed to continue to the next scenario until they correctly answered the comprehension question.

Effects of CBM-I

Interpretation Bias

To evaluate CBM-I’s effects on interpretations of novel, ambiguous scenarios, all participants completed the Recognition Rating Task (modified from Mathews & Mackintosh, 2000) following CBM-I administration. Similar in format to CBM-I, participants are asked to read and imagine themselves in a series of 10 ambiguous scenarios related to social situations. Each scenario ended with a word fragment (missing one letter), and was followed by a comprehension question. Unlike CBM-I, each Recognition Rating scenario included a title, and the final word did not resolve the ambiguity of the scenario. For example, participants might read “THE LOCAL CLUB: You are invited to attend a social event at a local club, although you don't know any of the members very well. As you approach the door you can hear conversation and loud music, but as you enter the room it stops for a mo_ent.” Participants would type the letter “m” to complete the word “moment,” and then get asked the corresponding comprehension question, “Do you know most of the club members very well?”

Next, participants were presented with the title of each scenario, along with four disambiguated versions of each scenario. Participants were asked to rate how similar each disambiguated version was to their recollection of the meaning of the original scenario on a scale of 1 (“very different in meaning”) to 4 (“very similar in meaning”). For each scenario, two disambiguated versions were related to social concerns (one positive, one negative), and two were unrelated to social concerns (one positive ‘foil’, one negative ‘foil’). The two interpretations unrelated to social concerns were included to evaluate whether CBM-I’s training effects are specific to social anxiety, or lead to less threatening interpretations more generally. For the example above, disambiguated versions included “conversation stops and club members glare at you” (negative, related to social concerns), “conversation stops so club members can greet you” (positive, related to social concerns), “you realize your favorite song was just playing (positive, unrelated to social concerns), and “you realize you forgot your wallet at home” (negative, unrelated to social concerns). The difference between endorsements of the positive versus negative disambiguated options indexes an interpretation bias.

To obtain an additional measure of social anxiety-relevant interpretation bias, participants completed eight social items from the Body Sensation Interpretation Questionnaire (BSIQ; Clark et al., 1997). In the BSIQ, participants are presented with ambiguous events, and are then asked to rate the extent they believe three alternative explanations for why the event might have occurred on a 0 (“not at all likely”) to 8 (“extremely likely”) scale. One explanation is always negative, whereas the other explanations are either neutral and/or positive. The average of the likelihood ratings for the negative events indexes an interpretation bias (following Steinman & Teachman, 2010). The BSIQ includes scenarios related to social situations, bodily sensations, and external events, and has good psychometric properties (Clark et al.). In the current study, only the eight scenarios related to social situations were administered. The eight social items were randomly split into two groups of four items (BSIQ-A and BSIQ-B). Half of the participants received the BSIQ-A as a pre-test measure of interpretation bias, and the BSIQ-B as a post-test measure of interpretation bias. The other half of the participants received the BSIQ-B at pre-test, and the BSIQ-A at post-test. In the current sample, Cronbach’s alpha for the different administrations of BSIQ-A and BSIQ-B ranged from .78 to .83 (M = .80).

Fear of Negative Evaluation

The Brief Fear of Negative Evaluation Scale (BFNE; Leary, 1983) is a 12-item questionnaire that assesses fear of negative evaluation, a key component in anxiety related to social situations. Participants rate how characteristic 12 statements are of them (e.g., “I am afraid that others will not approve of me”) on a 5-point Likert scale (from 1 [“not at all”] to 5 [“extremely”]). The BFNE has adequate reliability and validity (Leary). The BFNE was administered following training. Cronbach’s alpha was .91 in the current study, suggesting excellent reliability.

Anticipatory Anxiety

To evaluate if CBM-I affected how participants anticipated they would respond to social interactions, participants completed the Anticipated Social Interaction Questionnaire (ASIQ; modified from Murphy et al., 2007). Participants were told to imagine they would be meeting two people they did not know for a five-minute conversation, and then asked to use a Likert scale to rate their predicted anxiety (from 1 [“extremely relaxed”] to 7 [“extremely anxious”]) and predicted performance (from 1 [“extremely poor”] to 7 [“extremely well”]) in the upcoming social interaction

Procedure

This study was completed over the Internet, using Amazon’s Mechanical Turk (mTurk). Following consent, participants completed the SIAS, followed by the four items from the BSIQ. Participants were then randomly assigned to a CBM-I condition, and completed the CBM-I task. All participants were then asked to rate how difficult it was for them to solve the word fragments they saw during the CBM-I task on a scale of 1 (“extremely easy”) to 5 (“extremely difficult”). Participants who did not see word fragments were asked to select “I did not see fragments.” Next, participants completed the following measures in fixed order: Recognition Rating Task, BFNE, the remaining four items from the BSIQ and the ASIQ. Finally, participants were debriefed. Note that we chose not to do the all the measures at pre- and post-intervention, due to concerns about practice effects given the brevity of the CBM-I intervention.

Results

Descriptive Statistics

The training conditions did not differ in terms of baseline social anxiety symptoms, as measured by the SIAS (F(4, 345)=1.51, p = .199, ŋp2 = .02), or baseline interpretation bias, as measured by the BSIQ (F(4, 345) =1.76, p = .136, ŋp2 = .02). Conditions also did not differ in terms of ethnicity (χ2 = 4.50, p = .810), race (χ2 = 23.63, p = .098), education (χ2 = 15.53, p = .486), gender (χ2 = 15.25, p = .054), or whether or not participants reported US citizenship (χ2 = 6.54, p = .162). However, conditions differed in terms of age (F(4, 341) = 4.16, p = .003, ŋp2 = .05). Follow-up LSD tests suggest that the CBM-Control and CBM-2 conditions were older, on average, than the CBM-0 and CBM-1 conditions (all p ≤ .042). Age did not differ between the CBM-Control and CBM-2 conditions (p = .336), or between the CBM-0 and CBM-1 conditions (p = .896). Age of participants in the CBM-3 condition did not differ from any other condition (all p ≥ .100). Given the condition differences in age, all results reported below were re-run with age as a covariate. The pattern of results generally stayed the same when age was included as a covariate (except where noted), so this covariate was not included in the following analyses. See Table 1 for participant characteristics and descriptive statistics of baseline measures, separated by condition.

Table 1.

Participant Characteristics and Descriptive Statistics.

Condition: CBM-
Control
(n = 80)		 CBM-0
(n = 81)		 CBM-1
(n = 61)		 CBM-2
(n = 64)		 CBM-3
(n = 64)
Sex (% female) 62.50 61.73 57.38 82.81 60.94
Mean (SD) of Age 39.04 (12.71) 32.38 (11.38) 32.65 (11.97) 37.08 (12.47) 35.70 (11.69)
Mean (SD) of SIAS 22.80 (13.69) 27.14 (15.08) 27.21 (15.38) 23.09 (14.15) 26.19 (16.75)
Mean (SD) of BSIQ −0.09 (0.97) 0.21 (1.07) 0.04 (1.00) −0.18 (0.83) 0.11 (1.10)
Open in a new tab

Note. CBM = Cognitive bias modification. SIAS = Social Interaction and Anxiety Scale; BSIQ = Body Sensation Interpretation Questionnaire (z-score of Mean Negative Social Ratings).

Difficulty of CBM-I Conditions

To evaluate if completing the word fragments in the positive CBM-I task was perceived to be more difficult as the number of missing letters in word fragments increased, a planned comparison was conducted to test for a linear effect. Note that only data from CBM-1, CBM-2, and CBM-3 conditions were included; CBM-0 was not included because those participants were told to respond to the difficulty item by selecting, “I did not see fragments”). We cut three participants’ data from this measure who reported, “I did not see fragments” even though they were in a condition that included fragments. Results suggested a linear trend, as expected, such that as the number of missing letters in fragments increased, the perceived difficulty also increased (F(1,183) = 4.22, p = .041). Follow-up planned contrasts revealed that participants did not rate fragment completion difficulty differently for the CBM-1 and CBM-2 conditions (t(183) = .15, p = .884), but participants rated fragment completion in the CBM-3 condition to be significantly more difficult than the CBM-2 condition (t(183) = 2.21, p = .028) and the CBM-1 condition (t(183) = 2.05, p = .041). Taken together, these results suggest that the CBM-3 condition was likely more difficult than the other positive CBM-I conditions, supporting the validity of this manipulation to increase desirable difficulty.

Effects of CBM-I

Interpretation Bias

Interpretation bias for Recognition Rating data was calculated by subtracting scores for negative disambiguated options from positive disambiguated options, so higher scores indicate a more positive/less negative interpretation bias. This calculation was done to reduce the number of analyses, and because we did not have distinct hypotheses for effects of CBM-I for positive versus negative disambiguated options (beyond the obvious reverse direction of effects). A repeated measures analysis of variance (ANOVA) was conducted with one within subjects factor (Topic: Social, Foil) and one between subjects factor (Condition: CBM-Control, CBM-0, CBM-1, CBM-2, CBM-3). Results revealed a main effect of Topic (F(1,333) = 144.47, p < .001, ŋp2 = .30), such that participants were more likely to endorse disambiguated Social interpretations, relative to Foil interpretations3. Results also revealed a main effect of Condition (F(4,333)=3.62, p = .007, ŋp2 = .04). Follow-up LSD tests suggested that across Topics, participants in the positive CBM-I conditions (CBM-0, CBM-1, CBM-2, and CBM-3) endorsed more positive/less negative interpretations, relative to participants in the CBM-Control condition (all p ≤ .046, except the comparison between CBM-Control and CBM-3 did not reach significance, p = .051, unless age was included as a covariate, p = .011). The four positive CBM-I conditions did not significantly differ from each other (all p ≥ .143). Importantly, these effects were subsumed by the expected Topic by Condition interaction (F(4,333)=3.01, p = .018, ŋp2 = .04).

To break down this interaction, we conducted separate univariate ANOVAs on Social and Foil interpretation options. As expected, there was a significant effect of Condition on Social disambiguated interpretations (F(4,333) = 3.81, p = .005, ŋp2 = .04, see Figure 1). Follow-up LSD tests suggest that the CBM-Control condition endorsed less positive/more negative interpretations, relative to each of the positive conditions (all p ≤ .034), and the four positive CBM-I conditions did not significantly differ from each other (all p ≥ .196). There was no effect of Condition on Foil disambiguated interpretations (F(4,333) = 1.35, p = .252, ŋp2 = .02), suggesting that CBM-I effects were specific to social-anxiety relevant topics and did not lead to a more positive interpretation style more generally.

Figure 1. Effect of Condition on Social-Anxiety Relevant Recognition Ratings (full sample).

Figure 1

Open in a new tab

Note. CBM = Cognitive bias modification. Higher numbers indicate greater endorsement of positive relative to negative interpretation options.

To evaluate the effect of CBM-I on post-intervention interpretation bias using a format that is less similar to the materials used for training, a repeated measures ANOVA was conducted using BSIQ data (after creating separate z-scores for the BSIQ-A and B forms), with one within-subjects factor (Time: Pre-intervention, Post-intervention), and two between-subjects factors (Condition, and Order: BSIQ-A administered pre-intervention and BSIQ-B post-intervention, or BSIQ-B administered pre-intervention and BSIQ-A post-intervention). Results did not reveal any main or interaction effects (all p ≥ .12).

Fear of Negative Evaluation

To assess the effect of CBM-I on fear of negative evaluation (measured by the BFNE), a univariate ANOVA with one between-subjects factor (Condition) was conducted. Condition did not affect fear of negative evaluation (F(4,322) = .12, p = .976, ŋp2 = .001).

Predicted Anxiety and Performance in an Anticipated Social Interaction

To evaluate the effect of CBM-I on predicted anxiety and performance in an anticipated social interaction (measured by the ASIQ), two univariate ANOVAs with one between-subjects factor (Condition) were conducted. Condition did not affect predicted anxiety (F(4,317) = .61, p = .658, ŋp2 = .01) or performance (F(4,317) = 1.12, p = .349, ŋp2 = .01).

Taken together, results suggest that while Internet-based CBM-I significantly shifted interpretations (as measured by the Recognition Rating Task) in the expected direction, it did not generalize to another measure of interpretation bias (the BSIQ) or measures of fear of negative evaluation (the BFNE) or anticipatory anxiety (the ASIQ).

Moderation of CBM-I Effects

Baseline Social Anxiety as a Moderator

To evaluate if baseline social anxiety symptom severity moderated effects of CBM-I, analyses with the full sample were re-run with baseline SIAS included as a continuous moderator. Given the focused nature of this moderator question, we only report main effects for baseline SIAS, and interactions involving both baseline SIAS and Condition (e.g., we do not report the general Topic or Condition effects already discussed above). For all outcomes, there was a main effect of baseline SIAS (all p < .001), such that participants with lower levels of baseline social anxiety symptom severity were also “healthier” following the intervention. Specifically, following CBM-I, individuals with lower levels of baseline social anxiety symptom severity were more likely to have a more positive/less negative interpretation bias (as measured by the BSIQ and Recognition Rating Task), less fear of negative evaluation (as measured by the BFNE), and lower levels of anxiety and better predicted performance in an anticipated social interaction (as measured by the ASIQ). Notably, baseline SIAS did not interact with Condition in any analyses (all p ≥ .109).

Of note, when age was included as a covariate, there was a significant baseline SIAS by Condition interaction for the Recognition Rating measure of interpretation bias (F(4,323) = 2.72, p = .030, ŋp2 = .03). To understand the interaction, a median split was conducted on the baseline SIAS scores, and then separate univariate ANOVAs were conducted on the resulting Low and High baseline social anxiety groups. For both groups, there was a main effect of Condition (both p ≤ .012), but follow-up tests revealed different condition effects. Specifically, in the Low baseline social anxiety group, follow-up tests suggest that participants in the four positive CBM-I conditions (CBM-0, CBM-1, CBM-2, CBM-3) endorsed more positive/less negative interpretations, relative to participants in the CBM-Control condition (all p ≤ .028), and the four positive CBM-I conditions did not significantly differ from each other (all p ≥ .127), replicating the pattern for the full sample. However, in the High baseline social anxiety group, follow-up tests suggest that participants in the CBM-0 and CBM-1 conditions endorsed more positive/less negative interpretations, relative to the CBM-Control condition (both p ≤.026), but the CBM-Control condition, CBM-2, and CBM-3 conditions did not differ from each other (all p ≥ .051). Further, the CBM-0, CBM-1, and CBM-2 conditions did not differ from each other (all p ≥ .473). This suggests that for those high in baseline symptoms, only the ‘easier’ versions of the CBM-I task were effective, while for those low in baseline symptoms, both easy and difficult forms of positive training reduced interpretation bias.

Baseline Interpretation Bias as a Moderator

To evaluate if baseline interpretation bias moderated effects of CBM-I, analyses with the full sample were re-run with z-scores of baseline BSIQ included as a continuous moderator. As with SIAS, given the focused nature of this moderator question, we only report main effects for baseline BSIQ, and interactions involving both baseline BSIQ and Condition.

Similar to results with baseline SIAS as a moderator, for all outcomes, there was a main effect of baseline BSIQ (all p ≤ .001), such that participants with lower levels of baseline interpretation bias were “healthier” following the intervention. Specifically, following CBM-I, individuals with lower levels of baseline interpretation bias were more likely to have a more positive/less negative interpretation bias (as measured by the Recognition Rating ask), less fear of negative evaluation (as measured by the BFNE), and lower levels of anxiety and better predicted performance in an anticipated social interaction (as measured by the ASIQ). There were no interaction effects involving baseline BSIQ and Condition for the two ASIQ items (both p ≥ .189), but there were interaction effects involving BSIQ and Condition for the Recognition Rating task (F(4,328) = 2.52, p = .041, ŋp2 = .03), and the BFNE scale (F(4,317) = 3.09, p = .016, ŋp2 = .04).

To unpack the Condition by Baseline BSIQ interaction predicting Recognition Rating scores, the disambiguated interpretation options were averaged across the Social and Foil topics (given the Condition by Baseline BSIQ by Topic interaction did not reach significance; F(4,328) = 1.87, p = .115, ŋp2 = .02).4 Next, a median split was conducted on the baseline BSIQ variable, and then separate univariate ANOVAs were conducted on the resulting Low and High baseline interpretation bias groups. For the High baseline interpretation bias group, the main effect of Condition did not reach significance (F(4,150) = 1.96, p = .104, ŋp2 = .05), but for the Low baseline interpretation bias group, there was a main effect of Condition (F(4,178) = 3.43, p = .010, ŋp2 = .07). Follow-up LSD tests suggest that participants in the four positive CBM-I conditions (CBM-0, CBM-1, CBM-2, CBM-3) endorsed more positive/less negative interpretations, relative to participants in the CBM-Control condition (all p ≤ .008), and the four positive CBM-I conditions did not significantly differ from each other (all p ≥ .757). These results suggest that positive CBM-I training led to more positive/less negative interpretations for those who started out with relatively minimal bias, but training condition did not impact those with a relatively high baseline bias.

To unpack the Condition by Baseline BSIQ interaction predicting BFNE, separate univariate ANOVAs testing for the effects of Condition were conducted for the Low and High baseline interpretation bias groups. In both analyses, the main effect of Condition did not reach significance (both p ≥ .605), suggesting that while there was some evidence for Baseline BSIQ moderating a Condition effect on BFNE, this interaction should be interpreted cautiously given there was not a clear effect of Condition when the data were examined separately as a function of being high versus low in baseline interpretation bias.

Effects of CBM-I on Highly Socially Anxious Subsample

One hundred participants had baseline SIAS scores that met or exceeded the recommended cutoff for clinical levels of social anxiety (34 or higher; Brown et al., 1997). Thus, to more directly evaluate the clinical utility of CBM-I, the analyses were re-run with this highly socially anxious subsample. To test for training effects on the Recognition Rating task, a repeated measures ANOVA was conducted with one within subjects factor (Topic: Social, Foil) and one between subjects factor (Condition). Results did not reveal a main effect of Topic (F(1,88) = 1.82, p = .181, ŋp2 = .02), or a Topic by Condition interaction (F(4,88) = 1.68, p = .163, ŋp2 = .07). However, results did reveal a main effect of Condition (F(4,88) = 4.46, p = .003, ŋp2 = .17, see Figure 2). Follow-up LSD tests suggest that the CBM-0, CBM-1, and CBM-2 conditions led to more positive/less negative interpretations compared to the CBM-Control and CBM-3 conditions (all p ≤ .027). Of note, the CBM-0, CBM-1, and CBM-2 conditions did not differ from each other (all p ≥ .319), and the CBM-Control and CBM-3 conditions did not differ from each other (p = .943). Thus, these results again suggest that only the ‘easier’ versions of the CBM-I training were effective for those high in symptoms. The results reported above for BSIQ, BFNE, and ASIQ did not change (i.e., Condition effects remained non-significant) when analyses were re-run with this subsample.

Figure 2. Effect of Condition on Recognition Ratings (highly socially anxious sample).

Figure 2

Open in a new tab

Note. CBM = Cognitive bias modification. These scores reflect the average of the social-anxiety relevant and foil interpretation options. Higher numbers indicate greater endorsement of positive relative to negative interpretation options.

Discussion

The current study evaluated the efficacy of a brief, Internet-based CBM-I paradigm to determine if it modified social anxiety-relevant interpretations to be more positive/less negative, and if it reduced fear of negative evaluation and anticipatory anxiety. This study also tested if the amount of active generation required to complete the CBM-I task (operationalized by number of letters missing from word fragments) affected the magnitude of CBM-I effects. Finally, this study evaluated the moderating effects of baseline social anxiety symptom severity and interpretation bias on CBM-I’s effects. Overall, results suggested that the online CBM-I task successfully modified interpretations to be more positive/less negative, relative to a control condition, adding to the growing evidence that CBM-I is efficacious at modifying interpretations (see meta-analyses by Hallion & Ruscio, 2011, and Menne-Lothmann et al., 2014), even over the Internet (e.g., in line with Salemink et al., 2014, and Williams et al., 2013). However, in general, training did not affect other outcomes, providing little support for the clinical utility of the training using this very brief form over the Internet.

It is surprising that training effects did not transfer to the BSIQ, an established measure of interpretation bias. This replicates findings from a past single session CBM-I study, which found that CBM-I for anxiety sensitivity modified interpretations measured by the Recognition Rating task, but did not affect panic threat ratings on the BSIQ (Steinman & Teachman, 2010). Perhaps the similar format of the CBM-I training scenarios and the Recognition Rating Task make the Recognition Rating Task more likely to show training effects, or perhaps our decision to split the BSIQ into two versions (BSIQ-A and BSIQ-B, each comprised of only four items) meant the measure no longer adequately sampled the domain, and made it less likely to show change.

Given the brevity of the CBM-I task (only one session of 36 scenarios), it is perhaps unsurprising that training did not affect fear of negative evaluation and anticipated anxiety and performance in a social situation. While it is possible that the brevity of the current intervention led to these null findings, Salemink et al. (2014) had similar results following eight sessions of training 64 scenarios; again, CBM-I modified interpretations, but did not affect psychopathology relative to a control condition. In retrospect, the BFNE may not have been an optimal post-CBM-I outcome measure in the current study. This is because the BFNE requires participants to draw on past social experiences5 when responding to items. Unless participants were given time to have new social interactions following training (but before the assessment), or if instructions had been modified to consider future social situations, it is unlikely that responses to the BFNE would reflect only post-training concerns.

In the current study, we chose to manipulate a lexical aspect of word fragments (e.g., the number of missing letters) to increase desirable difficulty. An alternate way to enhance difficulty would be to manipulate how similar or different the disambiguation of each scenario is to participants’ original (presumably negative) interpretations6. For example, scenarios could vary between being disambiguated to be less negative (easiest condition) to neutral (less easy condition) to slightly positive (harder condition) to very positive (hardest condition). Note that this idea is similar to training done within a single condition by Mathews, Ridgeway, Cook, and Yiend (2007), in which scenarios became gradually more positive over the course of training. Future studies can evaluate if different methods of manipulating difficulty lead to stronger CBM-I effects.

The study’s large sample size allowed for testing of two potential moderators of CBM-I effects: baseline social anxiety symptom severity, and baseline social anxiety-relevant interpretation bias. Notably, baseline interpretation bias moderated CBM-I condition effects on interpretation bias and fear of negative evaluation. Specifically, training led to more positive and less negative interpretations for those who started out with relatively lower levels of interpretation bias, but not for those who started out with relatively higher levels of interpretation bias. This is contrary to some past research, which found CBM-I effects were enhanced for participants with higher levels of baseline bias (e.g., Micco et al., 2014; Salemink & Wiers, 2011). Perhaps the small dose of training in the current study is sufficient to help those with less extreme and less rigid biases, but more training is needed to help those with more intense initial biases. Another possible explanation for the conflicting findings could follow from sample differences across studies that result in discrepant ranges and distributions on the key baseline characteristics (e.g., age, severity). For instance, Micco et al. (2014) used a clinical adolescent and young adult sample, and Salemink and Wiers (2011) used an adolescent analog sample, while we used an unselected adult sample.

Further, baseline social anxiety symptoms moderated CBM-I’s effects on interpretation bias. Specifically, for those high in baseline symptoms, “easier” CBM-I conditions (i.e., those missing zero, one, or two letters from the word fragments) were effective at modifying interpretation bias, while the “harder” CBM-I condition (i.e., the one missing three letters) was not effective at modifying bias. For those low in baseline symptoms, both easy and hard CBM-I conditions effectively modified bias. This finding is further supported by the analyses conducted with our highly socially anxious subsample, which indicated that only the ‘easier’ versions of the positive CBM-I training were effective for those high in symptoms. This suggests that while CBM-I can be effective for both those high and low in symptom severity, different amounts of desirable difficulty may be better for different groups of participants.

Our finding that greatly increasing the “activeness” of training for highly anxious participants may actually reduce the effects of CBM-I is in line with results from Rohrbacher et al. (2014), who found that traditional CBM-I improved mood following training, while a version designed to be more active did not affect mood. Perhaps, for some participants, leaving out three letters made the task too frustrating or anxiety-provoking, thereby precluding CBM-I effects on bias. Alternatively, perhaps the greater task difficulty led participants with high social anxiety to attend more to lexical aspects of the CBM-I task (e.g., figuring out the word fragment), rather than them attending to the more meaningful contingency being established between ambiguity and benign outcomes. Individuals with high (compared to low) social anxiety may be more vulnerable to this effect due to executive functioning difficulties (see Cohen et al., 1996).

A few limitations should be kept in mind. First, the CBM-I used in this study was quite short. This prevents us from knowing whether lack of transfer effects is due to conducting training over the Internet (versus in a lab), or due to an insufficient dose of training. Second, this study used an unselected sample. It is likely that the training did not match the particular concerns of many participants in the study, and may have therefore been unlikely to show effects for those participants. Third, given that this study was conducted over the Internet, it did not include behavioral measures of approach or avoidance. Fourth, our CBM-0 condition was not completely passive; it still included comprehension questions that were designed to reinforce the valenced interpretation. Despite these limitations, these findings provide evidence that it is possible to modify interpretations over the Internet, and suggest that the desirable amount of active generation likely varies given the baseline level of social anxiety symptom severity.

Research Highlights.

  • Evaluated the efficacy of an Internet-based Cognitive Bias Modification paradigm.

  • Findings suggest it is possible to modify interpretations over the Internet.

  • Desirable difficulty of training varied by baseline social anxiety severity.

Acknowledgments

The authors would like to thank Andrew Mathews and Bundy Mackintosh for sharing their study materials. Also, thanks to Fred Smyth, Emily Umansky, and Sriram Natarajan for their assistance with programming this study. Additionally, thanks to Lauren Hallion and members of the Teachman Program for Anxiety and Cognition Treatment (PACT) Lab, especially Nauder Namaky, for their helpful feedback and suggestions. This study was supported by NIA R01AG033033 grant awarded to Bethany Teachman. Note, B. Teachman has a significant financial interest in Project Implicit, Inc., which provided services in support of this project under contract with the University of Virginia.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

1

Although 403 participants gave informed consent, only 350 participants are included in analyses. We excluded participants that dropped out of the study prior to being randomized to a CBM-I condition (n = 53).

2

This study was part of a larger study evaluating effects of CBM-I on information processing bias outcomes. For a full list of measures, please contact the first author.

3

Note that when age is included as a covariate, the main effect of Topic no longer reaches significance (F(1,328) = 2.09, p = .150, ŋp2= .01).

4

This three-way Topic by Condition by Baseline BSIQ interaction reaches significance when Age is included as a covariate (F(4,323) = 2.59, p = .037, ŋp2 = .03).

5

We are thankful to an anonymous reviewer for raising this important point.

6

We are thankful to an anonymous reviewer for this suggestion.

References

  1. Bjork RA. Memory and metamemory considerations in the training of human beings. In: Metcalfe J, editor. Metacognition: Knowing about knowing. MIT Press; Cambridge, MA: 1994. pp. 185–205. S. A. [Google Scholar]
  2. Bjork RA. Assessing our own competence: Heuristics and illusions. In: Gopher D, Koriat A, editors. Attention and performance xvii. Cognitive regulation of performance: Interaction of theory and application. MIT Press; Cambridge, MA: 1999. pp. 435–459. [Google Scholar]
  3. Brown EJ, Turovsky J, Heimberg RG, Juster HR, Brown TA, Barlow DH. Validation of the social interaction anxiety scale and the social phobia scale across the anxiety disorders. Psychological Assessment. 1997;9(1):21–27. doi: 10.1037/1040-3590.9.1.21. [Google Scholar]
  4. Clark DA, Beck AT. Cognitive theory and therapy of anxiety and depression: Convergence with neurobiological findings. Trends in Cognitive Sciences. 2010;14(9):418–424. doi: 10.1016/j.tics.2010.06.007. doi: 10.1016/j.tics.2010.06.007. [DOI] [PubMed] [Google Scholar]
  5. Clark DM, Salkovskis PM, Öst L-G, Breitholtz E, Koehler KA, Westling BE, Gelder M. Misinterpretation of body sensations in panic disorder. Journal of Consulting and Clinical Psychology. 1997;65(2):203–213. doi: 10.1037//0022-006x.65.2.203. doi: 10.1037/0022-006X.65.2.203. [DOI] [PubMed] [Google Scholar]
  6. Cohen LJ, Hollander E, DeCaria CM, Stein DJ, Simeon D, Liebowitz MR, Aronowitz BR. Specificity of neuropsychological impairment in obsessive-compulsive disorder: a comparison with social phobic and normal control subjects. Journal of Neuropsychiatry and Clinical Neuroscience. 1996;8:82–85. doi: 10.1176/jnp.8.1.82. [DOI] [PubMed] [Google Scholar]
  7. Fox E, Mackintosh B, Holmes EA. Travellers' tales in cognitive bias modification research: A commentary on the special issue. Cognitive Therapy and Research. 2014;28:239–247. doi: 10.1007/s10608-014-9604-1. [Google Scholar]
  8. Hallion LS, Ruscio AM. A meta-analysis of the effect of cognitive bias modification on anxiety and depression. Psychological Bulletin. 2011;137(6):940–958. doi: 10.1037/a0024355. doi: 10.1037/a0024355. [DOI] [PubMed] [Google Scholar]
  9. Hertel PT, Mathews A. Cognitive bias modification: Past perspectives, current findings, and future applications. Perspectives on Psychological Science. 2011;6(6):521–536. doi: 10.1177/1745691611421205. doi: 10.1177/1745691611421205. [DOI] [PubMed] [Google Scholar]
  10. Hoppitt L, Mathews A, Yiend J, Mackintosh B. Cognitive mechanisms underlying the emotional effects of bias modification. Applied Cognitive Psychology. 2010;24:312–325. doi: 10.1002/acp.1678. [Google Scholar]
  11. Leary MR. A brief version of the fear of negative evaluation scale. Personality and Social Psychology Bulletin. 1983;9:371–3761. [Google Scholar]
  12. MacLeod C, Mathews A. Cognitive bias modification approaches to anxiety. Annual Review of Clinical Psychology. 2012;8:189–217. doi: 10.1146/annurev-clinpsy-032511-143052. doi: 10.1146/annurev-clinpsy-032511-143052. [DOI] [PubMed] [Google Scholar]
  13. Mathews A, Mackintosh B. Induced emotional interpretation bias and anxiety. Journal of Abnormal Psychology. 2000;109(4):602–615. doi: 10.1037/0021-843X.109.4.602. [PubMed] [Google Scholar]
  14. Matthews A, Mackintosh B. A cognitive model of selective processing in anxiety. Cognitive Therapy and Research. 1998;22(6):539–560. doi: 10.1023/A:1018738019346. [Google Scholar]
  15. Mattick RP, Clarke JC. Development and validation of measures of social phobia scrutiny fear and social interaction anxiety. Behaviour Research and Therapy. 1998;36(4):455–470. doi: 10.1016/s0005-7967(97)10031-6. doi: 10.1016/S0005-7967(97)10031-6. [DOI] [PubMed] [Google Scholar]
  16. Menne-Lothmann C, Viechtbauer W, Höhn P, Kasanova Z, Haller SP, Drukker M, Lau JYF. How to boost positive interpretations? A meta-analysis of the effectiveness of cognitive bias modification for interpretation. PLOS One. 2014;9(6):e100925. doi: 10.1371/journal.pone.0100925. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Micco JA, Henin A, Hirschfeld-Becker DR. Efficacy of interpretation bias modification in depressed adolescents and young adults. Cognitive Therapy and Research. 2014;38(2):89–102. doi: 10.1007/s10608-013-9578-4. doi: 10.1007/s10608-013-9578-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Murphy R, Hirsch CR, Mathews A, Smith K, Clark DM. Facilitating a benign interpretation bias in a high socially anxious population. Behaviour Research and Therapy. 2007 doi: 10.1016/j.brat.2007.01.007. doi: 10.1016/j.brat.2007.01.007. [DOI] [PubMed] [Google Scholar]
  19. Pyc MA, Rawson KA. Testing the retrieval effort hypothesis: Does greater difficulty correctly recalling information lead to higher levels of memory? Journal of Memory and Language. 2009;60:437–449. doi: 10.1016/j.jml.2009.01.004. [Google Scholar]
  20. Rodebaugh TL, Woods CM, Heimberg RG, Liebowitz MR, Schneier FR. The factor structure and screening utility of the social interaction anxiety scale. Psychological Assessment. 2006;18(2):231. doi: 10.1037/1040-3590.18.2.231. [DOI] [PubMed] [Google Scholar]
  21. Rohrbacher H, Blackwell SE, Holmes EA, Reinecke A. Optimizing the ingredients for imagery-based interpretation bias modification for depressed mood: Is self-generation more effective than imagination alone? Journal of Affective Disorders. 2014;152-154:212–218. doi: 10.1016/j.jad.2013.09.013. doi: 10.1016/j.jad.2013.09.013. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Salemink E, Kindt M, Rienties H, van den Hout M. Internet-based cognitive bias modification of interpretations in patients with anxiety disorders: A randomised controlled trial. Journal of Behavior Therapy and Experimental Psychiatry. 2014;45(1):189–195. doi: 10.1016/j.jbtep.2013.10.005. doi: 10.1016/j.jbtep.2013.10.005. [DOI] [PubMed] [Google Scholar]
  23. Salemink E, Wiers RW. Modifying threat-related interpretive bias in adolescents. Journal of Abnormal Child Psychology. 2011;39(7):967–976. doi: 10.1007/s10802-011-9523-5. doi: 10.1007/s10802-011-9523-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Steinman SA. Modifying interpretations among individuals high in anxiety sensitivity. University of Virginia; Charlottesville, VA: 2010. Master's. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Steinman SA, Teachman BA. Modifying interpretations among individuals high in anxiety sensitivity. Journal of Anxiety Disorders. 2010;24(1):71–78. doi: 10.1016/j.janxdis.2009.08.008. doi: 10.1016/j.janxdis.2009.08.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Williams AD, Blackwell SE, Mackenzie A, Holmes EA, Andrews G. Combining imagination and reason in the treatment of depression. Journal of Consulting and Clinical Psychology. 2013;81(5):793–799. doi: 10.1037/a0033247. doi: 10.1037/a0033247. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES