Skip to main content
Lippincott Open Access logoLink to Lippincott Open Access
. 2021 Jan 25;162(8):2237–2245. doi: 10.1097/j.pain.0000000000002211

Order does matter: the combined effects of classical conditioning and verbal suggestions on placebo hypoalgesia and nocebo hyperalgesia

Elżbieta A Bajcar a, Karolina Wiercioch-Kuzianik a, Dominika Farley a, Ewa Buglewicz a, Borysław Paulewicz b, Przemysław Bąbel a,*
PMCID: PMC8280968  PMID: 34256381

Not the type of the procedure (verbal suggestion or conditioning), but the direction of the last-used procedure shapes pain-related expectancies and determines placebo effects.

Keywords: Classical conditioning, Expectancy, Nocebo, Placebo, Verbal suggestions

Abstract

In most experimental studies in which verbal suggestion and classical conditioning are implemented together to induce placebo effects, the former precedes the latter. In naturally occurring situations, however, the information concerning pain does not always precede but often follows the pain experience. Moreover, this information is not always congruent with experience. This study investigates whether the chronology of verbal suggestion and conditioning, as well as their congruence, affects placebo hypoalgesia and nocebo hyperalgesia. The effects induced in 15 groups were compared. The participants in 8 experimental groups were presented with verbal suggestions that were either congruent or incongruent with classical conditioning. The verbal suggestions were provided either before or after conditioning. In 2 other experimental groups, placebo conditioning or nocebo conditioning was implemented without any verbal suggestion; in 2 groups, verbal suggestion of hypoalgesia or hyperalgesia without conditioning was applied. The control groups without any suggestions or conditioning were also included. Placebo hypoalgesia induced by congruent procedures was significantly stronger when the suggestion of hypoalgesia preceded rather than followed conditioning. The order of the congruent procedures did not affect the magnitude of nocebo hyperalgesia. In the groups in which incongruent procedures were implemented, placebo hypoalgesia or nocebo hyperalgesia was in line with the direction of the last-used procedure, regardless of whether it was conditioning or verbal suggestion. The results show that not the type of the procedure (verbal suggestion or conditioning), but the direction of the last-used procedure shapes pain-related expectancies and determines placebo effects.

1. Introduction

A number of studies have focused on the role of direct experiences acquired through classical conditioning and verbal suggestions in shaping placebo hypoalgesia and nocebo hyperalgesia.5 Their results show that classical conditioning alone,69,33 as well as verbal suggestion alone,43 has the potential to induce placebo effects. However, verbal suggestion seems to be crucial if these 2 procedures are combined. Verbal suggestions can enhance conditioned placebo hypoalgesia,48 while classical conditioning may not contribute to verbally induced nocebo hyperalgesia.20 Moreover, information about the lack of association between the previously conditioned placebo stimulus and pain intensity can abolish the effects of learning.6

In most studies which combined verbal suggestion and conditioning, the former usually preceded the latter.2,9,14,19,35,46 In clinical practice, however, the pain experience can occur either before or after the suggestion provided by the clinician. To date, no research has been performed to examine how the order in which verbal suggestion and conditioning are implemented affects the magnitude of placebo effects. We assume that previous pain-related experiences can enhance the credibility of verbally provided pain-related information and therefore make it more effective. Thus, we hypothesized that verbal suggestion that is provided after conditioning would produce stronger placebo effects than verbal suggestion provided before conditioning.

The pain-related information, however, is not always congruent with pain experience. Previous studies imply that verbal suggestion prevails over experience.3,12,13,29 Verbal suggestion provided after conditioning of hypoalgesia could completely abolish the effect of conditioning.12 Furthermore, verbally provided drug-related information was able to reverse drug effects.3,13,29 However, there are also studies showing that verbal suggestion of hypoalgesia can be nullified by the subsequent experience of hyperalgesia.9,49,50 Finally, 1 previous study showed that nocebo hyperalgesia induced by classical conditioning and verbal suggestion can be minimized by a procedure that combines conditioning of hypoalgesia with the suggestion of pain relief.10 The results of the existing studies are inconclusive; thus, the goal of our study was to examine the contribution of incongruent procedures to placebo effects. We hypothesized that verbal suggestion that is incongruent with conditioning would produce placebo effects in accordance with the verbal suggestion, although it would be weaker than the effects produced by congruent procedures.

It is widely assumed that placebo and nocebo effects are mediated by expectancy.12,18,24 Several studies, however, suggest that expectancy is not always involved in placebo effects induced by classical conditioning.69 The last goal of the current study was to investigate the role of expectancy in shaping placebo effects induced by pure conditioning, verbal suggestion, and procedures that combine verbal suggestion with congruent and incongruent conditioning. We hypothesized that expectancies are involved in placebo effects induced by verbal suggestion with or without classical conditioning.

To fulfill the study aims, we conducted the very first experimental study comparing placebo effects induced by classical conditioning, verbal suggestions, and procedures that mix all possible combinations of suggestion and conditioning. This is also the first study investigating extensively the mechanisms that underlie placebo effects induced by congruent and incongruent procedures.

2. Materials and methods

2.1. Design

In the study, 12 experimental groups were tested. In 8 of 12 experimental groups, both verbal suggestions and classical conditioning were used. In 4 experimental groups, verbal suggestions congruent with classical conditioning were applied; in 4 other groups, verbal suggestions were incongruent with classical conditioning and were also applied either before or after classical conditioning. In the remaining 4 experimental groups, only 1 manipulation was used: either verbal suggestions (of hypoalgesia or hyperalgesia, depending on the group) or classical conditioning (placebo or nocebo, depending on the group). Moreover, 3 control groups were tested. One of them was a natural history group, which served as a control group for the experimental groups with suggestions only. The other 2 control groups (placebo control and nocebo control) served as control groups for experimental groups with placebo and nocebo conditioning, respectively (see Fig. 1 for the experimental design). Designing 3 control groups enabled us to mimic the experience from experimental groups more precisely and prevent potential habituation and sensitization bias.

Figure 1.

Figure 1.

Study design and experimental procedure. A horizontal line symbolizes no manipulation.

2.2. Participants

A total of 419 healthy volunteers including 232 women (55.37%) aged 23.29 ± 3.58 years participated in the study. Participants were recruited through announcements on classified advertisement websites and social media. They received financial compensation for their participation. Participants were randomly assigned to one of the 12 experimental and 3 control groups. The randomization procedure was performed by a computer program, which would draw the number of the group for each consecutive participant. The number of participants in each group and the characteristics of each group are presented in Table 1.

Table 1.

Characteristics of participants in each experimental group and in total: mean values and SDs.

Group N Age [yo] Height [cm] Weight [kg] TT [mA] PT [mA] FPQ
General FSP FMP FM/DP
Group 1 28 23.39 ± 2.81 171.82 ± 9.96 72.11 ± 17.67 4.67 ± 2.28 17.37 ± 10.00 70.36 ± 19.93 29.36 ± 8.38 17.25 ± 6.52 23.75 ± 8.39
Group 2 28 22.96 ± 2.63 172.79 ± 7.71 65.68 ± 11.83 4.66 ± 1.52 20.05 ± 8.21 64.39 ± 20.93 27.82 ± 8.00 15.79 ± 5.96 20.79 ± 7.81
Group 3 28 23.50 ± 3.60 172.82 ± 09.05 67.07 ± 13.13 4.43 ± 1.30 20.25 ± 12.96 66.64 ± 16.77 27.18 ± 08.05 17.11 ± 5.66 22.36 ± 7.39
Group 4 28 22.68 ± 3.39 171.79 ± 9.92 64.64 ± 14.08 5.29 ± 2.59 18.95 ± 15.53 70.14 ± 19.89 29.93 ± 8.78 16.82 ± 6.27 23.39 ± 7.46
Group 5 28 23.82 ± 3.43 172.89 ± 11.10 67.54 ± 15.14 4.31 ± 1.84 19.87 ± 12.81 69.64 ± 16.72 28.79 ± 8.10 17.29 ± 5.21 23.57 ± 6.45
Group 6 28 23.29 ± 3.34 173.57 ± 9.35 68.11 ± 14.69 4.73 ± 1.83 19.41 ± 10.46 65.86 ± 16.17 28.39 ± 8.13 15.86 ± 4.32 21.61 ± 6.11
Group 7 28 23.62 ± 3.50 173.18 ± 10.25 66.86 ± 11.02 4.52 ± 1.82 19.63 ± 11.31 79.25 ± 17.41 32.50 ± 07.01 19.57 ± 5.80 27.18 ± 7.85
Group 8 27 24.19 ± 3.68 166.81 ± 9.76 66.81 ± 11.72 5.20 ± 2.41 17.72 ± 12.32 67.37 ± 17.89 28.52 ± 8.22 16.23 ± 4.72 22.56 ± 8.40
Group 9 28 22.68 ± 2.28 166 ± 8.02 66.82 ± 14.47 3.99 ± 1.17 14.80 ± 8.71 73.14 ± 16.81 30.21 ± 7.45 17.96 ± 5.39 24.96 ± 7.38
Group 10 28 23.71 ± 3.34 165.96 ± 9.54 65.93 ± 14.43 4.97 ± 3.31 18.83 ± 12.28 69.50 ± 22.69 28.14 ± 10.06 16.21 ± 5.51 25.14 ± 10.09
Group 11 28 23.96 ± 3.25 172.25 ± 10.56 70.39 ± 16.94 4.59 ± 1.82 20.93 ± 14.58 69.07 ± 20.23 28.93 ± 8.66 17.64 ± 6.48 22.5 ± 7.00
Group 12 28 22.68 ± 3.13 170.50 ± 8.29 63.18 ± 15.89 4.96 ± 1.89 20.96 ± 11.50 64.54 ± 21.97 28.00 ± 11.26 15.75 ± 6.30 20.79 ± 7.52
Group 13 28 25.04 ± 4.48 172.61 ± 10.19 67.39 ± 17.62 4.40 ± 1.51 19.66 ± 13.24 75.07 ± 18.24 31.36 ± 7.82 18.14 ± 6.59 25.57 ± 7.35
Group 14 28 24.00 ± 3.72 172.75 ± 8.80 69.86 ± 14.18 4.95 ± 3.50 26.26 ± 19.15 74.14 ± 19.42 30.82 ± 7.36 18.86 ± 7.14 24.46 ± 8.15
Group 15 28 23.75 ± 4.23 173.14 ± 11.07 68.43 ± 14.80 4.66 ± 2.68 22.86 ± 15.67 64.50 ± 6.00 27.43 ± 11.20 15.46 ± 6.94 21.61 ± 9.80
All 419 23.55 ± 3.43 171.27 ± 9.49 67.39 ± 14.50 4.69 ± 2.19 19.84 ± 12.95 69.58 ± 19.73 29.16 ± 8.72 17.07 ± 5.00 23.35 ± 8.02

FM/DP, Fear of Medical/Dental Pain; FMP, Fear of Minor Pain; FPQ, Fear of Pain Questionnaire; FSP, Fear of Severe Pain; PT, pain threshold; TT, tactile threshold.

The inclusion criteria were age between 18 and 35 years, no physical or mental illness, and no previous experience with pain studies. Only participants who were abstaining from drugs, alcohol, or stimulants around the time of the study and had no pain on the day of the study could participate. In addition, participants completed the Fear of Pain Questionnaire (FPQ-III)40 to control for potential differences in fear of pain between the groups which would possibly confound the results. Participants gave their informed, written consent to participate in the study, and they were told that its aim was to investigate people's reactions to electrical stimulation. They were debriefed about the actual aim of the study after it was completed. Participants were also informed that they could withdraw their consent at any time without providing a reason. The study protocol was approved by the Research Ethics Committee at the Institute of Psychology, Jagiellonian University, Kraków, Poland, and preregistered on the Open Science Framework webpage: https://osf.io/4vnk2.

2.3. Sample size

Sample size was estimated based on the effect sizes (∼d = 0.70) from previous studies6,7 using G*Power 3.1 software.25 The sample size was planned for the independent Student t test. It was estimated that a minimum sample of 19 subjects per group would be required (alpha = 0.05, power of 80%). Since other statistical analyses were also planned (analysis of variance [ANOVA], linear regression), it was decided that data collection would conclude when the sample size for each of the experimental groups reached the number of minimum 28 participants (total sample size of 419 participants).

2.4. Stimuli

2.4.1. Pain stimuli

Electrocutaneous pain stimuli were delivered to the inner side of the nondominant forearm of the participant through 2 durable stainless-steel disk electrodes (diameter 8 mm, with 30-mm spacing). The electrocutaneous stimuli were square pulses of 200-μs duration, delivered by a constant current high voltage stimulator (Digitimer, Welwyn Garden City, England, Model DS7AH).

The intensity of the electrocutaneous stimuli was set individually for each participant based on the results of the calibration procedure (see Procedure). Three intensities of pain stimuli were calculated based on the calibration procedure: low, high, and moderate. The stimuli were calculated in mA as follows, where T is averaged pain threshold and t is averaged tactile threshold: (1) moderate-intensity stimulus = 1.5 × T; (2) low-intensity stimulus = 0.8 × T + 0.2 × t; and (3) high-intensity stimulus = 2.2 × T − 0.2 × t. The reason for choosing this particular method of calculation of the intensity of pain stimuli was the need to ascertain that the intensity of the placebo stimulus would always be above t and below T and that the intensity of the nocebo stimulus would be at the same distance regarding the control stimulus (1.5 T) as the placebo stimulus. A similar method of calculation has been used and described previously.6,7

2.4.2. Visual stimuli

Low-, high-, and moderate-intensity pain stimuli were preceded by the presentation of color stimuli (orange or blue) serving as either placebo/nocebo or control stimuli on a computer screen (17 inches, resolution 1280 × 1024) placed in front of the participant at a distance of approximately 50 cm. The colors were chosen because there is evidence suggesting that these colors do not influence the intensity of experienced pain.52 The color slides (orange or blue) were counterbalanced within groups, that is, in each of the experimental group, for 50% of the participants, the orange color served as placebo/nocebo stimulus while the blue color as control stimulus, and for 50% of the participants, the blue color served as placebo/nocebo stimulus, while the orange color as a control stimulus. All slides were presented according to a predetermined pseudorandom sequence in full-screen mode.

2.4.3. Sham device

An electrostimulating TENS/EMS unit (TensCare Ltd, United Kingdom) was introduced to participants as an additional device that would make pain intensity lower (in the groups where placebo suggestion was used) or higher (in the groups where nocebo suggestion was used) after presentation of one of the 2 colors. The TENS unit was turned on briefly in these groups to show the participants that the device was indeed operational.

The experimental procedure was fully automatized with PsychoPy2 software.42 This software integrated stimuli application and data collection in real time.

2.5. Measures

Pain intensity and pain expectancy were rated on an 11-point numeric rating scale ranging from 0 = “no pain” to 10 = “the most intense pain that is tolerable.”26,32

2.6. Trial

There were 96 trials in total (72 in the manipulation phase and 24 in the testing phase). Each trial consisted of: (1) a visual stimulus displayed for 9 seconds; (2) a pain stimulus applied 7 seconds after the beginning of the trial with the color slide still visible. Thirty trials were preceded by expectancy ratings; another 30 were followed by pain intensity ratings, and the rest (36) were presented with no accompanying ratings. The order of all trials in both the manipulation and testing phase was pseudorandomized. All the numeric rating scales were presented on a color slide. The color of the slide matched the pain stimulus that was being rated.

2.7. Procedure

To become accustomed to electrocutaneous stimulation, each participant received the same set of 10 electrical stimuli, ranging from 5 mA to 50 mA, delivered every 5 seconds. The intensity of electrocutaneous pain stimuli was determined in the calibration procedure, which was based on the method of limits and used in previous studies.1,6,7,9,17,19 Two ascending series of electrocutaneous stimuli in increments of 1 mA, starting from 0 mA, with an interstimulus interval of 5 seconds, were applied. Participants reported the first tactile and the first painful sensation. The obtained values were then averaged separately for tactile (t) and pain thresholds (T) and were used to calculate stimuli at 3 levels of intensity: moderate, low, and high (see Stimuli).

2.7.1. Manipulation phase

There were 72 trials in the manipulation phase: 18 trials were preceded by expectancy ratings; 18 trials were followed by pain intensity ratings, and 36 trials were not accompanied by any ratings. The trials were divided into 3 blocks with 2-minute breaks between them.

2.7.2. Experimental conditions

There were 2 main experimental conditions in the experimental groups: (1) conditioning and (2) verbal suggestion. The conditioning consisted of a manipulation in which one of the 2 presented colors was paired with either a low-intensity pain stimulus (in placebo conditioning) or a high-intensity pain stimulus (in nocebo conditioning), and the other color was paired with a moderate-intensity pain stimulus (control stimulus). Placebo or nocebo verbal suggestions were used. The participants were told that one of the 2 presented colors would predict less intense pain (placebo suggestion) or more intense pain (nocebo suggestion) than the other color because of the activation of the sham device. Depending on the experimental group, the suggestion was presented before, after, or instead of conditioning (for the design of the study and manipulations used in each of the groups, see Fig. 1). In the placebo and nocebo control groups, participants were presented with 2 pain stimuli: low-intensity and moderate-intensity (placebo control group) or high-intensity and moderate-intensity (nocebo control group) preceded by colors presented in a noncontingent manner. In the natural history control group, there was no manipulation phase.

2.7.3. Testing phase

The testing phase consisted of 24 trials: 12 trials were preceded by expectancy ratings, and 12 trials were followed by pain intensity ratings.

2.8. Statistical analysis

Differences between the groups in age, height, body mass, tactile threshold, pain threshold, and FPQ-III scores were analyzed by means of 1-way ANOVA with the experimental group as a between-subject factor. As a form of manipulation check, participants were asked what, in their opinion, was the purpose of the experiment and whether there was any association between pain intensity and the colors. Participants who figured out the actual aim of the study were excluded from the main analyses. The exclusion of participants, who figured out the actual aim of the study, was aimed to lower the risk of participant bias and demand characteristics.39,51 As the purpose of this study was to investigate the effects of verbal suggestion, conditioning, and combined manipulations, we wanted to prevent the potential influence of additional participants' expectations regarding the purpose of the experiment.

The main analyses testing the induction of placebo hypoalgesia or nocebo hyperalgesia were performed separately on pain intensity and expectancy ratings obtained from the testing phase. The R Statistical Environment (R Core Team, 2018) was used for data analysis and plotting. Linear mixed models were used instead of preplanned ANOVA analyses because of the complexity of the model and the obtained results. Linear mixed models were fitted using the lme4 package.11 The lmerTest package36 was used to obtain approximate degrees of freedom and significance values. To improve the readability of the results, we present the regression tables for the models in which the effect of stimulus type (placebo/nocebo or control) was nested within groups. Separate reparametrized models were fitted to test the hypotheses concerning intergroup differences. Because all the additional tests were either planned or were not independent, we did not use Bonferroni correction. The significance level was set at P ≤ 0.05.

3. Results

There were 31 participants excluded from the analyses (group 1: 1; group 2: 2; group 3: 1; group 4: 3; group 5: 4; group 6: 2; group 7: 1; group 8: 1; group 9: 2; group 10: 6; group 11: 1; group 13: 3; group 14: 1; and group 15: 3). There were no significant differences in participants' characteristics in the final sample among all the experimental and control groups (2-sided tests): age (F(14, 418) = 0.70; P = 0.78; η2 = 0.02); height (F(14, 418) = 0.18; P = 1; η2 = 0.01); body mass (F(14, 418) = 0.42; P = 0.97; η2 = 0.01); tactile threshold (F(14, 418) = 0.68; P = 0.79; η2 = 0.02); pain threshold (F(14, 418) = 1.09; P = 0.37; η2 = 0.04); and FPQ (F(14, 418) = 1.48; P = 0.12; η2 = 0.05) (mean values and SDs are presented in Table 1).

There are 3 different control groups in our study, which complicates some of the analyses and may unnecessarily lower the precision of the estimates of the possibly null effects in the control groups. That is why we have tested if there is any reason to treat the 3 control conditions as separate. In the main analyses, the models in which all 3 control groups (random placebo, random nocebo, and natural history) were analyzed separately did not fit significantly better than the simpler models with all the control groups combined into one (pain intensity ratings: χ2(4) = 3.39, P = 0.49; expected pain intensity ratings: χ2(4) = 5.38, P = 0.25). For that reason, we followed the preregistration plan and all the main analyses were performed on the combined control groups (named; control group), which improved the precision of the stimulus-type effect estimate in the control groups.

To test the effect of group on pain intensity ratings, we fitted a linear mixed model with fixed effects of group, stimulus type (placebo/nocebo and control trials), their interaction, and participants' specific random effects of stimulus type and random intercept. Mean values and SDs of pain intensity ratings are presented in Table 2.

Table 2.

Descriptive statistics for pain intensity in testing phase for each group and in total: mean values and SDs.

Group Placebo/nocebo trials Control trials Difference
Group 1 3.15 ± 2.48 3.20 ± 2.56 −0.04 ± 0.66
Group 2 2.83 ± 1.80 2.73 ± 1.82 0.10 ± 0.39
Group 3 2.88 ± 1.78 3.03 ± 1.81 −0.15 ± 0.49
Group 4 3.11 ± 2.04 2.78 ± 1.78 0.33 ± 0.58
Group 5 2.96 ± 1.46 3.20 ± 1.62 −0.23 ± 0.60
Group 6 3.42 ± 1.93 3.17 ± 1.78 0.25 ± 0.51
Group 7 3.77 ± 1.79 3.60 ± 1.75 0.17 ± 0.46
Group 8 3.12 ± 2.00 3.36 ± 2.17 −0.24 ± 0.33
Group 9 2.49 ± 1.98 3.11 ± 2.31 −0.62 ± 0.86
Group 10 2.98 ± 2.27 2.71 ± 2.16 0.27 ± 0.68
Group 11 3.92 ± 2.20 3.66 ± 2.18 0.26 ± 0.57
Group 12 3.54 ± 1.88 3.73 ± 2.00 −0.19 ± 0.52
Group 13 3.37 ± 2.27 3.30 ± 2.33 0.07 ± 0.50
Group 14 4.11 ± 2.36 4.04 ± 2.38 0.07 ± 0.41
Group 15 3.74 ± 2.57 3.62 ± 2.54 0.12 ± 0.42
All 3.29 ± 2.09 3.28 ± 2.10 0.01 ± 0.59

We found significant effects in all experimental groups except the groups with conditioning alone (groups 1 and 2, respectively: t(406) = −0.20, P = 0.34; t(406) = 0.49, P = 0.16) and placebo suggestion alone (group 3, t(406) = −0.75, P = 0.07). There was no effect in the control group (t(406) = −0.02, P = 0.49) (Fig. 2). Placebo hypoalgesia was induced in groups with (1-sided test) placebo suggestion after placebo conditioning (group 5), t(406) = −1.12, P = 0.01; placebo suggestion after nocebo conditioning (group 8), t(406) = −1.14, P = 0.01; placebo suggestion before placebo conditioning (group 9), t(406) = −3.00, P < 0.001; and nocebo suggestion before placebo conditioning (group 12), t(406) = −0.92, P = 0.03. Nocebo hyperalgesia was induced in groups with (1-sided test) nocebo suggestion (group 4), t(406) = 1.58, P = 0.001; nocebo suggestion after nocebo conditioning (group 6), t(406) = 1.2, P = 0.008; nocebo suggestion after placebo conditioning (group 7), t(406) = 0.84, P = 0.05; nocebo suggestion before nocebo conditioning (group 10), t(406) = 1.30, P = 0.005; and placebo suggestion before nocebo conditioning (group 11), t(406) = 1.24, P = 0.007.

Figure 2.

Figure 2.

Within-group comparisons of pain intensity ratings. Black bars represent significant within-group effects with error bars indicating SDs. Notes. * P < 0.05; ** P < 0.01; *** P < 0.001; Ctrl—control group (groups 13-15); 1—placebo conditioning; 2—nocebo conditioning; 3—placebo suggestion; 4—nocebo suggestion; 5—placebo suggestion after placebo conditioning; 6—nocebo suggestion after nocebo conditioning; 7—nocebo suggestion after placebo conditioning; 8—placebo suggestion after nocebo conditioning; 9—placebo suggestion before placebo conditioning; 10—nocebo suggestion before nocebo conditioning; 11—placebo suggestion before nocebo conditioning; and 12—nocebo suggestion before placebo conditioning.

The same model was reparametrized to test the hypotheses regarding the differences between the groups in pain intensity. We found a significant difference (1-sided test) between group 5 (placebo suggestion after placebo conditioning) and 9 (placebo suggestion before placebo conditioning) in favor of the latter (t(406) = −2.65, P = 0.008), but there was no difference between groups 6 (nocebo suggestion after nocebo conditioning) and 10 (nocebo suggestion before nocebo conditioning). There was also no significant difference in the obtained effects between group 4 (nocebo suggestion) and group 6 (nocebo suggestion after nocebo conditioning, t(406) = −0.53, P = 0.60) or group 10 (nocebo suggestion before nocebo conditioning, t(406) = −0.41, P = 0.68).

Furthermore, to test the effect of group on expectancy ratings, we also fitted a linear mixed model with fixed effects of group, stimulus type (placebo/nocebo and control trials), their interaction, and participants' specific random effects of stimulus type and random intercept. There was no significant effect in the control group (t(406) = 0.28, P = 0.30). We found significant effects in all experimental groups (Fig. 3). We found statistically significant differences in expectancy between placebo/nocebo and control trials in groups (1-sided tests): placebo conditioning (group 1), t(406) = −0.82, P = 0.05; placebo suggestion (group 3), t(406) = −1.60, P = 0.001; placebo suggestion after placebo conditioning (group 5), t(406) = −2.32, P < 0.001; placebo suggestion after nocebo conditioning (group 8), t(406) = −2.16, P < 0.001; placebo suggestion before placebo conditioning (group 9), t(406) = −4.04, P < 0.001; and nocebo suggestion before placebo conditioning (group 12), t(406) = −1.56, P = 0.001. Also, statistically significant differences were found in groups with (1-sided test) nocebo conditioning (group 2), t(406) = 0.94, P = 0.03; nocebo suggestion (group 4), t(406) = 2.13, P < 0.001; nocebo suggestion after nocebo conditioning (group 6), t(406) = 2.02, P < 0.001; nocebo suggestion after placebo conditioning (group 7), t(406) = 2.68, P < 0.001; nocebo suggestion before nocebo conditioning (group 10), t(406) = 2.48, P < 0.001; and placebo suggestion before nocebo conditioning (group 11), t(406) = 1.16, P = 0.01.

Figure 3.

Figure 3.

Within-group comparisons of expected pain intensity ratings. Black bars represent significant within-group effects with error bars indicating SDs. Notes. * P < 0.05; *** P < 0.001; Ctrl—control group (groups 13-15); 1—placebo conditioning; 2—nocebo conditioning; 3—placebo suggestion; 4—nocebo suggestion; 5—placebo suggestion after placebo conditioning; 6—nocebo suggestion after nocebo conditioning; 7—nocebo suggestion after placebo conditioning; 8—placebo suggestion after nocebo conditioning; 9—placebo suggestion before placebo conditioning; 10—nocebo suggestion before nocebo conditioning; 11—placebo suggestion before nocebo conditioning; and 12—nocebo suggestion before placebo conditioning.

Finally, to determine whether the effect of conditioning on pain intensity ratings could be mostly due to the effect of expectancy, we fitted a linear mixed model to pain intensity ratings data with fixed effects of group, stimulus type and expectancy ratings, and the interactions of group with stimulus type (placebo/nocebo and control trials) and group with expectancy ratings, ie, we have accounted for a possibly different effect of expectancy within each group. As in the previous analyses, the model included participants' specific random effects of stimulus type and random intercept. When the expected pain intensity was included as a predictor, none of the previously observed placebo hypoalgesia or nocebo hyperalgesia was significant. Moreover, expectancy ratings were a significant predictor of pain intensity in every group except group 1 (placebo conditioning).

4. Discussion

This is the only study to date in which suggestion of hypoalgesia and hyperalgesia, placebo and nocebo conditioning, and all possible combinations of these procedures were used to induce placebo effects. Such an exhaustive experimental design made it possible to clarify the contribution of verbal suggestions and classical conditioning to placebo effects.

The most significant finding of this study is that the order of the procedures used to induce placebo effects is essential for these effects. Previous studies consistently showed that verbal suggestion of hypoalgesia implemented together with conditioning could significantly enhance the magnitude of placebo hypoalgesia.48 In most of these studies, however, the verbal suggestion was presented before conditioning. Our study, which manipulated the order of these 2 procedures, shows that placebo hypoalgesia is significantly stronger when the suggestion of hypoalgesia precedes rather than follows conditioning. The order of the procedures did not affect the magnitude of nocebo hyperalgesia. Moreover, the magnitude of nocebo hyperalgesia induced by these combined procedures did not differ significantly from that induced by suggestion of hyperalgesia alone.

The result showing that nocebo hyperalgesia was insensitive to the order effect suggests that placebo and nocebo effects are shaped and maintained by different mechanisms. This corresponds to the findings from the functional magnetic resonance imaging study showing that different brain regions are activated when the placebo or nocebo effect is induced.30 Moreover, in our study, verbal suggestion was sufficient to induce nocebo hyperalgesia but played a minor role in shaping placebo hypoalgesia—to induce the placebo effect, the combined procedures were needed. This is consistent with the previous study results, in which verbal suggestion effectively turned tactile and low-pain stimuli into the high-pain stimuli but was ineffective in eliciting the placebo effect.20 Our study results provide further evidence that the mechanisms underlying the placebo and nocebo effect differ.

Thus, the obtained results did not confirm our hypothesis that verbal suggestion would be more effective if reinforced by previous experience. However, they demonstrated that nocebo hyperalgesia and placebo hypoalgesia are not “2 sides of the same coin.” These findings also have important clinical implications: They show that the suggestion of hypoalgesia should be provided to the patient as soon as possible and preferably before the pain-related experience to obtain the strongest placebo effect.

However, the strongest evidence that the order of the procedures is of fundamental importance is provided by results obtained in groups in which verbal suggestions incongruent with conditioning were used. Incongruent procedures were used in a few previous studies to induce placebo effects.9,10,12,49,50 Their results were, however, equivocal. One of those studies suggested that verbal suggestion could nullify the effect of classical conditioning,12 whereas the others implied that conditioning could reverse the effect of verbal suggestions.9,49,50 Our study showed that the effects produced by incongruent procedures were determined by the direction of the last-used procedure (ie, placebo or nocebo) rather than by the type of the procedure (ie, verbal suggestion or conditioning). This result allows for reinterpretation of the findings of previous studies. In those studies, as in the current study, the obtained effect was in line with the last-used procedure, regardless of whether it was verbal suggestion,12 classical conditioning,9,49,50 or classical conditioning reinforced by verbal suggestion.10

Our findings did not confirm the hypothesis that suggestion prevails over conditioning; however, they provided further evidence that the order of the procedures is crucial for shaping placebo effects. These findings also have practical implications. They show that positive suggestion can minimize the negative effects of the previous pain-related experiences. They also suggest that providing patients with a positive experience could alleviate the adverse effects of information obtained from other patients or media.

Our study significantly expands the knowledge on the role of verbal suggestion and classical conditioning in shaping placebo effects. It should be noted here, however, that from a theoretical perspective, classical conditioning depends not only on the simple contiguity between the conditioned stimulus and unconditioned stimulus,45 and the strength of learning outcomes may be influenced by other factors, that is, number of learning trails19 or the conditioning schedule.4,16 It seems possible that placebo effects acquired by longer conditioning training or by conditioning that involves partial reinforcement may not be inhibited by the following procedure as easily as the effects produced by shorter training or continuous reinforcement. Therefore, future research should investigate this issue. Because conscious processes are involved in conditioning,38 it seems also reasonable to explore how these differently designed conditioning procedures shape them.

The expectancies were induced in all groups. When the expectancies were controlled for, however, none of the observed placebo effects were significant. These results support the view that expectancies are central to inducing placebo effects.12,18,24 Interestingly, expectancies did not always lead to a change in pain sensation; nocebo conditioning alone and placebo suggestion alone, although they triggered relevant expectancies, turned out to be insufficient to change pain perception and induce the placebo effect. This effect is in line with our previous study in which conditioning had an effect on expectancy but expectancy did not predict placebo hypoalgesia.6 It is also in line with a previous study in which classical conditioning enhanced the expectancies induced by verbal suggestions but did not enhance placebo hypoalgesia.22 It is suggested, however, that not only expectancy but also affective states can play a role in shaping placebo effects.27,31 This is evidenced also by the study, in which the conditioning procedure not only triggered expectancy but also affected the level of fear of pain which, in contrast to expectancy, predicted placebo hypoalgesia.6 Future research should answer the question on the role of affective states in shaping placebo effects to extend current theoretical approaches that emphasize the role of expectancies in this process.18,24

In the case of incongruent procedures, expectancies were in line with the last-used procedure. This result is in line with the only study to date in which incongruent procedures were used and expectancies were measured.9 Thus, our findings confirm the hypothesis on the role of verbal suggestion in shaping expectancies. Verbal suggestion alone and procedures including it produce placebo effects through expectancies, but these are not always congruent with those verbally suggested. Moreover, verbally induced expectancies may not always produce placebo effects in pain.

It should be stated here, however, that the findings on the role of expectancies in shaping placebo hypoalgesia may not apply to chronic pain patients.34 Healthy volunteers taking part in experimental studies are exposed to calibrated pain with which they had no previous experiences. Moreover, they repeatedly experience relief in pain, often reinforced by the suggestion that a placebo will lower their pain. These experimental manipulations can easily create positive expectancies. The chronic pain patients' perception of pain is biased by previous negative pain experiences. Moreover, chronic pain is accompanied by changes in structural and functional brain architecture which can influence pain processing. Thus, although conscious expectancies can influence the placebo effect in acute pain, unconscious processes may contribute to the placebo effect in chronic pain.

An important advantage of our study is that it includes all possible combinations of the procedures most commonly used to induce placebo effects. It allows not only to determine their contribution to placebo effects but also to elucidate the interplay between them. Moreover, the sample size of our study was large (N = 419) compared with the sample sizes of previous studies on the mechanisms of placebo effects.10,15,28,44

Some limitations of the study should be also noted. In this study, only healthy volunteers participated; therefore, the results should be generalized to the clinical population with caution. It should also be noted that pain induced by electrocutaneous stimulation differs from clinical pain therefore the obtained results may not be directly transferable to clinical pain. Moreover, all the study variables relied on self-reports. However, this is also the case in other studies on the mechanisms of placebo effects because there are currently no validated objective markers of nociception or pain.21,41 What is more, included participants were aged 18 to 35 years. This criterion, also used in other studies,17,23 was applied owing to age-related differences in pain perception.37

It should be noted that placebo effects obtained in this study were rather low. However, it is also the case in other experimental studies on healthy volunteers in which placebos that are not connoted medically (ie, colors, geometrical shapes) were used.19,33,47 These types of placebos do not induce large placebo effects, but neither do they produce expectancies independent of experimental manipulation. Thus, they allow for disentangling preconditioned and experimentally induced expectancies to investigate the latter. As a result, it is possible to understand the role of conscious processes in shaping placebo effects. We do believe that the findings of this basic science study on healthy volunteers would help to understand the mechanisms of placebo effects in pain and to design effective interventions for pain patients.

The practical applications of our research should be emphasized. Our research shows that the order in which the verbal suggestion and classical conditioning occur affects the magnitude of placebo hypoalgesia. Thus, to enhance the placebo effect, the experience of hypoalgesia should be preceded by a relevant suggestion. Furthermore, it seems that the direction of the last-used procedure determines what effect (placebo or nocebo) will be produced. Therefore, it is important to ensure that the patient's final experience is positive.

Conflict of interest statement

The authors have no conflicts of interest to declare.

Acknowledgments

The study was funded by the National Science Centre in Poland under the grant no. 2014/14/E/HS6/00415. The open-access publication fee was funded by the Priority Research Area Society of the Future under the program “Excellence Initiative – Research University” at the Jagiellonian University.

Footnotes

Sponsorships or competing interests that may be relevant to content are disclosed at the end of this article.

Contributor Information

Elżbieta A. Bajcar, Email: elzbieta.bajcar@uj.edu.pl.

Karolina Wiercioch-Kuzianik, Email: karolina.wiercioch@doctoral.uj.edu.pl.

Dominika Farley, Email: dominika.farley@uj.edu.pl.

Ewa Buglewicz, Email: ewa.buglewicz@gmail.com.

Borysław Paulewicz, Email: bpaulewicz@swps.edu.pl.

References

  • [1].Adamczyk WM, Wiercioch-Kuzianik K, Bajcar EA, Bąbel P. Rewarded placebo analgesia: a new mechanism of placebo effects based on operant conditioning. Eur J Pain 2019;23:923–35. [DOI] [PubMed] [Google Scholar]
  • [2].Amanzio M, Benedetti F. Neuropharmacological dissection of placebo analgesia: expectation-activated opioid systems versus conditioning-activated specific subsystems. J Neurosci 1999;19:484–94. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [3].Aslaksen PM, Zwarg ML, Eilertsen H-IH, Gorecka MM, Bjørkedal E. Opposite effects of the same drug: reversal of topical analgesia by nocebo information. PAIN 2015;156:39–46. [DOI] [PubMed] [Google Scholar]
  • [4].Au Yeung ST, Colagiuri B, Lovibond PF, Colloca L. Partial reinforcement, extinction, and placebo analgesia. PAIN 2014;155:1110–17. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [5].Bąbel P. Classical conditioning as a distinct mechanism of placebo effects. Front Psychiatry 2019;449. doi: 10.3389/fpsyt.2019.00449. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [6].Bąbel P, Adamczyk W, Świder K, Bajcar EA, Kicman P, Lisińska N. How classical conditioning shapes placebo analgesia: hidden versus open conditioning. Pain Med 2018;19:1156–69. [DOI] [PubMed] [Google Scholar]
  • [7].Bąbel P, Bajcar EA, Adamczyk W, Kicman P, Lisińska N, Świder K, Colloca L. Classical conditioning without verbal suggestions elicits placebo analgesia and nocebo hyperalgesia. PLoS One 2017;12:e0181856. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [8].Bajcar EA, Adamczyk WM, Wiercioch-Kuzianik K, Bąbel P. Nocebo hyperalgesia can be induced by classical conditioning without involvement of expectancy. PLoS One 2020;15:e0232108. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [9].Bajcar EA, Wiercioch-Kuzianik K, Adamczyk WM, Bąbel P. To experience or to Be informed? Classical conditioning induces nocebo hyperalgesia even when placebo analgesia is verbally suggested-results of a preliminary study. Pain Med 2020;21:548–60. [DOI] [PubMed] [Google Scholar]
  • [10].Bartels DJP, van Laarhoven AIM, Stroo M, Hijne K, Peerdeman KJ, Donders ART, van de Kerkhof PCM, Evers AWM. Minimizing nocebo effects by conditioning with verbal suggestion: a randomized clinical trial in healthy humans. PLoS One 2017;9. doi: 10.1371/journal.pone.0182959. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [11].Bates D, Mächler M, Bolker B, Walker S. Fitting linear mixed-effects models using lme4. J Stat Softw 2015;67:1–48. [Google Scholar]
  • [12].Benedetti F, Pollo A, Lopiano L, Lanotte M, Vighetti S, Rainero I. Conscious expectation and unconscious conditioning in analgesic, motor, and hormonal placebo/nocebo responses. J Neurosci 2003;23:4315–23. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [13].Bingel U, Wanigasekera V, Wiech K, Ni Mhuircheartaigh R, Lee MC, Ploner M, Tracey I. The effect of treatment expectation on drug efficacy: imaging the analgesic benefit of the opioid remifentanil. Sci Transl Med 2011;3:70ra14. [DOI] [PubMed] [Google Scholar]
  • [14].Carlino E, Torta DME, Piedimonte A, Frisaldi E, Vighetti S, Benedetti F. Role of explicit verbal information in conditioned analgesia. Eur J Pain 2015;19:546–53. [DOI] [PubMed] [Google Scholar]
  • [15].Colagiuri B, Quinn VF. Autonomic arousal as a mechanism of the persistence of nocebo hyperalgesia. J Pain 2018;19:476–86. [DOI] [PubMed] [Google Scholar]
  • [16].Colagiuri B, Quinn VF, Colloca L. Nocebo hyperalgesia, partial reinforcement, and extinction. J Pain 2015;16:995–1004. [DOI] [PubMed] [Google Scholar]
  • [17].Colloca L, Benedetti F. How prior experience shapes placebo analgesia. PAIN 2006;124:126–33. [DOI] [PubMed] [Google Scholar]
  • [18].Colloca L, Miller FG. How placebo responses are formed: a learning perspective. Philos Trans R Soc Lond B Biol Sci 2011;366:1859–69. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [19].Colloca L, Petrovic P, Wager TD, Ingvar M, Benedetti F. How the number of learning trials affects placebo and nocebo responses. PAIN 2010;151:430–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [20].Colloca L, Sigaudo M, Benedetti F. The role of learning in nocebo and placebo effects. PAIN 2008;136:211–18. [DOI] [PubMed] [Google Scholar]
  • [21].Cowen R, Stasiowska MK, Laycock H, Bantel C. Assessing pain objectively: the use of physiological markers. Anaesthesia 2015;70:828–47. [DOI] [PubMed] [Google Scholar]
  • [22].De Pascalis V, Chiaradia C, Carotenuto E. The contribution of suggestibility and expectation to placebo analgesia phenomenon in an experimental setting. PAIN 2002;96:393–402. [DOI] [PubMed] [Google Scholar]
  • [23].Egorova N, Park J, Orr SP, Kirsch I, Gollub RL, Kong J. Not seeing or feeling is still believing: conscious and non-conscious pain modulation after direct and observational learning. Sci Rep 2015;5:16809. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [24].Evers AWM, Colloca L, Blease C, Annoni M, Atlas LY, Benedetti F, Bingel U, Büchel C, Carvalho C, Colagiuri B, Crum AJ, Enck P, Gaab J, Geers AL, Howick J, Jensen KB, Kirsch I, Meissner K, Napadow V, Peerdeman KJ, Raz A, Rief W, Vase L, Wager TD, Wampold BE, Weimer K, Wiech K, Kaptchuk TJ, Klinger R, Kelley JM. Implications of placebo and nocebo effects for clinical practice: expert consensus. Psychother Psychosom 2018;87:204–10. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [25].Faul F, Erdfelder E, Lang A-G, Buchner A. G*Power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav Res Methods 2007;39:175–91. [DOI] [PubMed] [Google Scholar]
  • [26].Ferreira-Valente MA, Pais-Ribeiro JL, Jensen MP. Validity of four pain intensity rating scales. PAIN 2011;152:2399–404. [DOI] [PubMed] [Google Scholar]
  • [27].Flaten MA, Aslaksen PM, Lyby PS, Bjørkedal E. The relation of emotions to placebo responses. Philos Trans R Soc Lond B, Biol Sci 2011;366:1818–27. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [28].Flaten MA, Bjørkedal E, Lyby PS, Figenschau Y, Aslaksen PM. Failure to find a conditioned placebo analgesic response. Front Psychol 2018;1198. doi: 10.3389/fpsyg.2018.01198. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [29].Flaten MA, Simonsen T, Olsen H. Drug-related information generates placebo and nocebo responses that modify the drug response. Psychosom Med 1999;61:250–5. [DOI] [PubMed] [Google Scholar]
  • [30].Freeman S, Yu R, Egorova N, Chen X, Kirsch I, Claggett B, Kaptchuk TJ, Gollub RL, Kong J. Distinct neural representations of placebo and nocebo effects. Neuroimage 2015;112:197–207. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [31].Geers AL, Faasse K, Guevarra DA, Clemens KS, Helfer SG, Colagiuri B. Affect and emotions in placebo and nocebo effects: what do we know so far? Social Personal Psychol Compass:e12575.
  • [32].Hjermstad MJ, Fayers PM, Haugen DF, Caraceni A, Hanks GW, Loge JH, Fainsinger R, Aass N, Kaasa S. European palliative care research collaborative (EPCRC). Studies comparing numerical rating scales, verbal rating scales, and visual analogue scales for assessment of pain intensity in adults: a systematic literature review. J Pain Symptom Manage 2011;41:1073–93. [DOI] [PubMed] [Google Scholar]
  • [33].Jensen KB, Kaptchuk TJ, Kirsch I, Raicek J, Lindstrom KM, Berna C, Gollub RL, Ingvar M, Kong J. Nonconscious activation of placebo and nocebo pain responses. PNAS 2012;109:15959–64. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [34].Kaptchuk TJ, Hemond CC, Miller FG. Placebos in chronic pain: evidence, theory, ethics, and use in clinical practice. BMJ 2020;370:m1668. [DOI] [PubMed] [Google Scholar]
  • [35].Klinger R, Soost S, Flor H, Worm M. Classical conditioning and expectancy in placebo hypoalgesia: a randomized controlled study in patients with atopic dermatitis and persons with healthy skin. PAIN 2007;128:31–9. [DOI] [PubMed] [Google Scholar]
  • [36].Kuznetsova A, Brockhoff PB, Christensen RHB. lmerTest package: tests in linear mixed effects models. J Stat Softw 2017;82:1–26. [Google Scholar]
  • [37].Lautenbacher S, Peters JH, Heesen M, Scheel J, Kunz M. Age changes in pain perception: a systematic-review and meta-analysis of age effects on pain and tolerance thresholds. Neurosci Biobehav Rev 2017;75:104–13. [DOI] [PubMed] [Google Scholar]
  • [38].Lovibond PF, Shanks DR. The role of awareness in Pavlovian conditioning: empirical evidence and theoretical implications. J Exp Psychol Anim Behav Process 2002;28:3–26. [PubMed] [Google Scholar]
  • [39].McCambridge J, de Bruin M, Witton J. The effects of demand characteristics on research participant behaviours in non-laboratory settings: a systematic review. PLoS One 2012;7:e39116. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [40].McNeil DW, Rainwater AJ. Development of the fear of pain questionnaire-III. J Behav Med 1998;21:389–410. [DOI] [PubMed] [Google Scholar]
  • [41].Mouraux A, Iannetti GD. The search for pain biomarkers in the human brain. Brain 2018;141:3290–307. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [42].Peirce J, Gray JR, Simpson S, MacAskill M, Höchenberger R, Sogo H, Kastman E, Lindeløv JK. PsychoPy2: experiments in behavior made easy. Behav Res 2019;51:195–203. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [43].Petersen GL, Finnerup NB, Colloca L, Amanzio M, Price DD, Jensen TS, Vase L. The magnitude of nocebo effects in pain: a meta-analysis. PAIN 2014;155:1426–34. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [44].Piedimonte A, Guerra G, Vighetti S, Carlino E. Measuring expectation of pain: contingent negative variation in placebo and nocebo effects. Eur J Pain 2017;21:874–85. [DOI] [PubMed] [Google Scholar]
  • [45].Rescorla RA. Behavioral studies of pavlovian conditioning. Annu Rev Neurosci 1988;11:329–52. [DOI] [PubMed] [Google Scholar]
  • [46].Rhudy JL, Güereca YM, Kuhn BL, Palit S, Flaten MA. The influence of placebo analgesia manipulations on pain report, the nociceptive flexion reflex, and autonomic responses to pain. J Pain 2018;19:1257–74. [DOI] [PubMed] [Google Scholar]
  • [47].Świder K, Bąbel P. The effect of the type and colour of placebo stimuli on placebo effects induced by observational learning. PLoS One 2016;11:e0158363. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [48].Vase L, Riley JL, Price DD. A comparison of placebo effects in clinical analgesic trials versus studies of placebo analgesia. PAIN 2002;99:443–52. [DOI] [PubMed] [Google Scholar]
  • [49].Voudouris NJ, Peck CL, Coleman G. Conditioned placebo responses. J Pers Soc Psychol 1985;48:47–53. [DOI] [PubMed] [Google Scholar]
  • [50].Voudouris NJ, Peck CL, Coleman G. Conditioned response models of placebo phenomena: further support. PAIN 1989;38:109–16. [DOI] [PubMed] [Google Scholar]
  • [51].Weber SJ, Cook TD. Subject effects in laboratory research: an examination of subject roles, demand characteristics, and valid inference. Psychol Bull 1972;77:273–95. [Google Scholar]
  • [52].Wiercioch-Kuzianik K, Bąbel P. Color hurts. The effect of color on pain perception. Pain Med 2019;20:1955–62. [DOI] [PubMed] [Google Scholar]

Articles from Pain are provided here courtesy of Wolters Kluwer Health

RESOURCES