Uncovering the expectancy effect: the validation of Acupuncture Expectancy Scale

Abstract

Purpose

Research suggests that expectancy may modulate the response to medical interventions, including acupuncture. However, the paucity of validated tools to measure expectancy limits rigorous evaluation. We sought to validate a previously developed Acupuncture Expectancy Scale (AES) as an instrument to measure patients’ expected responses to acupuncture.

Methods

Participants were patients with stage I to III cancers seen in outpatient medical and radiation oncology clinics. They were drawn from three study cohorts that included 404 participants. We examined the reliability, validity and responsiveness of AES.

Results

The scores of AES had internal consistency (Cronbach’s α coefficient) of 0.95 and test-retest reliability of 0.62 over four weeks without acupuncture treatment. Those who had previously used acupuncture had higher AES compared to those who were acupuncture naïve (12.4 vs. 9.5, p=0.002). AES was higher in those who reported willingness to participate in an acupuncture trial compared to those who did not want to participate in an acupuncture trial (11.5 vs. 8.1, p<0.001). Those patients who enrolled in a pilot trial of acupuncture had higher AES score than the general outpatient population (13.0 vs. 9.8, p=0.02), and expectancy increased during the course of acupuncture treatment (13.0 to 16.5, p<0.017).

Conclusion

The AES is reliable and valid, and scores appear to increase during or after prior therapy. Incorporation of AES in clinical trials and outcome studies can evaluate the role of expectancy on acupuncture outcomes.

Introduction

There is substantial use of acupuncture in the United States (US) and world. In the US, the acupuncture use among adults has increased from 2 million to 3 million annually in the last five years, which represents a 50% increase.^1,2 Such wide use and growing interest may be the result of a combination of personal experience as well as recent research findings indicating acupuncture can be effective for some indications, such as chronic painful conditions, particularly osteoarthritis,^3–5 low back pain,^6,7, and headache.^8,9 Additionally, the integration of acupuncture into the European primary care setting appears to be cost effective for some of these conditions.^10–12 Meta-analyses based on these and other trials have found that acupuncture produces clinically relevant short- and long- term benefits when compared to routine or enhanced medical care. However, these analyses have also found that the difference between real and placebo acupuncture is either small or inconsistent with unclear clinical significance.^{13,14 15} While this finding challenges the traditional notion that acupuncture is “effective” or “efficacious,” the observed effect due to placebo/sham acupuncture demands rigorous investigation of a potential placebo response in acupuncture care. Further, because acupuncture is safe when practiced by licensed providers, patients may elect to choose safe treatments (i.e., either the real or sham acupuncture) which induce a known placebo response in order to feel better.

Response expectancy is one of the central variables in the placebo response^16–18 and may play an important role in patients’ responses to acupuncture. For example, Linde et al. found that the expectations for acupuncture outcome after the third session of acupuncture therapy is predictive of the response to acupuncture for pain in a pooled analysis of four randomized controlled trials (N=864).¹⁹ Myers et al. also found that pre-treatment expectancy is associated with improvement in functional status in patients (N=444) with acute low back pain when a combination of treatments options were offered (chiropractic, acupuncture, or massage).²⁰ A recent f-MRI study using an expectancy manipulation model for experimentally induced pain found that while real acupuncture needle stimulation may inhibit incoming noxious stimuli from the periphery, expectancy may function via a central emotional circuit.²¹

In order to rigorously investigate the role of expectancy in acupuncture care for chronic pain and symptom distress, a validated instrument must be developed to quantify response expectancy in the context of acupuncture care. We previously developed and preliminarily validated the Acupuncture Expectancy Scale (AES) in a Chinese acupuncture clinical population.²² Here we describe the validation of the AES in an English speaking, U.S. cancer patient population. Our specific hypotheses are:

1. The score of AES has acceptable reliability as measured by internal consistency and test-retest reliability.

2. The score of AES will predict intended participation in an acupuncture clinical trial as well as actual clinical trial participation as a measure of construct validity

3. The score of AES will increase during the course of acupuncture treatment as treatment experience may reinforce expectancy. This will serve as a measure of the responsiveness of the scale.

Methods

Overall Study Design

This study combines three different datasets to determine the reliability, validity and responsiveness of AES. Study 1: A cross-sectional survey study conducted in an outpatient breast cancer oncology clinic among postmenopausal women who have stage I-III breast cancer and are taking aromatase inhibitors(AI), (N=300). Subjects completed a one time survey, which included the AES. We used this sample to determine the internal consistency of the scale, to perform confirmatory factor analysis, and to test for construct validity. Study 2: A pilot, open-label acupuncture clinical trial among breast cancer survivors with symptomatic AI-related arthralgia (N=12). We used this sample to determine the effectiveness of AES as a measure in ongoing acupuncture treatment. AES was administered at baseline, week 4, and week 8. The details of the subject recruitment and conduct have been previously described.^23,24 Study 3: An ongoing cohort study among outpatient radiation therapy patients with stage I-III tumors who were receiving non-palliative radiation (N=92). The AES was administered at baseline and at 4 weeks without acupuncture intervention to determine the test-retest reliability of AES in the setting of ongoing cancer care but no acupuncture treatment. All three studies have been approved by the University of Pennsylvania Institutional Review Board. Informed consent was conducted and obtained from all study participants.

Acupuncture Expectancy Scale

The AES consisted of four items measuring the expectation of improvement of illness (symptom), enhanced coping, increased vitality, and symptom alleviation due to acupuncture therapy. It is important to note that for this study, we asked patients to endorse their expectancy of acupuncture for specific symptoms (e.g. joint pain for breast cancer survivors, and fatigue for patients undergoing radiation therapy). We did not ask patient’s expected outcome of acupuncture for treatment of cancer. Subjects were asked to rate from 1 to 5 on a five-point Likert scale, with 1 indicating “Not at all agree” and 5 indicating “Completely agree” with the expected improvement as result of acupuncture. The score of instrument ranged from 4 to 20 out of a possible 4 to 20, with higher scores indicating greater expectancy. The scale was originally developed in English but was translated into Chinese for the preliminary validation in the Chinese acupuncture clinic. Preliminary validation showed that the AES has a single factor structure with an acceptable reliability coefficient.²² Additionally, the score on the AES was positively correlated with both the perceived effectiveness of acupuncture treatment and the confidence in acupuncture care.

Statistical Analysis

Study 1

Descriptive statistics were performed for each scale item. The distributions of scores on the items were examined for missing data as well as to assess ceiling or floor effects (i.e., the majority of respondents choosing either extreme option).²⁵ We evaluated item-item and item-total correlations. We then calculated Cronbach’s alpha coefficient to evaluate the internal consistency of the scale. Additionally, we performed a principal components analysis to confirm the one domain structure that was found in our preliminary study. Interpretation of the results of the principal components analysis was guided by identifying Eigen values greater than 1 and the rotated loadings of variables on the identified components.

To evaluate construct validity, we compared the AES score between those who had acupuncture previously and those who had not. We also compared the score on the AES between those who expressed willingness to participate in an acupuncture trial and those who did not want to participate in acupuncture trial. Furthermore, we compared the AES between those who participated in the pilot acupuncture study (study 2) and those who were in study 1. Independent student’s T-test was used for these analyzes. We hypothesized a priori that patients’ expectancy of acupuncture may be higher among acupuncture experienced patients than acupuncture naïve patients. We also hypothesized a priori that expectancy predicted self-reported willingness to participate in acupuncture trial as well the actual participation.

Study 2

We evaluated the change in AES between baseline and end, baseline and middle, and middle and end of acupuncture therapy using the paired-student’s T-tests.

Study 3

To evaluate the test-retest reliability of the AES, we performed intra-class correlation between baseline and week 4 AES scores, which calculated reliability. All statistical analyses were performed using STATA 10.0. All statistics were two sided with an alpha of 0.05 indicating significance.

Results

Participants

The three study cohorts included 404 participants (see Table 1). The detailed characteristics of study 1 and 2 have been previously reported.^23,24 In summary, these studies consisted of women who had breast cancer and finished their primary cancer treatments, such as radiation and chemotherapy, and were currently receiving aromatase inhibitor medications. The median age of the participants was 59, with a range between 33 and 84. 84% were white. The radiation therapy cohort included 92 subjects with the following tumor types: 44 (47.8%) prostate, 27 (29.4%) breast, and 21 (22.8%) others. The median age of the participants was 66, with a range between 38 and 85. 38% were female, and 29.3% were non-white (mostly African American).

Table 1.

Study participants characteristics

	Study 1	Study 2	Study 3
	N=300	N=12	N=92
Median age, years (min-max)	61(33–84)	59 (52–70)	66 (38–85)
Sex, Female, no.(%)	300 (100%)	12 (100%)	35 (38%)
Race, no. (%)
White	253 (84.3%)	10 (83.3%)	65 (70.7%)
Non-white*	47 (15.7%)	2 (16.7%)	27 (29.3%)
Employment, no.(%)
Full-time	114 (38.5%)	5 (41.7%)	32 (35.2%)
Part-time	40 (13.5%)	2 (16.7%)	8 (8.8%)
Not currently employed	142 (48.0%)	5 (41.7%)	51 (56.0%)
Education
High school or less	122 (40.8%)	2 (16.7%)	23 (25.3%)
College	76 (25.4%)	3 (25.0%)	44 (48.0%)
Graduate or professional school	101 (33.8%)	7 (58.3%)	24 (26.7%)
Tumor types:**
Breast	300 (100%)	12 (100%)	27 (29.4%)
Prostate			44 (47.8%)
Others			21 (22.8%)

Not all cells add up the total N due to missing data

^*Non-white: mostly African American

^**Other tumors included 9 Head & Neck, 9 Gastrointestinal, and 3 lung

Scale Properties

The four items are shown in Table 2. No item had over 5% missing data (the range was 4% to 5%). While we saw no ceiling effects, we observed substantial floor effects. Between 22% and 39% of subjects endorsed the lowest response options for the 4 items of the AES. The score was calculated for 283 (94.3%) of study participants and ranged from 4 to 20 out of a possible 4 to 20, with higher scores indicating greater expectancy. The mean was 9.8, with a standard deviation (SD) of 4.6, and a median of 8. Twenty percent of subjects had the minimum summative score. As we stratify the AES score by subject-reported willingness to participate in an acupuncture trial, the distribution can be seen in Figure 1. While 34% of subjects endorsed the minimum summative score among those who would not participate in an acupuncture trial, the AES score among those who indicated that they would like to participate were more normally distributed with 5% or less of the sample endorsing the extreme score on either end.

Table 2.

Acupuncture Expectancy Scale – Distribution of items (N=300)

Every individual may have different expectation for the effects of acupuncture. If we use the following sentences to describe your expectation of acupuncture's effect on your illness/symptom* after the entire course of acupuncture therapy, how much do you agree? For each statement, please choose the closest answer.
	(1)	(2)	(3)	(4)	(5)
	Not at All	A Little	Moderately	Mostly	Completely
	Agree, %	Agree, %	Agree, %	Agree, %	Agree, %
My illness* will improve a lot	23	30	22	17	8
I will be able to cope with my illness* better	22	28	23	19	8
The symptoms of my illness* will disappear	39	26	19	11	5
My energy level will increase	29	29	20	15	7

^*illness is replaced by specific symptoms in the study (e.g. joint pain, fatigue) to be specific to the research question

Figure 1.

Distribution of AES score by willingness to participate in an acupuncture trial

In a principal component factor analysis to confirm the factor structure in the original study, we identified and retained one component with an Eigen value of 3.52. This explained over 88% of the variance. Varimax rotated loadings ranged between 0.91 and 0.95.

Reliability

Our 4-item scale had an internal consistency (Cronbach’s alpha coefficient) of 0.95 with corrected item-total correlations for the scale ranged between 0.84 and 0.92.

Construct Validity

The score of AES appeared to be different in a priori defined populations (see Figure 2). Among the 300 participants in study 1, 27(9%) previously had acupuncture. As expected, subjects who had acupuncture previously had a 31% increase in expectancy related to acupuncture outcomes compared to those who were acupuncture naïve (12.4 vs. 9.5, p=0.002). Those subjects who were offered and reported that they were willing to participate in an acupuncture trial had a 42% higher AES score than those who did not want to participate (11.5 vs. 8.1, p<0.001). Merging baseline data from the 12 subjects who participated in the acupuncture trial (study 2) and study 1, clinical trial participants had a higher AES score than those general outpatients in an oncology setting (13.0 vs. 9.8, p=0.02).

Figure 2.

AES mean score differences among subgroups

Test-retest Reliability

Among the study participants enrolled in study 3, the AES score had no appreciable change at 4 weeks in the absence of acupuncture (10.1 vs. 10.7, p=0.43) (see Figure 2). The intra-class correlation between baseline measure and week 4 is 0.62, 95% confidence interval (0.48–0.75).

Responsiveness

Among the 12 subjects who participated in an 8-week 10 session single arm acupuncture clinical trial, expectancy increased from baseline, to week 4 of treatment, and to week 8 (13 vs. 15 vs. 16.5 respectively), see Figure 2. The AES was significantly increased from baseline to week 8 (P=0.017). The comparisons between baseline and week 4 (p=0.25) and between week 4 and week 8 (p=0.29) were not significant.

Discussion

What leads to the powerful effects seen in the placebo/sham acupuncture groups compared to routine care in recent clinical trials? And how can we harness these effects more broadly for clinical care? To assess response expectancy as a potential source of ‘non-specific’ therapeutic effects, it is essential to develop a psychometrically sound measure that can be used in acupuncture research and clinical care. In this study, we extended our previous work and further validated a simple 4-item Acupuncture Expectancy Scale (AES) to measure subjects’ specific expectation about the outcomes of acupuncture therapy in English-speaking U.S. cancer patient populations. The scale is found to be reliable, valid, and is sensitive to change during treatment.

Several outcomes were different between this study and our previous research conducted in the Chinese acupuncture patient population.²² The American participants were mostly acupuncture naïve and there were substantial flooring effects in their responses to individual items with about 1/5 firmly believing acupuncture was unlikely to have any effect. In contrast, the Chinese participants were experienced with acupuncture and there were substantial ceiling effects in their responses, with many believing acupuncture would have significant positive effects. As suggested by our data, prior acupuncture experience may increase response expectancy towards acupuncture. Thus, the American/ Chinese differences may be related to the fact that the Chinese acupuncture patients were receiving acupuncture at the time of the survey. Secondly, the reliability coefficient of Cronbach’s alpha was substantially higher in this study compared to the original Chinese validation study (0.95 vs. 0.82). It is possible that for those patients who have had acupuncture, their rating of expectancy is modeled after the actual acupuncture experience, so the variability of endorsement for individual items was greater for the patients who have had acupuncture. Nevertheless, the differences noted in the two studies highlight how different cultures and prior experience with conventional and complementary therapies may affect the expectancy of acupuncture.

Several previous studies have focused on the role of expectancy on clinical outcomes of acupuncture in the context of clinical trials. Thus far, most studies^26,27 found higher pretreatment expectancy was associated with greater clinical improvement as measured by validated disease outcome measures. However, the use of different types and often un-validated scales to measure expectancy makes it difficult to gain insight into the degree expectancy plays a role in pain and symptom relief. It is possible that expectancy may play a bigger or smaller role in various conditions,²⁸ as the psychological state of the patient may vary by condition, such as for cancer as compared with chronic knee pain. Also, the use of the validated AES across the disease spectrum may be important to isolate the specific effect of expectancy for various diseases outcomes (e.g. pain, function, satisfaction) in acupuncture care. Such understanding may help clinicians understand for which disease or specific outcomes, expectancy may exert greater power in the clinical outcome for patients. This would allow more careful facilitation of positive expectancy in certain patients.

Our study also highlights that positive response expectancy is associated with increased use of therapies, both stated and actual in clinical trial context. As the use of acupuncture and other types of conventional or complementary therapies occurs in societal and cultural contexts, expectancy can be influenced by science, media, health care professionals, family and friends. Research using the AES with appropriate study designs may help uncover how social and scientific forces shape expectancy and affect the utilization of acupuncture, and ultimately affect clinical outcomes.

Recent advancement in neuroscience and imaging methods has increased the understanding of potential biological mechanisms of expectancy and placebo response. In a pharmacological paradigm, the expectancy of pain relief may be mediated via both brain opioid and dopaminergic pathways.^29–32 Recent investigations in acupuncture yielded intriguing findings. In a fMRI study using expectancy manipulation and thermal pain model, Kong et al. found that although the acupuncture analgesic effect and expectancy-evoked placebo analgesia produced similar reductions in patient-reported pain, the neuro-pathways underlying such reports were distinct.^21,33 Similarly, using C-carfentanil PET imaging, Harris et al. found that while both real and sham acupuncture produced similar clinical benefits for patients with fibromyalgia, the mechanism underlying the pain reduction was different -- real acupuncture increased both short- and long-term mu-opioid receptor (MOR) binding potential in multiple pain and sensory processing regions, while sham acupuncture actually resulted in only a small reduction of MOR binding potential.³⁴ The above studies suggest that while real acupuncture may produce analgesia via peripheral to central, bottom-up modulation, expectancy (placebo effect) may induce analgesia via top-down, i.e., centrally mediated modulation. Using AES combined with advanced imaging techniques and clinical outcomes in actual disease models may help further elucidate the complex interaction between expectancy and specific efficacy of acupuncture needling in producing meaningful clinical outcomes. Such understanding will pave the way to better optimize the physiological effect of acupuncture needling and the psychological effect of setting appropriate outcome expectancy to produce clinical improvement of pain, symptoms, and functions.

Several limitations of our study need to be acknowledged. Our study participants were cancer patients and mostly female. Validation of the scale in other clinical and healthy populations is needed to learn how the scale will perform in other settings with different disease entities. Our scale was intended to measure specific expectancy towards acupuncture, so it may not capture other important cognitive processes such as trust and perceived barriers that may play important role in both use and efficacy of acupuncture. Furthermore, large prospective studies are needed to evaluate the predictive validity of this scale.

Despite the limitations, this study represents an incremental and necessary step in developing methods to uncover the mechanism of the “powerful placebo” effect seen in acupuncture research. The rapid development of neuroscience is likely to provide increased understanding of the biological mechanism of acupuncture. The development of a psychometrically sound measure of response expectancy related to acupuncture will further translate such mechanistic understanding into actual clinical context. The substantial non-specific effect seen in the previous acupuncture literature provides an impetus for effective utilization of such power in clinical care. Appropriate use of the AES combined with other newly developed biological markers may powerfully facilitate clinical and translational research that ultimately creates patient-centered integrative approaches to effectively ameliorate the suffering of chronic pain and other symptom distress.

Figure 3.

Longitudinal change of AES with or without acupuncture