Double-blinding with a new placebo needle; a further validation study

Abstract

Background

We demonstrated the masking properties of a new, non-penetrating, double-blind placebo acupuncture needle. Practitioners correctly identified some of the needles; if they were confident in this opinion, they would be unblinded. Therefore we aimed to investigate the clues that led to correct identification, and the confidence in this decision.

Methods

Ten acupuncture practitioners, blindly and randomly, applied ten each of three types of needle to the shoulder: blunt, non-penetrating needles that pressed the skin (‘skin-touch placebo needle’); new non-penetrating needles that penetrated soft material (stuffing) but did not reach the skin (‘non-touch control needle’); and matching penetrating needles. Afterwards, practitioners were asked to judge the type of needle, their confidence in their decision, and what clues led them to their judgements.

Results

Out of the 30 judgements made by each practitioner, the mean number of correct, incorrect and unidentifiable answers were 10.4 (SD 3.7), 15.2 (SD 4.9) and 4.4 (SD 6.1), respectively. There was no significant difference in the confidence scores for 104 correct [mean, 54.0 (SD 20.2) %] and 152 incorrect [mean, 50.3 (SD 24.3) %] judgements. Ten needles were identified with 100% confidence – three correct, and nine incorrect. For needles correctly identified, the proportions of non-touch (p = 0.14) and skin-touch (p = 0.17), needles were no greater than chance, but the proportion of penetrating was (p < 0.01). 53% of judgements were made from the “feeling of needle insertion”, but 57% of these were wrong.

Conclusion

Practitioners had a slight tendency to guess the penetrating needles correctly, but were uncertain about most of their judgments, posing only a very small risk to double blinding.

INTRODUCTION

Background

Double-blind design, where both the patient and practitioner are masked to the treatment condition, is critical for accurately measuring the efficacy of any treatment, including acupuncture, but blinding acupuncture practitioners is a great challenge.^{1 2} Without blinding practitioners, measurement of treatment effects may be influenced by bias, so acupuncture research may be seen as having less methodological rigor than conventional medical research.^3-8

Blinding of acupuncturists was not considered feasible^{1 2 9 10} until we designed and validated a new double-blind needle.^11-14 The blunt needle simply presses against the skin, but meets some resistance from soft material (lower stuffing) in the guide tube to give the impression that it penetrates.^11-14 In this paper, we call this the ‘skin-touch placebo needle’ to distinguish it from another type of needle we describe later.

In our first validation study, ten experienced acupuncturists each applied 23 of the non-penetrating placebo needles and 17 conventional penetrating needles to the LI4 point, a total of 400 applications. After removing each needle, they made a judgement whether the needle was ‘penetrating’, ‘non-penetrating placebo’ or ‘unidentifiable’. They judged 170 needles correctly, 166 incorrectly, and rated 64 as unidentifiable, which was a chance distribution.^{11 12} Subsequent investigations provided further validation: experienced practitioners made statistically an equal number of correct and incorrect judgements of the type of needle in one study; and, in another study, they made a larger number of incorrect judgements than correct ones.^{13 14} Therefore we concluded that the new needle is effective for blinding.^11-14

However, because of the way we conducted those previous studies, we could not tell to what extent practitioners were able to truly identify any of the needles, for example from the sensation in their fingers as they used them. If a proportion of needles can be truly identified with confidence, this would jeopardize the blinding.

For this study, we designed a new version of the placebo needle in which the tip does not reach the skin but still meets resistance from soft material. Our aim was to test whether this produced a different sensation for the practitioners, but we were also interested in whether a non-touch placebo needle was realistic, since researchers have questioned whether placebo needles that touch the skin might produce stimulation and therefore not be true placebos.^{15 16}

The aims of the present study were to explore to what extent practitioners can determine the true nature of these needles from the feeling during use; what these judgements were based on, and how confident they were in them.

METHODS

Participants

We recruited ten licensed and experienced acupuncturists (mean duration of acupuncture experience of 10.8 (SD 12.9) years; mean age: 40.1 (SD 11.1) years; 4 men and 6 women) from the teaching or research staff of Japan School of Acupuncture, Moxibustion and Physiotherapy, The Educational Foundation Hanada Gakuen, Tokyo, Japan (table 1). Before the study, its purpose and format were explained and the participants provided written consent. The Showa University Ethics Committee gave its approval.

Table 1. Acupuncturist’s judgements on 30 needles (10 of each type).

Acupuncturist	Years of experience	% of correctly identified needles		Number of needles judged as ‘control’			Number of needles judged as ‘placebo’			Number of needles judged as ‘penetrating’			Number of needles unidentified
				Non-touch control needle	Skin-touch placebo needle	Penetrating needle	Non-touch control needle	Skin-touch placebo needle	Penetrating needle	Non-touch control needle	Skin-touch placebo needle	Penetrating needle	Non-touch control needle	Skin-touch placebo needle	Penetrating needle
No.1	8.5	36.7		4	5	4	1	2	1	5	3	5	0	0	0
No.2	1.5	33.3		2	3	2	2	2	2	6	5	6	0	0	0
No.3	1.5	13.3		1	0	1	0	0	1	2	3	3	7	7	5
No.4	1.5	23.3		4	0	8	3	3	2	3	7	0	0	0	0
No.5	2.5	60.0		5	2	2	4	5	0	0	3	8	1	0	0
No.6	3.5	33.3		2	2	2	4	3	2	3	4	5	1	1	1
No.7	5.5	33.3		2	3	0	1	2	2	4	0	6	3	5	2
No.8	34	33.3		2	4	2	2	2	0	2	4	6	4	0	2
No.9	15	30.0		7	3	3	0	0	1	2	7	2	1	0	4
No.10	34	50.0		5	2	3	0	4	1	5	4	6	0	0	0
Mean (SD)	10.8 (12.9)	34.7 (12.9)	Total	34	24	27	17	23	12	32	40	47	17	13	14

Note: Red numbers indicate correct identifications.

Design of double-blind needles

We used three types of needles in this study. 1) the non-penetrating placebo needle, the tip of which presses against the skin but cannot penetrate it (‘skin-touch placebo needle’), 2) the matching ‘penetrating needle’ with 10mm insertion depth. These have been described in detail elsewhere.^11-14 Additionally, in this study, we used 3) the newly developed needle (‘non-touch control needle’), the tip of which cannot reach the skin. The appearance of these three needles is indistinguishable (fig 1). The diameter of the needles was 0.16 mm.

Figure 1.

Double-blind needles.

A, Means of needle insertion. B, The non-touch control (left), skin-touch placebo (middle) and penetrating needles (right) comprise an opaque guide tube and upper stuffing to give resistance to the needle body during its passage through the guide tube. The needle body of the non-touch control needle is shorter than the guide tube, and the needle body of the skin-touch placebo needle is just long enough to allow its blunt tip to press against the skin, but the needle body of the penetrating needle is longer than the guide tube by an amount equal to the insertion depth when the needle body is advanced as far as possible. The non-touch control and skin-touch placebo needles contain lower stuffing to give a similar sensation to that of skin puncture and tissue penetration. Each needle has a stopper, which prevents the needle handle from advancing further when the tip of the needle reaches the specified position. The pedestal on each needle is adhesive, allowing it to stick firmly to the skin surface. The left and right needle for each type of needle is before and after needle insertion, respectively.

Validation test for practitioner blinding

For each practitioner we prepared ten of each type of needle and shuffled them to achieve a random order.

We informed the practitioners about the design of the needles, and instructed them to apply the needles as if treating a stiff neck using any appropriate acupuncture or tender points they chose. The acupuncturists applied each needle separately, taken randomly from the shuffled set of 30 needles, into the shoulder of the author NT, 15 on each side, using the ‘alternating twirling technique’ i.e. insertion by rapid rotation of the needle. The acupuncturist then advanced the needle until the stopper made contact with the top of the guide tube and finally withdrew it (fig 1), while observing the reactions of the patient. There was no further needle manipulation used in this study, reflecting one of the techniques most commonly used by practitioners in Japan. The acupuncturist then removed the entire needle assembly from the skin. The assistant, blind to the true nature of needles and the practitioner’s judgements, numbered the needle and sealed it in an opaque envelope. The practitioner then reported his/her judgement as to whether the needle was non-touch control, skin-touch placebo, penetrating needle or unidentifiable. He/she then rated his/her confidence in making this judgement (i.e. the degree of certainty about his/her decision) on a visual analogue scale (VAS), the end-points of which were 0 for no confidence and 100 for complete confidence. He/she also reported clues that led to his/her judgement of identity of the needle; the options were “feeling of needle insertion”, “feeling of needle removal”, “feeling in the left hand holding the guide tube (‘Oshide’ in Japanese)” (fig 1), “body movement”, “bleeding”, and “lack of bleeding”.

Author NT, blinded to the practitioners’ responses, subsequently opened the opaque envelopes to record the type of needles. We took all possible precautions to ensure that the identity of the needle was not revealed to the practitioners, the patient or the investigators during the acupuncture trials.

Data analysis

The chi-squared test was used to determine whether the numbers of needles fitted an expected probability. Practitioners’ confidence scores for correct and incorrect identifications were compared using Mann-Whitney’s U test; comparisons between three groups were made using Kruskal-Wallis test. Spearmans’ rank correlation coefficient was used to indicate the relationship between the years of practitioners’ experience for acupuncture practice and number of correctly identified needles. All statistical analyses were performed using SPSS, version 15.0J (SPSS Inc, Chicago, IL).

RESULTS

All ten acupuncture practitioners completed all judgements for all needles. The mean of the practitioners’ overall confidence in making their judgements on all 300 needles was 44.2 (SD 28.0) %. Of the total 300 needles applied, the practitioners identified 104 (34.6%) correctly (non-touch control = 34, skin-touch placebo = 23, penetrating = 47); 152 (50.7%) incorrectly (non-touch control = 49, skin-touch placebo = 64, penetrating = 39); and reported 44 (14.7%) as unidentifiable (non-touch control = 17, skin-touch placebo = 13, penetrating = 14), see table 1. The 104 correctly identified needles overall fitted the probability of 1/3 (χ² = 0.24, p = 0.62). There was no significant difference in the confidence scores for correct [mean, 54.0 (SD 20.2) %] and incorrect [mean, 50.3 (SD 24.3) %] judgements (p = 0.09). Practitioners stated they were 100% confident in 12 judgements: three were correctly identified (one non-touch control and two penetrating needles), seven were incorrectly identified as penetrating (one non-touch control and six skin-touch placebo needles) and two were incorrectly identified as non-touch control (skin-touch placebo needles). For each of the non-touch control and skin-touch placebo needles, the proportion of correctly identified needles fitted the probability of 1/3 (χ² = 2.18, p = 0.14; χ² = 1.86, p = 0.17). The 47 penetrating needles that were correctly identified exceeded the chance probability of 1/3 (χ² = 17.59, p < 0.01) (table 1). However, there was no significant difference in the confidence scores for different types of needle (p = 0.69) (fig 2).

Figure 2.

Confidence level in the practitioners’ judgements of ‘non-touch control’, ‘skin-touch placebo’ and ‘penetrating’ for non-touch control, skin-touch placebo and penetrating needles. The top, middle and bottom lines of the boxes correspond to the 75th, 50th (median) and 25th percentiles, respectively. The whiskers extend from the 10th percentile to the 90th percentile. The circles indicate the arithmetic mean.

Note: N.S. indicates that there was no significant difference.

Practitioners were more likely to make the judge needles as ‘penetrating’ (119 out of 256 needles identified, χ² = 19.9, p < 0.01). However, 47 correct identifications of the 119 needles fitted the probability 1/3 (χ² = 2.03, p = 0.15) as in 23 correct identifications of 52 needles judged as skin-touch placebo (χ² = 2.78, p = 0.10) and 34 correct identifications of 85 needles judged as non-touch control (χ² = 1.70, p = 0.19). The mean confidence score for 47 correctly identified penetrating needles was 55.9 (SD 21.4) % which was higher, though not significantly (p = 0.17), than 48.5 (SD 17.1) % of correctly identified skin-touch placebo and 55.2 (SD 20.4) % of the correctly identified non-touch control (fig 2). Importantly, there was no significant difference in confidence between non-touch control, skin-touch placebo and penetrating needles identified as penetrating (p = 0.15); and the same goes for these that identified as non-touch control (p = 0.34) and skin-touch placebo (p = 0.70) (fig 2).

The practitioners made their judgements on the nature of the needles principally on the “feeling of needle insertion” and were far less dependent on other clues. The mean score for confidence of the practitioners in 69 needles correctly identified from “feeling of needle insertion” was 57.6 (SD 19.0) % (table 2).

Table 2. Number and confidence score (mean ± SD %) in practitioners’ identification of needles identified from clues.

Clue for identification	Total number reported / cases of correct identification	Confidence in total / cases of correct identification
Feeling of needle insertion	160 / 69	54.3 ± 20.6 / 57.6 ± 19.0
Feeling of needle removal	17 / 7	49.6 ± 22.0 / 55.9 ± 17.7
Feeling in the left hand holding the guide tube (‘Oshide’)	13 / 6	34.9 ± 16.1 / 41.6 ± 17.7
Body movement	11 / 3	29.2 ± 19.2 / 16.7 ± 2.8
Bleeding	0	-
Lack of bleeding	0	-
Feeling of needle insertion + feeling of needle removal	41 /14	47.5 ± 22.5 / 48.1 ± 18.5
Feeling of needle insertion + body movement	1 / 0	77.2 / −
Feeling of needle insertion + feeling of needle removal + body movement	1 / 1	91.6 / 91.6
Others	12 / 4	70.0 ± 34.0 / 47.3 ± 22.9

Note: Forty-four needles were unidentified.

The highest percentage of correct answers was 60.0% in the acupuncturist with 2.5 years of acupuncture experience, and the lowest was 13.3% in the acupuncturist with 1.5 years of acupuncture experience (table 1). There is no significant correlation between the years of experience in acupuncture and the numbers of correctly identified needles (r = 0.42, p = 0.23) (table 1).

DISCUSSION

In previous validation studies we found that recently developed double-blind needles have the potential to mask practitioners.^11-14 But there still remained a question about whether practitioners were sometimes certain of their judgement about a needle, which could jeopardize blinding. Practitioners in this study judged a great majority of correctly identified needles with uncertainty, which indicates that the identity of the needles was well blinded from the practitioners. However, practitioners in this study had a tendency to judge needles to be penetrating compared with the previous studies.^11-14 Further, a few (less than 3%) correctly identified needles were reported with 100% confidence. These results suggest potential limitation in success of perfect practitioner blinding in clinical acupuncture study. Therefore, in future use, the confidence in the practitioner’s guesses should be recorded to see whether the true identity of the needle is revealed.

The 60% and 50% of correct identification obtained by practitioners No. 5 and No. 10 were conspicuous scores, as was the lowest figure, 13% in practitioner No. 3. However, the success rates of these three seeming outliers were within 2 standard deviations (± 12.9) of the mean success rate of 34.7. Thus statistically the three practitioners did not perform much better or worse than the others. There was no significant correlation between the experience years of acupuncture practice and the number of the correctly identified needles. Duration of experience does not seem to affect judgement.

In our previous validation studies, acupuncture points to which the needles were applied were located in the upper extremity.^11-14 In this study acupuncturists applied the needles to the shoulder, to replicate a clinical setting. However, the double-blind needle still needs to be validated in a true clinical trial, and by other centres.

Although a non-penetrating needle that presses the skin may be physiologically active, we call our non-penetrating needle “placebo” because it lacks a key ingredient of acupuncture: skin penetration. At present we are not aware of any conclusive evidence whether placebo, sham or another type of needle has a specific therapeutic effect on any medical condition obtained under double-blind conditions using an appropriate control. The new, non-touch needle designed for this study could be physiological inert in terms of stimulation by needle tip, and our results suggest it would be suitable for practitioner blinding. It would be important to compare the effect of these two needles on the symptoms of a clinical condition, though it may be difficult to mask patients. Thus, the explanation given to patients will be the next challenge in adopting such non-touch control needles in research.

CONCLUSION

In conclusion, this study suggests that unblinding is not likely with this placebo needle, since practitioners were not certain about a large majority of their correct identifications. However, it showed a potential limitation in success of perfect practitioner blinding because practitioners had a tendency to guess needles to be penetrating, and because a few of true identities of needles may be revealed.