Acupuncturist Perceptions of Serving as a Clinical Trial Practitioner

Abstract

Objective

This article describes the experience of four acupuncturists in terms of what it meant for them to be a practitioner in a recently completed sham-controlled acupuncture randomized control trial (RCT) with a standardized protocol.

Design

At the completion of the RCT for women with ovarian dysfunction, study acupuncturists (2 male MD/acupuncturists and 2 female professional acupuncturists) were queried about their perceptions of participating in the RCT using both written responses to 5 open-ended questions and a focus group interview. Data was analyzed to categorize responses and identify themes.

Setting

Virginia, USA

Results

The acupuncturists’ experience of participating in a RCT was generally very positive, including: usual practitioner/participant relationships, collegial sharing, and increased patient volume and diversity. There was angst expressed about the unknown RCT results. While there were concerns about standardizing the acupuncture session (“dilutes the power of acupuncture therapeutics”), the acupuncturists’ were supportive of the pre-established protocol. The acupuncturists overall did not have concerns with a sham intervention arm because the sham recipients did not know their treatment arm and felt as satisfied with study participation as the true acupuncture recipients.

Conclusions

Despite initial misgivings about both a standardized protocol and a sham arm, all practitioners discovered positive aspects of being a study acupuncturist. The analysis highlights the need for communication before, during and after a clinical trial between the study investigators and the intervention practitioners.As stake holders in the perception of CAM therapies with the public and with conventional medicine practitioners, it would benefit future research on CAM to similarly assess experiences of being a CAM study practitioner in order to enhance provider recruitment and reduce provider drop-out.

Introduction

Qualitative research methods can be effective for the investigation of both study participant and study practitioner experiences and the meanings they attach to those experiences.¹ A review article reported that only 30% of randomized clinical trials (RCTs) of complex healthcare interventions in the Cochrane Effective Practice and Organization of Care register had an associated qualitative research component, of which only two-thirds were published as standalone papers or within another paper.² All of those qualitative studies involved interviewing study participants. Within the literature specific to acupuncture, four articles were identified detailing qualitative research with participants who received acupuncture therapy through a research trial with a comparison arm.^3-8

As infrequent as qualitative studies of participants are within a RCT, rarer still is a qualitative study of the practitioners involved in the intervention. Only three publications were located with qualitative research of study acupuncturists who served as a practitioner for a clinical trial.^9-11 MacPherson et al conducted a pragmatic trial of acupuncture compared with usual medical care for low back pain.¹² Interviews with their six study acupuncturists yielded two papers.^{9, 11} Goldman et al conducted a sham-controlled acupuncture RCT with a semi-flexible acupuncture protocol for people with repetitive motion arm pain¹³, within which was conducted a qualitative study of 8 of their 12 study acupuncturists.¹⁰

As called for by others, complementary alternative medicine needs to be more broadly designed and include more comprehensive research methods, such as qualitative analysis. Qualitative methods are useful in addressing research topics that may be inaccessible using quantitative analysis.^{1, 14} Towards that goal, the aim of this study was to examine acupuncturists’ perceptions of what it was like for them to be a practitioner in a sham-controlled randomized clinical trial. Some authors have implied that a conflict of approach exists when a “western” study design of a clinical trial is implemented with a CAM modality¹⁶. As stake holders in the perception of CAM therapies with the public and with conventional medicine practitioners, there is great benefit in the assessment of CAM provider experiences of being a provider in a classically designed clinical trial. Research overall on CAM modalities is important as it is one source of information that prospective and current users of CAM employ to evaluate a given therapy.¹⁵ More relevant for this article, academics need CAM providers to participate in research, and CAM providers need academic research to support evidence-based practice. This study is an example of cooperation between academic researchers and CAM providers to examine the process of conducting the research from the point of view of the acupuncturists.

Materials and Methods

Underlying Randomized Clinical Trial

A randomized clinical trial (“clinical RCT”) was conducted to assess the effectiveness of acupuncture for restoring ovulation and normalizing ovarian hormones in women with Polycystic Ovarian Syndrome.¹⁷ Participants were randomized to receive 12 standardized sessions designed to deliver either active “true” acupuncture or a placebo “sham” acupuncture over 8 weeks. The sham acupuncture arm used the Park Sham device¹⁸ at points off the meridiens. All treatments were standardized and identical within an intervention arm. The acupuncturists were not blinded to the intervention or to the aim of the study.

Initially the clinical RCT utilized only a single MD/acupuncturist, and he subsequently served as the lead study acupuncturist as other practitioners were added. Over the course of four years, three additional acupuncturists were added to the clinical RCT to accommodate more participants both in the original geographic area and when the RCT was expanded to a new geographic region. In total there were four study acupuncturists, none of the acupuncturists withdrew from the RCT as a study practitioner.

Sample

This study used a qualitative sampling technique which selects for those who have a particular knowledge of the topic of interest, a “judgment” sample.¹⁹ The sample consisted of the four acupuncturists who participated in the underlying clinical RCT. One acupuncturist was unable to attend the focus group due to geographic distance and logistic constraints. None of the acupuncturists were a provider in any other research study while a practitioner in this trial.

The description of the sample is as follows. Two of the acupuncturists were male, both had MD degrees with acupuncture training from the Helms Institute, were clinicians affiliated with an academic department of Physical Medicine and Rehabilitation, and were Fellows of the American Academy of Medical Acupuncture. One of those practitioners was additionally a Clinical Preceptor at the Helms Institute. Two of the acupuncturists were female, both were professional non-MD community acupuncturists, both had 4 year Master diplomas from an acupuncture school, and one additionally had a Diplomate in Acupuncture from the National Certification Commission for Acupuncture and Oriental Medicine. Three of the acupuncturists owned their own private practices. The range of post-training clinical acupuncture experience for each of the acupuncturists prior to joining the study was 4 to 10 years.

Study Procedures

After the final study intervention session, the four participating acupuncturists were queried as to their perceptions about participating in the RCT. (At this time, the clinical RCT participants were in a follow-up phase and all data were still blinded as to the treatment arm.) This inquiry consisted of two parts, the first being the acupuncturists’ written response to 5 questions (Table 1) asking what it was like for them to participate in the study, which was a similar approach as used previously by other qualitative acupuncture research.¹⁰ These semi-structured questions elicited the personal feelings and opinions of the acupuncturists regarding participation in the study and specific aspects of the study design. A final question queried general recommendations for acupuncturists who participate in future studies. Semi-structured interview techniques, such as used in the first part of this study, are useful for “delving deeply” into a particular topic.¹⁹ The list of standardized questions guides the interview in order to insure that certain information that is of interest to the researcher is covered.

Table 1.

Written response questions

What did you anticipate it would be like to be an acupuncturist for a clinical trial? How were your expectations the same or different than it really was like? How did you feel about giving a standardized acupuncture session (i.e., the study dictated which points to use and for how long with no deviation over the 8 weeks of intervention)? How did it feel to give a placebo treatment? How did it feel to not be able to tell your patient what treatment she was giving, and about not being able to talk to the study principal investigator or research coordinator about the treatment that was given to a particular woman? What recommendations do you have for researchers who are planning future acupuncture studies?

A focus group was subsequently conducted with three of the study acupuncturists to elaborate on their experiences in the clinical RCT. The focus group methodology allows for exploration of a specific issue within a social context.²⁰ The goal of this inquiry was to better understand the shared viewpoint of the acupuncturists’ interpretations of their experiences as part of a RCT. In this way, the researchers were better able to explore what the acupuncturists’ thought about the experience and why, particularly in relation to each other’s opinions. As information was explored and re-phrased within the group, the acupuncturists were able to identify what was important to them as a group essentially defining group norms and values.²¹

Table 2 displays the open-ended a priori questions for the interview. . The questions in both tables were designed by the Principal Investigator (PI) of the clinical RCT and a trained qualitative researcher who was not involved with the clinical RCT, with considerations of focus group questions used in a prior qualitative study of complementary-alternative medicine (CAM) practitioners.²² The clinical RCT PI was not present at the focus group session.

Table 2.

Focus group questions

What are your overall reactions/feelings/opinions about this research study now that it is complete? How does participating in this research differ from how you thought it might be? In general, what does it mean to you to participate in research? How did it feel to give a “standardized acupuncture session?”

Data Analysis

Qualitative data analysis techniques were used to interpret the study data. The transcribed text of the focus group and the written responses to the five questions were entered into the qualitative software program NVivo 8. Each transcription was read and re-read, in a process similar to open coding²³, to organize important segments of data that could be grouped into categories or themes. All data analysis was conducted by the qualitative researcher.

Researcher bias was controlled by the qualitative researcher fully describing the details and meanings of the acupuncturists using examples from their own words, thus allowing the reader to make his/her own interpretations of the information presented.²⁴ The thematic descriptions and meanings were shared with the study acupuncturists, particularly when the qualitative researcher required their input clarification. All study data, including the researcher’s journal of methodological and analytical notes, the transcribed email responses to the semi-structured interview questions, the audio-tape of the focus group and the written transcriptions were preserved to maintain an audit trail. This allows validation of the study results in that others could replicate the study.²⁵ Five categories or themes were generated by this method of inquiry: “It doesn’t reflect what I do”, “Standardized treatment is certainly easier to do”, “A quiet, safe place to rest even for sham recipients”, “Angst over unknown RCT study results”, and “Reflections about the study”.

Results

Theme 1: “It doesn’t reflect what I do”

This theme involved discussions about how the acupuncturists’ style and training differed, what they thought about possible findings from the RCT study, and how the public and practitioners of Western medicine would perceive the results. The acupuncturists noted that the RCT study results would not represent important differences in practice, which they perceived existed based on the providers’ varied training, as previously described. The acupuncturists commented that they believed that how they were trained made a difference in how they practiced acupuncture. “We both have acupuncture in our toolkits…but even the way we do acupuncture is not the same because it’s in a different framework.” However, in this study those practice differences were not of consequence, as they all used the same standardized study treatment protocol.

Regardless of training, the acupuncturists agreed that findings from this study might not accurately represent the full potential effectiveness of acupuncture. A standardized treatment protocol was used as part of the design for this RCT, which the acupuncturists felt altered the treatment approach they would have used with the study participants by not allowing individualization between patients and sessions. Two of the acupuncturists expressed wanting positive results to validate the effectiveness of acupuncture; the other felt even if there were positive results, the implications would be limited.

“ … either way, whether it’s positive or negative, it’s just a commentary on a very specific protocol. It’s not a referendum on acupuncture.”

There were concerns that if the results were negative, this would reflect poorly on the image of acupuncture among the general public and Western Medicine practitioners. While the acupuncturists expressed concerns that the study results would not be a true reflection of the efficacy of acupuncture treatment, they recognized the need to use a standardized treatment to meet the requirements of the study design.

Theme 2: “Standardized treatment is certainly easier to do”

The acupuncturists discussed the positive aspects of using a standardized acupuncture treatment despite the fact that it was not an accurate representation of how acupuncture is practiced. The MD/acupuncturist, who participated in the focus group, acknowledged that he often incorporated more of a standardized treatment protocol in his everyday practice in that it was more consistent with Western medical treatment. All acupuncturists reported that there was much less effort and thought needed when giving a standardized treatment which saved time as they could administer the treatment right away. In this way, using the standardized treatment meant the acupuncturists did not need to evaluate the study participant’s health concerns at every visit; which in their usual practice took time and effort. Using the standardized treatment made participation as a practitioner in the study easier.

“The use of a standardized treatment meant that I could begin acupuncture treatment immediately without 10-15 minutes of evaluation time. I still felt that I was able to establish a good rapport with almost all of the participants I treated. Some clients who are less verbal or more guarded prefer minimal evaluation time even in my private practice.”

Theme 3: “A quiet safe place to rest even for sham recipients”

Other benefits from using a RCT study design with acupuncture were discussed. The acupuncturists commented how study participants assigned to the sham treatment arm enjoyed and perceived benefits from the sessions with the acupuncturists even though they were not told which treatment group they were in until the end of the study. The acupuncturists expressed feelings of ethical conflict about having a sham arm. They worried that study participants would feel betrayed if they found out. This was lessened when they realized that study participants perceived benefits from the sessions whether or not they received the sham or real acupuncture treatment. The acupuncturists described positive feelings about administering the sham treatments, as well as the real acupuncture treatments, as they saw study participants’ positive response to both.

“… my general impression was that the patients felt cared for and looked after and enjoyed the experience, whether it was the sham treatment or the real treatment they enjoyed it. People like human interaction and they like the chance to lie there for 20 minutes and relax and have a break from their busy hectic life.”

The acupuncturists felt that the sham group would also generate positive findings in the clinical RCT. “I would not be surprised if the placebo treatment proved to be comparable to the true treatment in its efficacy.” This lead to the recommendation that study participants be educated on the possible positive effects of sham treatment prior to implementing the study. Most likely this was in response to the ethical aspects the acupuncturist felt towards not deceiving the study participants and not wanting them to feel that it was a “waste of their time.”

Theme 4: “Angst over unknown RCT study results

As the timing of this inquiry was after the final study acupuncture visit and before the RCT results were known, a theme of angst over the unknown RCT results seemed to underlie various comments by the practitioners. This angst is reflected in Theme 1 through the training distinctions that the acupuncturists made as a reason for potential negative RCT results, and their concern that the standardized acupuncture protocol would not accurately reflect medical results that they would expect from individualized acupuncture sessions. This angst was also present in their concerns about having a sham arm (“Using the standardized treatments left me feeling less confident of the success of the treatments”). Additionally, despite the fact that the study acupuncturists met regularly to review study protocols and to clarify and discuss issues, the acupuncturists suggested that more routine meetings would have been beneficial. In discussing how they each implemented the study protocol, differences were noted. They agreed that these differences could make a difference in the findings yet are difficult to quantify.

“Over the course of the study I realized how many little variations could occur within such a seemingly simple set of rules.”

Theme 5: “Reflections about the study”

The final theme centers on the acupuncturists’ comments about why they participated in a RCT study, the concerns they had at the beginning, the positive and negative aspects of being a practitioner for the RCT and recommendations they had for future studies. Overall it was a positive experience for the acupuncturists to be a practitioner in the clinical RCT, despite some initial misgivings (“I expected to feel like an automaton rather than a physician”). As mentioned previously, the two professional non-MD acupuncturists participated in the study with the hopes that results from this study would contribute to the data showing the effectiveness of acupuncture. One commented that she knew it would be very difficult for a RCT to show positive results with acupuncture; however she was interested in learning about the research process nonetheless. The MD/acupuncturist commented that participating in research is an expectation of his job, and so he was very glad to participate in the study.

All of the acupuncturists described being pleased that they did participate in the study and described positive benefits such as mentioned previously. The acupuncturists reported that the study participants represented a broader socio-economic and racial population than their usual clientele base.

“… [I] found out there are a lot more people out there who are interested in acupuncture than I may have realized before because they can’t afford it and because this study was paid for -- the acupuncture was paid for -- the women, many women were eager to participate who would not have otherwise been able to.”

In addition, they reported enjoying the collegial aspects of working in the study with other practitioners and with the research team. The acupuncturists reported understanding why a blinded study design was necessary and, once they saw that the study participants perceived a benefit whether they received the sham or real acupuncture treatment, this was no longer an issue for them.

Discussion

In summary, the acupuncturist practitioners in this clinical RCT overall felt it was a positive experience to be part of research (Themes 2 and 5). They had mixed feelings on having a sham intervention because of the acupuncturist’s ethical concerns and whether the research would show that this CAM intervention was of benefit (Themes 1 and 3). Lastly, they reported that a standardized protocol conflicted with individualization of treatment, the latter of which is typically utilized by acupuncturists (Theme 1).

While our study acupuncturists would agree that CAM research is important, their comments support these two beliefs:

1. Positive or negative findings of a CAM study do not foretell whether the CAM therapy (acupuncture in this instance) will be helpful for an individual patient with a given condition, and

2. The patient’s own experience of benefit is very valuable, even if that benefit is not supported by a biological measure of treatment success¹⁵, or in the case of some RCTs, the treatment received was a placebo.

The acupuncture providers in this study held the same viewpoint about CAM treatment success as the patients in a study by Verhoef¹⁵ Verhoef et al interviewed 27 patients about the information employed and the role of scientific evidence in deciding to use CAM therapy as part of their cancer treatment. While the cancer patients agreed on the need for scientific evidence, there was little agreement on what actually constituted evidence to them. As stated above, regardless of the scientific evidence, what worked for them or for someone they trusted was a top source of credible information on the efficacy of a treatment. In contrast, a fundamental notion underlying all research is that distributions have inherent variability and that studies should be designed to minimize that variability. Academic investigations are focused on the research results related to the study population as a group, and any variability from one person to another is wrapped into a measure of dispersion (e.g., standard deviation) for reporting purposes. Our findings show that the acupuncturists in this study maintained their usual individual provider perspectives even as they were part of the investigative team.

Our themes are supportive of several findings from the McManus report from a sham-controlled RCT of acupuncture for arm pain.¹⁰ Specifically, McManus et al wrote “several acupuncturists had complex feelings about the use of sham needling and used various strategies to cope with ethical questions that arose while administering sham treatments.” On a related note, they reported that the practitioners felt confused and/or demoralized when the sham recipients improved. These sentiments are echoed in our themes 1 and 3. Additionally, the acupuncturists in the McManus article overall felt their study involvement was a positive experience professionally, as was true for our acupuncturists (theme 5).

There were two findings from the McManus publication that were not supported by our cohort of practitioners. First, the acupuncturists in that report hoped that their positive intentions (i.e., a non-specific intention effect) might counter the sham protocol; this conscious intention effort was not voiced by our study practitioners. Second, the McManus acupuncturists commented that it was challenging to adhere to the protocol due to their desire to use non-protocol clinical tools (e.g., external herbal treatment); our acupuncturists did not express a desire to use non-protocol techniques/tools. Our acupuncturists did, however, express frustration about having a fixed protocol that did not vary by session or between patients (theme 1). The RCT¹³ underlying the McManus qualitative research did allow some variability in acupuncture points, thus frustration with a fixed protocol would not be relevant to those study acupuncturists. As reported elsewhere, CAM practitioners who implemented a fixed research protocol (aromatherapists in the study by Kyle et al ²²) expressed a similar frustration as our acupuncturists with the limited study treatment choice and/or the difference between the intervention protocol and their normal practice. On the other hand, both our acupuncturists and the aromatherapists from Kyle et al ²² discovered positive aspects of having a standardized protocol (e.g., easier to implement for the practitioner).

The first of the two MacPherson et al qualitative publications⁹ reported that the study acupuncturists discussed the importance of building rapport with clients, the importance of individualizing the treatment, the need to get their patients involved in helping themselves, and whether the study patients were similar to their regular clientele. The second of the MacPherson qualitative publications¹¹ focused on the fact that the study acupuncturists provided self-help advice to the study participants, and that they saw the self-help advice as important for improved health over the longer term. The underlying low back pain research trial¹² compared acupuncture (“pragmatic” trial without a specified protocol) to usual care, which may explain why their acupuncturists’ comments did not revolve around a sham comparison arm nor a standardized study protocol.

One take-home message from our investigation was that the study acupuncturists desired more frequent meetings than actually occurred. The underlying RCT involved initial training by the lead acupuncturist in a face-to-face session with a mock or real study participant prior to administering any study intervention. This was followed with three team meetings over the course of four years with all the study acupuncturists, in addition to continual phone calls and emails between the acupuncturists, PI, research coordinator, and the lead acupuncturist. Based on the focus group comments, this schedule of acupuncturist meetings was perceived as being insufficient as they requested more frequent team meetings (theme 4). McManus et al¹⁰ specifically commented on the need for frequent monitoring of the acupuncturists and the need to schedule meetings of the team acupuncturists “to share strategies and support”.

As reviewed by Carter¹⁶, researching CAM is challenging and subject to criticism from both proponents of evidence-based medicine as well as those who believe that RCTs fracture the essence of holistic therapy. Thus, an inherent conflict of approach can exist within a clinical trial of a CAM modality. Carter¹⁶ discussed several elements of CAM therapies that are difficult to translate into an RCT design. Several of these friction points were voiced by the acupuncturists in this study. Carter stated that randomization “may distort practice and create tensions in the therapeutic relationship,” which is related to our theme 3. Carter highlighted that “CAM tends to focus less on the disease and more on individual assessment/presentation of the client”. This was reflected in the acupuncturists’ comments about the lack of treatment individualization and routine evaluation of the woman’s latest symptoms (themes 1 and 2). Carter noted that the influence of the practitioner is a feature of most CAM interventions; our study acupuncturists felt that they were still able to form a therapeutic relationship with research participants (theme 3) even within the boundaries of the study protocol.

How does this article advance the theory or practical implication of CAM for health practice? Study provider input is important for future study designs, and, unfortunately, it is rarely systematically collected. Literature abounds on the potential bias in results from participant loss-to-follow-up, but little attention has been given to the impact of potential bias from provider drop out or, analogously, “recruitment bias” from a lack of involvement by providers. CAM provider input is especially important because CAM providers are not typically part of an academic environment and may be less likely to have their views included in the medical literature. In order for CAM studies to meet the rigor of traditional academic research, CAM practitioners should be much more frequently incorporated into the research infrastructure from the time of the investigational design through the subsequent transition into practice. We hope this paper will influence CAM study design methodology by setting forth the experiences of these study acupuncturists in order to enhance the participation and satisfaction of CAM practitioners in future research.

Strengths and Limitations

This is the fourth known publication to assess the experience of acupuncture practitioners in the context of their study involvement, and the second to examine acupuncturists who served as practitioners in a sham-controlled trial. A unique strength of this paper is that the perspectives of both professional non-MD acupuncturists and MD/acupuncturists were incorporated. While the MD/acupuncturists may have had more experience with the use of placebo and standardized treatments in RCTs, all acupuncturists in this study expressed understanding the importance of research and the difficulties of accurately researching a CAM modality. A second strength is that the data was collected before the clinical RCT results were known, thus the interviews provided input that was not biased by knowing the impact of the acupuncture intervention. Lastly, this study incorporated input from all the acupuncturists who were involved in the clinical RCT, because none of the practitioners dropped out of the study team and all agreed to give their input.

Limitations of this inquiry include the fact that there were only four acupuncturists in the RCT study, so these findings may not be representative of acupuncturists in general. On the other hand, the acupuncturists in this investigation had similar sentiments to those in the McManus report, so the small sample size may not be relevant. In addition, inherent in the social aspects of the focus group, individual dissent from the group consensus may have been silenced.²¹ As is true with all qualitative research, these findings cannot be generalized to all study acupuncturists. Lastly, the study acupuncturists had varying amounts of time as a study practitioner, and this may have influenced their responses.

Conclusions

Our findings support the contention that some aspects of RCT designs are in conflict with the individualized approach of acupuncture practice, although the underlying clinical trial is an example of a CAM RCT where this underlying conflict did not appear to jeopardize the research nor pose unacceptable constraints on the practitioners. These findings add important information to the limited body of publications on the experience of being an acupuncture provider within a RCT framework. Such knowledge can aid the design of future studies, and help Principal Investigators to understand the perspectives of acupuncturists who may be new to the research world. We encourage researchers to consider adding qualitative studies alongside randomized controlled trials of CAM interventions to expand upon the research findings in ways that only interviews can capture.