Challenges of Systematic Reviewing Integrative Health Care

Abstract

This article is based on an extensive review of integrative medicine (IM) and integrative health care (IHC). Since there is no general agreement of what constitutes IM/IHC, several major problems were identified that make the review of work in this field problematic. In applying the systematic review methodology, we found that many of those captured articles that used the term integrative medicine were in actuality referring to adjunctive, complementary, or supplemental medicine. The objective of this study was to apply a sensitivity analysis to demonstrate how the results of a systematic review of IM and IHC will differ according to what inclusion criteria is used based on the definition of IM/IHC. By analyzing 4 different scenarios, the authors show that, due to unclear usage of these terms, results vary dramatically, exposing an inconsistent literature base for this field.

Introduction

As an emerging field, integrative health care (IHC), also called integrative medicine (IM), faces the problem of a widely accepted and applied definition. In an earlier publication,^1,2 the authors noted that the definitions for IM/IHC can range from the definition of IHC as simply incorporating complementary and alternative medicine (CAM) into biomedicine, to the notion that IHC constitutes a new form of medical practice. While the articles in the Khorsan et al¹ systematic review used the term integrative, the terminology is not well defined in the literature. In addition, The Khorsan et al systematic review report used the guiding principle for our inclusion criteria that IHC research is the study of the incorporation of CAM with biomedicine as a collaborative and integral part of the health care system, that is, the integration of conventional (allopathic) medicine and CAM, involving shared management of the patient, shared patient care, shared practice guidelines, and shared common values and goals to treat the well-being of the whole person.^1,3 In applying this systematic review methodology, the authors found that many of those captured articles that used the term integrative medicine were in actuality referring to adjunctive, complementary, or supplemental medicine. By adjunctive we mean that the CAM therapy is used in addition to allopathic care but as supplemental or complementary to the biomedical care. Adjunctive therapy in this sense has no requirement about joint management, collaboration, or a partnership between providers. It simply means the patient is using CAM and biomedicine concurrently. Even where the providers are cognizant of the use of both forms of therapy, the CAM is seen primarily as adjunctive. This would be the case in cancer treatment, for example, where acupuncture is used to treat nausea arising from the use of chemotherapy, where clearly the primary therapy for the actual cancer is chemotherapy. We are making no implied judgment here about the worth of adjunctive care or whether this is a positive role for CAM to play, we are concerned only with whether the definition of IHC/IM as currently being proposed excludes or includes adjunctive care.

The authors of the systematic review discovered the lack of a clear definition and clarity of the term integrative health care medicine/integrative medicine (IHC/IM), and the absence of taxonomy for models of IHC/IM make it very difficult to efficiently conduct systematic reviews of this field at this time. Depending on what definition is used, the amount of literature can be either very small or very large. If IM means genuine cooperation and comanagement of patients, then in our previous article, of the 11,591 original citations captured in the literature under the category integrative, most were not truly integrative. We reviewed 20 studies that we deemed eligible for qualitative synthesis, which were further reduced to 6 articles that met the strict inclusion criteria of being randomized controlled trials. Excluded from that review were articles that used the term integrative but actually were referring to adjunctive therapy (n = 52) as we define it. Please see Figure 1 that is published elsewhere for a description of identified studies.¹

Figure 1.

Systematic review flow chart.

In this article we examine further the type of confusion that can occur in systematic reviews in this field depending on whether adjunctive therapy is included in the definition for IHC and what type of adjunctive therapy is excluded if IHC is defined as in the more expansive definition as given by such writers as boon et al.^1,3 They provide the following working definition. Writers such as Boon et al give the following working definition for IHC as the combination of an interdisciplinary, nonhierarchical blending of both CAM and conventional medicine that employs a collaborative team approach guided by consensus building, mutual respect, and a shared vision of health through a partnership of patient and practitioners to treat the whole person by synergistically combining therapies and services in a manner that exceeds the collective effect of the individual practice.^1,3

Bell et al⁴ define IHC as “a transformative system represented by a higher-order system of systems of care that emphasizes wellness and healing of the entire person (bio-psycho-socio-spiritual dimensions) as primary goals, drawing on best both conventional and CAM approaches in the context of a supportive and effective physician-patient relationship.” Stumpf et al⁵ refer to the malleable definitions of IHC as “one first step towards understanding the phenomenon.” But it is misleading to term what is happening in the field of IHC/IM if, in fact, what is “occur ring “more closely resembles assimilation of CAM” by biomedicine [rather] than true acculturation.” ⁵ As we will show in this report, the term integrative cannot be taken at face value as the definition because we believe it to mean (and it could, in fact, mean) adjunctive therapy, which is something quite different from integrative as we will try to portray. Therefore, systematic review methodology is challenged by the term integrative when trying to determine the quantity, quality, and effectiveness of IHC practices.

In this article, we have chosen to compare articles that met the very expansive definition of IHC/IM as we define it with articles that would meet the definition if it included adjunctive therapy. The objective of this article is to apply a sensitivity analysis to explore how the results of a systematic review of IHC/IM differ according to what inclusion criteria are used based on the definition of IHC/IM. The primary purpose of this article is to examine what is lost or gained depending on the type of studies included or excluded.

Methods

We describe the full methodology of this systematic review elsewhere.^1,2 In brief, we searched PubMed, Allied and Complementary Medicine (AMED), BIOSIS Previews, EMBASE, the entire Cochrane Library, MANTIS, Social SciSearch, SciSearch Cited Ref Sci, PsychInfo, CINAHL, and NCCAM grantee publications listings, from database inception to May 2009, as well as searches of the gray literature. Available studies published in English language were included. Three independent reviewers rated each article and assessed the methodological quality of studies using the Scottish Intercollegiate Guidelines Network (SIGN 50).⁶ There were no available systematic reviews or meta-analyses published that met our inclusion criteria. Our initial search terms were Integrat* and Medicine; Integrat* and Health* (for health care); multidisciplinary care; complementary or alternative and conventional medicine or health care; and delivery of health care and integrat*. Our initial search yielded 11,591 citations. Of these, only 660 were judged to be relevant to the purpose of our search and were coded into their respective categories of study design.

We originally published an article reporting only on the randomized controlled trials and controlled clinical trials, which amounted to 6 articles that truly met the inclusion criteria that fit our definition of IM. In order to compare definitions of adjunctive versus IHC/IM for those studies in our original systematic review, we subjectively chose 4 articles. Two of the 4 articles were examples of IHC/IM.^7–10 One of the 2 was included in our systematic review (met the expansive definition) while the other that we described contained integrative medicine in its title but involved adjunctive or complementary medicine in its methodology. In addition, 2 of the 4 articles were examples of adjunctive or complementary medicine. However, 1 of the 2 had adjunctive or complementary medicine in its title, but was integrative in its methodology (See Tables 1 and 2).

Table 1.

Studies matrix.

	Integrative title	Adjunctive title
Integrative intervention	Edelman D, Oddone EZ, Liebowitz RS, Yancy WS Jr, Olsen MK, Jeffreys AS, Moon SD, Harris AC, Smith LL, Quillian-Wolever RE, Gaudet TW. A multidimensional integrative medicine intervention to improve cardiovascular risk. J Gen Intern Med. Jul 2006;21(7):728–34.	Krucoff MW, Crater SW, Green CL, Maas AC, Seskevich JE, Lane JD, Loeffler KA, Morris K, Bashore TM, Koenig HG. Integrative noetic therapies as adjuncts to percutaneous intervention during unstable coronary syndromes: Monitoring and Actualization of Noetic Training (MANTRA) feasibility pilot. Am Heart J. Nov 2001;142(5):760–9.
Adjunctive intervention	Narahari SR, Aggithaya MG, Prasanna KS, Bose KS. An integrative treatment for lower limb lymphedema (elephantiasis). J Altern Complement Med. Feb 2010;16(2):145–9.	Berman BM, Lao L, Langenberg P, Lee WL, Gilpin AM, Hochberg MC. Effectiveness of acupuncture as adjunctive therapy in osteoarthritis of the knee: a randomized, controlled trial. Ann Intern Med. Dec 21, 2004;141(12):901–10.

Table 2.

Study characteristics.

Citation	Population description	Description of intervention	Description of control	Intervention duration and time-points	Primary outcomes assessed/results	Adverse events	Sign score*
Adjunctive title and adjunctive intervention
Berman et al. (2004)	570 patients with osteoarthritis of the knee (mean age [±SD], 65.5 ± 8.4 years).	Acupuncture: True acupuncture sessions	Sham acupuncture: Procedure was given on the same schedule as the experimental group and used the same active needle placements, except actual insertion did not occur at these 9 points. Sham points: Acupuncturists inserted 2 needles into the sham points and then immediately applied 2 pieces of adhesive tape next to the needles. Education control: The education–attention control consisted of 6 two hour group sessions based on the Arthritis Self-Management Program and taught by an experienced, Arthritis Foundation–trained patient education specialist.	23 true acupuncture sessions over 26 weeks gradually tapering treatment according to the following schedule: 8 weeks of 2 treatments per week followed by 2 weeks of 1 treatment per week, 4 weeks of 1 treatment every other week, and 12 weeks of 1 treatment per month.	WOMAC (Pain): While pain among participants who were receiving true acupuncture decreased more than in the sham group at all of the postbaseline assessments, this difference was not statistically significant at week 8. By week 14, the mean pain score had decreased by 3.6 units in the acupuncture group (a 40% decrease from baseline) compared with −2.7 in the sham group (P = 0.02). These differences remained at week 26 (P = 0.003). WOMAC (Function): The true acupuncture group’s improvement in function from baseline was significantly greater than that of the sham control group at weeks 8 (P = 0.01), 14 (P = 0.04), and 26 (P = 0.009). A change of more than 12 units by 14 weeks is an almost 40% improvement from baseline. Effect Size: ND	No adverse effects were associated with acupuncture.	++
Adjunctive title and integrative intervention
Krucoff et al. (2001)	150 coronary disease patients (mean age [±SD], 63 ± ND years).	The Monitoring and Actualization of Noetic Training (MANTRA): Feasibility of applying 4 noetic therapies—stress relaxation, imagery, touch therapy, and prayer plus UC	UC: absence of any noetic treatment assignment by MANTRA protocol.	Baseline period was defined as the period from when the patient signed the consent form to the time the patient entered the catheterization laboratory. The physiologic stress period was defined as the period beginning with the arteriotomy in the catheterization laboratory until 30 mins after the last contrast injection. The recovery period was defined as the period from 30 mins after the last contrast injection through the time of hospital discharge. Long-term follow-up was defined as 6 months from the day of the PCI.	DUREL (Spirituality): There were no significant outcomes differences across these groups or when protocol noetic therapy was administered (NS). SAQ (Anxiety): No differences between arms (NS). Adverse Periprocedural Outcomes: There was a 25% to 30% absolute reduction in adverse periprocedural outcomes in patients treated with any noetic therapy compared with standard therapy. The lowest absolute complication rates were observed in patients assigned to off-site prayer. All mortality by 6-month follow-up was in the noetic therapies group. In patients with questionnaire scores indicating a high level of spiritual belief, a high level of personal spiritual activity, a low level of community-based religious involvement, or a high level of anxiety, noetic therapies appeared to show greater reduction in absolute in-hospital complication rates compared with standard therapy. Effect Size: ND	ND	++
Integrative title and adjunctive intervention
Narahari et al. (2010)	467 Llymphatic filariasis patients (mean age [±SD], ND ± ND years).	Biomedical, Ayurvedic, and yoga medicine	ND	Baseline, 14th day, 45th day, and 90th day.	Volume reduction: Treatment showed the volume reduction of 22.3% on 14th day, 30.8% on 45th day, and 41.6% on 90th day of treatment for large limbs; for small limbs it was 11.1%, 24.3%, and 31.8%, respectively. Statistical significance ND.	Contact dermatitis to nalpamaradi oil used for IMLD in five limbs. About 10% of patients developed scattered sterile pustules over the massaged limbs.	−
Integrative title and integrative intervention
Edelman et al. (2006)	154 cardiovascular risk outpatients (mean age [±SD], 53.4 ± 4.8 years (UC) and 52.2 ± 5.2 years (PHP)).	Personalized Health Plan (PHP)-The plan identified specific health behaviors important for each subject to modify; the choice of behaviors was driven both by cardiovascular risk reduction and the interests of each individual subject. Techniques used in implementation included mindfulness meditation, relaxation training, stress management, motivational techniques, and health education and coaching.	UC: absence of any noetic treatment assignment by PHP protocol.	Baseline, 5, and 10 months	Framingham risk score (10-year risk of CHD): Baseline 10-year risk of CHD was 11.1% for subjects randomized to UC (n = 77), and 9.3% for subjects randomized to PHP (n = 77). Over 10 months of the intervention, CHD risk decreased to 9.8% for UC subjects and 7.8% for intervention subjects. Based on a linear mixed-effects model, there was a statistically significant difference in the rate of risk improvement between the 2 arms (P = 0.04). In secondary analyses, subjects in the PHP arm were found to have increased days of exercise per week compared with UC (3.7 vs. 2.4, P = 0.002), and subjects who were overweight on entry into the study had greater weight loss in the PHP arm compared with UC (P = 0.06).	ND	+

Notes:

^*SIGN checklist for RCTs and controlled clinical trials; ⁺⁺Strong. All or most of the criteria have been fulfilled; ⁺Article is neither exceptionally strong nor exceptionally weak; ⁻Weak. Few or no criteria fulfilled.

Abbreviations: NS, Not Significant; Wk(s), Week(s); Min, Minutes; Mon(s), Month(s); QOL, Quality of Life; HR, Hour; EoT, End of Treatment; SE, Side Effect; SD, Standard Deviation; ND, Not Described; DUREL, Duke University Religion Index; SAQ, Spielberger anxiety questionnaires; WOMAC, Western Ontario and McMaster Universities Arthritis Index.

Results

“Complementary therapies” was a term that was introduced to MeSH terms (Medical Subject Headings) for MEDLINE in 1986 and redefined in 2002 at the National Library of Medicine (NLM). In MeSH, the controlled vocabulary thesaurus, complementary therapies are defined as “therapeutic practices which are not currently considered an integral part of conventional allopathic medical practice.” They may lack biomedical explanations, but as they become better researched, some (physical therapy modalities, diet, and acupuncture) become widely accepted, whereas others (humors and radium therapy) quietly fade away, yet are important historical footnotes. Therapies are termed as complementary when used in addition to conventional treatments and as alternative when used instead of conventional treatment. The MeSH subject tree includes acupuncture, holistic care, midbody therapies, musculoskeletal manipulations, and so on. The term integrative medicine (IM) was introduced as a MeSH term in 2009. IM is defined by MeSH as the “discipline concerned with using the combination of conventional (allopathic) medicine and alternative medicine to address the biological, psychological, social, and spiritual aspects of health and illness.” When we searched PubMed in 2011 for the term integrative medicine, we found 5235 total citations of which 246 were classified as clinical trials. In our original systematic review, we found that the majority of the citations from our search results included study citations for adjunctive therapies and complementary therapies. Below we examine the challenge faced when deciding if an article is truly about IM/IHC or whether it is about adjunctive therapies or complementary medicine.

Cases of adjunctive therapy/complementary medicine

We examine first the case where a research article claims its intervention to be IM/IHC but turns out to be adjunctive or complementary (see Table 1).

As noted previously, we found that in our systematic review the definitions of integrative healthcare run the gamut from those who see it as simply the integration of CAM in some form of relationship with biomedicine (usually institutional and sometimes referred to as adjunctive therapy or complementary/combination medicine)¹¹ to those who propose that it is a new form of medicine as a combination of an interdisciplinary, nonhierarchical blending of both CAM and conventional medicine that employs a collaborative team approach guided by consensus building, mutual respect, and a shared vision of health through a partnership of patient and practitioners to treat the whole person by synergistically combining therapies and services in a manner that exceeds the collective effect of the individual practice.²

In order to compare examples of adjunctive therapy versus IHC/IM, we chose 2 studies from our systematic review. Both studies were considered adjunctive in their methodology, were tagged as adjunctive/complementary therapy based on our definition of IHC/IM above and were excluded from our systematic review.^7,8 However, 1 of the studies did pass our secondary review⁷ when its abstract was reviewed by 2 reviewers and was deemed inclusionary because it included the phrase integrative treatment in its title and abstract⁷ while the other did not.⁸ The study by Narahari et al⁷ combined traditional ayurvedic practices with biomedicine to treat lymphedema caused by lymphatic filariasis (LF). The study states that it followed an integrative protocol. However, it is not clear how patients received standard biomedical care in addition to ayurvedic practices. We posit that if patients had received both ayurvedic practices and standard care, incorporation of CAM into conventional medicine without a joint management does not constitute IM/IHC. Indeed, this study states “patients with LF are examined by a team of biomedical, Ayurvedic, and yoga medicine experts, who assess both patient and pathology from all three perspectives,”⁷ but the use of CAM as an adjunctive treatment to conventional care without a unifying paradigm may lead to worse outcomes because the selected CAM modality may adversely interact with conventional modality and vice versa.^7,8

The second study, by Berman et al, tested the efficacy of acupuncture for reducing the pain and dysfunction of osteoarthritis compared with sham acupuncture. Similar to the above study, patients continued to receive biomedical care during the study and were allowed to receive their usual medications.⁸ The objective of this study was to see if acupuncture can have a vital role in adjunctive therapy as part of a multidisciplinary integrative approach to treating symptoms related to knee osteoarthritis.⁸ There was little to no joint management, collaboration, and/or partnership between providers giving treatment. In addition, an IM/IHC philosophy or process was not incorporated in this study.

The 2 studies described above represent instances of complementary therapy or adjunctive therapy that are close to conforming to the definition of ICH/IM, but for both studies, we were unable to find integrative management of care as defined by our systematic review and, therefore, exclude both studies from our systematic review.

Two cases of integrative health care

The second situation is where there was IM/IHC, that is, the incorporation of CAM with biomedicine as a collaborative and integral part of the health care. This involved the integration of conventional (allopathic) medicine and CAM and involved shared management of the patient, shared patient care, shared practice guidelines, and shared common values and goals to treat the well-being of the whole person.

Two studies met our inclusion criteria for IM/IHC.^9,10 The Edelman et al study on multidimensional IHC interventions to improve cardiovascular risk, defines integrative medicine as “an individualized, patient-centered approach to health, combining a whole-person model with evidence-based medicine.”⁹ Their multidimensional intervention was guided by IM/IHC principles and tested the effect of personalized health planning (PHP) involving the “use of patient-centered, individualized therapeutic approaches.”⁹ This study incorporated a transdisciplinary (nonhierarchical blending) approach by employing a collaborative team guided by consensus building and a shared vision of health through a partnership of patient and practitioners. In essence, IM/IHC should aim to treat the whole person by synergistically combining therapies and services in a manner that exceeds the collective effect of the individual practice. Therefore, Edelman et al labeled this study as both integrative in title and integrative in intervention.

Like the Edelman et al study, the study by Krucoff et al¹⁰ tested an IM/IHC intervention. Unlike Edelman et al, Krucoff et al state in their title that they examined integrative noetic therapies as adjuncts to percutaneous intervention during unstable coronary syndromes. However, both studies’ primary objective was to test the effect of holistic, patient-centered health planning and incorporation of team-oriented health care practices incorporating CAM interventions with biomedicine as a collaborative and integral part of the health care system. We note that both studies refer to IHC as IM.

Discussion

The above cases illustrate the challenge facing those who wish to conduct systematic reviews on IM/IHC. As we note in an earlier article, this is reflective of the fact that this is an emerging field in which groups are vying to establish the field and its definition. More recently, the University of Arizona Center for Integrative Medicine announced the creation of a formal specialty for medical doctors in IM.¹² We see here a major shift in the original meaning of IM as a type of practice bringing together complementary and alternative medicine (CAM) and biomedicine as a branch of allopathic medicine. Using the term medicine implies hierarchy, while according to Boon et al’ the term integrative health care “employs a collaborative team approach guided by consensus building, mutual respect, and a shared vision of health care that permits each practitioner and the patient to contribute their own/particular knowledge and skills within the context of a shared, synergistically-charged plan of care.”² Therefore, the term medicine for authors such as Boon et al is not an appropriate descriptor for this health care model.

The MeSH hierarchy scheme offered through the National Library of Medicine for Medline offers the correct definitions for the terms discussed in this report; however there is not a clear distinction of the intersection between these terms. One challenge is that IM is a new term recently introduced to MeSH and there is no structure that falls below this tree yet. By defining IM and applying the term correctly in the literature, we as researchers can help to dissect this problem of definitions and contribute to building the MeSH tree and vocabulary for IM to dissolve these conflicting problems with definitions.

Until the field “sorts itself out,” and a common agreement about what the field really is emerges and what the correct term for the field is going to be, it will continue to pose a challenge for scholars, particularly for those doing systematic reviews. At the moment, depending on which definition is used, the field can be very large or quite modest. It also poses a problem for synthesis of the data from the studies. Is a study of adjunctive care really pertinent to assessing IM/IHC? How are we to assess the outcomes from IM/IHC unless we can first determine what it is that is being assessed? At the very least, this article suggests that we cannot rely on titles or abstracts as the basis for the search. The articles must be examined to see what, in fact, was done. One solution might be for journals to insist on certain criteria for calling a study one of IM/IHC.

We would therefore suggest there are at least three possible solutions: (1) authors can try to ensure that the term Integrative, both in their articles and in their titles, is used correctly, (2) editors can also insist that titles accurately describe what is in the article, and (3) the MeSH hierarchy scheme, offered through the National Library of Medicine for Medline, can develop a clear distinction of the intersection between the terms discussed in this article and develop a structure that falls below the “integrative tree.”