Skip to main content
BMC Complementary and Alternative Medicine logoLink to BMC Complementary and Alternative Medicine
. 2018 Dec 22;18:341. doi: 10.1186/s12906-018-2384-5

Acupuncture decreased the risk of coronary heart disease in patients with rheumatoid arthritis in Taiwan: a Nationwide propensity score-matched study

Mei-Yao Wu 1, Ming-Cheng Huang 1,2, Hou-Hsun Liao 2,3, Jen-Huai Chiang 4,5, Yu-Chen Lee 1,6, Chung-Y Hsu 7, Mao-Feng Sun 1,2,✉,#, Hung-Rong Yen 1,2,8,9,10,11,✉,#
PMCID: PMC6303917  PMID: 30577824

Abstract

Background

Patients with rheumatoid arthritis (RA) have a higher risk of coronary heart disease (CHD). Acupuncture, a commonly used treatment for patients with RA, has not been reported to prevent CHD in patients with RA. We aimed to assess the risk of developing CHD in acupuncture users and non-users of patients with RA.

Methods

We identified 29,741 patients with newly diagnosed RA from January 1997 to December 2010 from the Registry of Catastrophic Illness Patients Database from the Taiwanese National Health Insurance Research Database. Among them, 10,199 patients received acupuncture (acupuncture users), and 19,542 patients did not receive acupuncture (no-acupuncture users). After performing 1:1 propensity score matching by sex, age, baseline comorbidity, conventional treatment, initial diagnostic year, and index year, there were 9932 patients in both the acupuncture and no-acupuncture cohorts. The main outcome was the diagnosis of CHD in patients with RA in the acupuncture and no-acupuncture cohorts.

Results

Acupuncture users had a lower incidence of CHD than non-users (adjusted HR = 0.60, 95% CI = 0.55–0.65). The estimated cumulative incidence of CHD was significantly lower in the acupuncture cohort (log-rank test, p < .001). Subgroup analysis showed that patients receiving manual acupuncture of traditional Chinese medicine style, electroacupuncture, or combination of both all had a lower incidence of CHD than patients never receiving acupuncture treatment. The beneficial effect of acupuncture on preventing CHD was independent of age, sex, diabetes mellitus, hypertension, hyperlipidemia, and statins use.

Conclusions

This is the first large-scale study to reveal that acupuncture might have beneficial effect on reducing the risk of CHD in patients with RA. This study may provide useful information for clinical utilization and future studies.

Keywords: Acupuncture, Coronary heart disease, National Health Insurance Research Database, Rheumatoid arthritis, Taiwan

Background

Rheumatoid arthritis (RA) is one of the most prevalent chronic and systemic inflammatory disorders, with the characteristics of joint destruction, functional disability, and early death [1]. Autoantibody production, synovial inflammation, cartilage and bone destruction are the main pathological findings in RA patients [2]. Current pharmacotherapies of RA include non-steroidal anti-inflammatory drugs (NSAIDs), glucocorticoids, and disease-modifying anti-rheumatic drugs (DMARDs) [3]. DMARDs, including synthetic and biological DMARDs, have been demonstrated to suppress inflammation to reduce structural damage.

The high mortality rate of patients with RA was mainly because of the significantly higher risk of cardiovascular diseases in patients with RA than in the general population [4]. The previous study revealed that patients with RA had an approximately twofold higher risk of coronary heart disease (CHD) [5]. The risk of CHD in patients with RA is highly associated with the traditional risk factors of CHD, which include hypertension, diabetes mellitus (DM), hyperlipidemia, smoking, obesity, and physical inactivity [6]. Chronic inflammation and immune dysregulation play an important role in inducing atherosclerosis and endothelial dysfunction in patients with RA [7]. Among current pharmacotherapy of RA, methotrexate (MTX) might prevent ischemic cardiovascular disorders in patients with RA [8]. Etanercept, a tumor necrosis factor-α (TNF-α) inhibitor, was demonstrated to attenuate endothelial dysfunction in a rat model [9]. These findings suggest that anti-inflammatory therapy may reduce the CHD risk in patients with RA.

A previous cohort study revealed that approximately 27.3% of patients with RA were traditional Chinese medicine (TCM) users in Taiwan, and approximately 23.6% of TCM users with RA received acupuncture [10]. The anti-inflammatory mechanisms of acupuncture have been demonstrated in murine models [11, 12]. Previous clinical trials have demonstrated that acupuncture suppressed inflammation in patients with RA [13, 14]. However, no prior study evaluated whether acupuncture can prevent CHD in patients with RA. To investigate whether acupuncture decreases the incidence of CHD in patients with RA, we conducted this cohort study.

Methods

Data sources

The Taiwanese National Health Insurance Research Database (NHIRD) is a good database for the cohort study because it contains the long-term follow-up data of most of the population in Taiwan. The National Health Insurance (NHI) program was implemented in 1995 in Taiwan by the National Health Insurance Administration. TCM services have been reimbursed through the NHI since 1996 [15]. The reimbursed population reached more than 99.5% in 2015 [16]. NHIRD is a big database that can prevent sampling bias [17].

The data source of this study was the Registry of Catastrophic Illness Patients Database; patients with autoimmune diseases, such as RA [18], systemic lupus erythematosus, multiple sclerosis, Sjögren syndrome, and cancer, are included in this database, which is a subsection of the NHIRD [19]. All of the patients with RA in the database are given catastrophic illness certificates based on the clinical and laboratory diagnoses by rheumatologists. The NHI program waived RA patients’ copayments for receipt of RA-related treatments, including Western medicine and TCM. In this database, the International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) codes were the most commonly used diagnostic codes. Because the NHIRD contains identified secondary data for research, the present study was waived from informed consent. This study was approved by the Research Ethics Committee of China Medical University and Hospital, Taichung, Taiwan (CMUH104-REC2–115).

Study cohort identification and covariates

Patients who were newly diagnosed with RA (ICD-9-CM code 714.0) between January 1, 1997 and December 31, 2010 were identified, as shown in Fig. 1. Patients younger than 18 years old or with incomplete information on the age and sex were excluded. Patients who received acupuncture between the initial diagnosis of RA and happening of CHD or December 31, 2010 were identified as the acupuncture cohort. In the acupuncture cohort, we defined the index date as the first time that the patients started to receive acupuncture. The immortal time of the acupuncture cohort was the period between the initial diagnosis of RA and the index date. Patients who suffered from CHD were identified through ICD-9-CM codes 410, 411, 413, 414.0, 414.8, and 414.9. Patients diagnosed as having CHD before the index date or within one year after their initial diagnosis of RA were excluded. Patients were followed up until December 31, 2011 or withdrawn from the NHIRD.

Fig. 1.

Fig. 1

Study population flowchart diagram. Using the inclusion and exclusion criteria, we identified 10,199 patients who underwent acupuncture and 19,542 patients who never underwent acupuncture. After performing 1:1 propensity score matching, there were 9932 patients in the acupuncture and no-acupuncture cohorts

To reduce confounding factors, we used 1:1 propensity-score matching by sex, age (per 5 years), comorbidity, conventional drug use, the initial diagnostic year of RA, and the index year through multiple logistic regression analysis. The comorbidities analyzed in our study included DM (ICD-9-CM: 250), hypertension (ICD-9-CM: 401–405), hyperlipidemia (ICD-9-CM: 272), congestive heart failure (ICD-9-CM: 402.01, 402.11, 402.91, 404.01, 404.03, 404.11, 404.13, 404.91, 404.93, and 428.0), cerebral vascular diseases (ICD-9-CM: 430–438), depression (ICD-9-CM: 296.2–296.3, 300.4, and 311), anxiety (ICD-9-CM: 300.0, 300.2, 300.3, 308.3, and 308.91), alcoholism (ICD-9-CM: 291, 303, 305.00–305.03, 790.3, and V11.3), tobacco dependence (ICD-9-CM: 305.1), and obesity (ICD-9-CM: 278 and A183). Patients with comorbidity were identified if the ICD-9-CM codes of the selected comorbidity appeared two or more times in the outpatient or inpatient claims. The conventional drugs analyzed in our study included MTX, hydroxychloroquine, sulfasalazine, TNF-α inhibitors, oral steroids, NSAIDs, and statins. Finally, equal numbers of patients in the acupuncture and no-acupuncture cohorts were analyzed in this study. Patients were divided into 3 groups according to their age: 18–39 years old, 40–59 years old, and older than 60 years old.

Types of acupuncture and disease categories in the acupuncture cohort

We analyzed the acupuncture type that patients received by the treatment codes, which included manual acupuncture of TCM type (B41, B42, B45, B46, B80-B84, B90-B94, P27041, P31103, P32103, and P33031) and electroacupuncture (B43, B44, B86–89, and P33032). Figure 2 illustrated these two types of acupuncture treatments.

Fig. 2.

Fig. 2

Diagram of the two types of acupuncture treatment. a Manual acupuncture of traditional Chinese medicine type (b) Electroacupuncture

Statistical analyses

Statistical analysis was performed using SAS 9.4 (SAS Institute, Cary, NC, U.S.A.), and p < 0.05 in two-tailed tests indicate statistical significance. The difference of baseline characteristics between the acupuncture and no-acupuncture cohorts was expressed as standardized mean differences. Standardized mean differences with less than 0.2 s.d. indicated a negligible difference in the mean or proportions between the two cohorts. For each variable, we used Cox proportional hazard regression to analyze the hazard ratios (HRs) and 95% confidence interval (95% CI). The Kaplan-Meier method was used to determine the cumulative incidence of CHD in both cohorts, and the log-rank test was used to compare incidence curves between the acupuncture and no-acupuncture cohorts.

Results

The identified patient numbers in both cohorts were equal after performing the 1:1 propensity score matched method (Fig. 1). There were no significant differences in the sex, age, baseline comorbidity, and conventional drug use between the no-acupuncture and acupuncture cohorts (Table 1). The included patients were female predominant with a mean age at approximately 50 years old in both cohorts. Almost all patients in both cohorts have used NSAIDs, and more than 90% of patients have also used oral steroids. Approximately 15% of patients took statins in both the no-acupuncture and acupuncture cohorts. Manual acupuncture of the TCM type was the most commonly used acupuncture treatment.

Table 1.

Characteristics of rheumatoid arthritis patients according to whether they received acupuncture treatment

Variable Rheumatoid arthritis
Underwent acupuncture
Standardized mean difference
No (n = 9932) Yes (n = 9932)
n % n %
Sex
 Female 8212 82.68 8225 82.81 0.003
 Male 1720 17.32 1707 17.19 0.003
Age group
 18–39 2267 22.83 2013 20.27 0.003
 40–59 6099 61.41 6659 67.05 0.003
 ≥60 1566 15.77 1260 12.69 0.003
 Mean ± SD (years) 50.39 (13.7) 50.36 (12.53) 0.002
Baseline Comorbidity
 Diabetes mellitus 1211 12.19 1193 12.01 0.006
 Hypertension 2358 23.74 2339 23.55 0.005
 Hyperlipidemia 1488 14.98 1475 14.85 0.004
 Congestive heart failure 94 0.95 107 1.08 0.013
 Stroke 741 7.46 731 7.36 0.004
 Depression 631 6.35 618 6.22 0.005
 Anxiety 1525 15.35 1499 15.09 0.007
 Alcoholism 20 0.20 24 0.24 0.009
 Tobacco dependence 27 0.27 30 0.30 0.006
 Obesity 80 0.81 77 0.78 0.003
Drug use
 Methotrexate 7131 71.80 7130 71.79 0
 Hydroxychloroquine 8372 84.29 8423 84.81 0.014
 Sulfasalazine 7289 73.39 7278 73.28 0.003
 TNF-α inhibitors 1137 11.45 1151 11.59 0.004
 NSAIDs 9908 99.76 9909 99.77 0.002
 Oral steroids 9449 95.14 9438 95.03 0.005
 Statins 1449 14.59 1438 14.48 0.003
Types of acupuncture
 Manual acupuncture of TCM type 8614 86.73
 Electroacupuncture 340 3.42
 Combination of manual acupuncture and electroacupuncture 978 9.85
Immortal time, days (mean, median) 1064.07 (989.63) 1044.96 (1056.50)
Acupuncture visits, mean 9.81
Follow-up time, years (mean, median) 4.02 (3.03) 4.91 (3.44)

During the follow-up period, 909 patients in the acupuncture cohort and 1233 patients in the no-acupuncture cohort developed CHD (Table 2). Compared with the no-acupuncture cohort, the incidence of CHD was significantly lower in the acupuncture cohort (adjusted HR = 0.60, 95% CI = 0.55–0.65). RA patients with old age, DM, or hypertension were prone to developing CHD.

Table 2.

Cox model with hazard ratios and 95% confidence intervals for coronary heart disease in patients with rheumatoid arthritis who did or did not receive acupuncture treatment

Variable No. of event
(n = 2142)
Crudea Adjustedb
HR (95%CI) p-value HR (95%CI) p-value
Acupuncture
 No 1233 1.00 reference 1.00 reference
 Yes 909 0.63 (0.58–0.69) <.0001 0.60 (0.55–0.65) <.0001
Sex
 Female 1692 1.00 reference 1.00 reference
 Male 440 1.32 (1.19–1.47) <.0001 1.21 (1.09–1.34) 0.0005
Age group
 18–39 126 1.00 reference 1.00 reference
 40–59 1458 4.22 (3.52–5.07) <.0001 3.85 (3.20–4.63) <.0001
 ≥60 548 8.76 (7.22–10.64) <.0001 5.35 (4.37–6.56) <.0001
Baseline Comorbidity (ref = non-site comorbidity)
 Diabetes mellitus 443 2.25 (2.03–2.50) <.0001 1.46 (1.30–1.64) <.0001
 Hypertension 877 2.65 (2.43–2.89) <.0001 1.77 (1.61–1.95) <.0001
 Hyperlipidemia 391 1.64 (1.47–1.83) <.0001 1.08 (0.95–1.22) 0.2342
 Congestive heart failure 33 1.94 (1.37–2.73) 0.0002 1.00 (0.71–1.42) 0.9783
 Cerebral vascular diseases 267 2.09 (1.84–2.38) <.0001 1.22 (1.07–1.40) 0.0037
 Depression 127 1.18 (0.99–1.42) 0.0664 0.97 (0.80–1.17) 0.7509
 Anxiety 312 1.20 (1.07–1.36) 0.0028 0.94 (0.83–1.07) 0.3402
 Alcoholism 2 0.54 (0.14–2.17) 0.3886 0.51 (0.13–2.05) 0.3427
 Tobacco dependence 1 0.25 (0.04–1.74) 0.1606 0.22 (0.03–1.57) 0.1316
 Obesity 18 1.38 (0.87–2.20) 0.171 1.15 (0.72–1.83) 0.5623
Drug use
 Methotrexate 1210 0.47 (0.43–0.51) <.0001 0.64 (0.58–0.70) <.0001
 Hydroxychloroquine 1680 0.60 (0.55–0.67) <.0001 0.80 (0.720–.89) <.0001
 Sulfasalazine 1437 0.66 (0.61–0.73) <.0001 0.91 (0.82–1.00) 0.0559
 TNF-α inhibitors 105 0.35 (0.29–0.43) <.0001 0.48 (0.39–0.59) <.0001
 NSAIDs 2130 1.36 (0.34–5.44) 0.6638 1.73 (0.43–6.93) 0.4412
 Oral steroids 2010 0.66 (0.55–0.79) <.0001 0.82 (0.68–0.98) 0.033
 Statins 350 1.07 (0.96–1.20) 0.2408 0.69 (0.61–0.78) <.0001

Crude HRa represents the relative hazard ratio. Adjusted HRb represents the adjusted hazard ratio mutually adjusted for acupuncture use, age, sex, diabetes mellitus, hypertension, hyperlipidemia, congestive heart failure, cerebral vascular diseases, depression, anxiety, alcoholism, tobacco dependence, obesity, Methotrexate, Hydroxychloroquine, Sulfasalazine, TNF-α inhibitors, oral steroids, NSAIDs and statins in Cox proportional hazard regression

The beneficial effect of acupuncture on the incidence of CHD was observed in both women and men (Table 3, adjusted HR = 0.62 in women, 95% CI = 0.56–0.68; adjusted HR = 0.58 in men, 95% CI = 0.48–0.70). Acupuncture significantly decreased the incidence of CHD in all age groups. Regardless of whether patients had DM, hypertension, hyperlipidemia, or stroke, acupuncture lowered the risk of CHD. Moreover, the beneficial effect of acupuncture in preventing CHD was independent of conventional drug use, including statins, MTX, hydroxychloroquine, sulfasalazine, oral steroids, and TNF-α inhibitors. Overall, the cumulative incidence of CHD from the index date was significantly lower in the acupuncture cohort (Fig. 3).

Table 3.

Incidence rates, hazard ratio and confidence intervals for coronary heart disease in rheumatoid arthritis patients who did or did not receive acupuncture treatment, stratified by sex, age, comorbidity and drug use

Variables Rheumatoid arthritis Compared with no-acupuncture cohort
No acupuncture
(n = 9932)
Acupuncture
(n = 9932)
Crude HR
(95%CI)
Adjusted HR
(95%CI)
Event Person-years IR Event Person-years IR
Total 1223 39,930.25 30.63 909 48,742.08 18.65 0.63(0.58–0.69)*** 0.60(0.55–0.65)***
Sex
 Female 970 33,542.81 28.92 722 40,610.75 17.78 0.64(0.58–0.7)*** 0.62(0.56–0.68)***
 Male 253 6387.44 39.61 187 8131.32 23.00 0.60(0.50–0.73)*** 0.58(0.48–0.70)***
Age group
 18–39 81 10,544.53 7.68 45 10,665.24 4.22 0.56(0.39–0.81)** 0.53(0.37–0.77)***
 40–59 816 24,725.99 33.00 642 32,628.29 19.68 0.61(0.55–0.68)*** 0.59(0.53–0.66)***
 ≥60 326 4659.72 69.96 222 5448.55 40.74 0.61(0.51–0.72)*** 0.62(0.52–0.73)***
Baseline Comorbidity
Diabetes mellitus
 No 969 36,055.56 26.88 720 43,658.67 16.49 0.64(0.58–0.70)*** 0.60(0.55–0.66)***
 Yes 254 3874.69 65.55 189 5083.41 37.18 0.58(0.48–0.71)*** 0.59(0.49–0.72)***
Hypertension
 No 717 32,065.40 22.36 538 38,485.07 13.98 0.64(0.57–0.72)*** 0.62(0.56–0.70)***
 Yes 506 7864.85 64.34 371 10,257.00 36.17 0.59(0.51–0.67)*** 0.58(0.51–0.67)***
Hyperlipidemia
 No 1000 35,475.99 28.19 741 43,080.05 17.20 0.63(0.58–0.70)*** 0.61(0.56–0.67)***
 Yes 223 4454.26 50.06 168 5662.03 29.67 0.59(0.49–0.73)*** 0.57(0.47–0.70)***
Congestive heart failure
 No 1202 39,641.02 30.32 897 48,354.94 18.55 0.63(0.58–0.69)*** 0.61(0.56–0.66)***
 Yes 21 289.23 72.61 12 387.14 31.00 0.44(0.22–0.90)* 0.50(0.24–1.07)
Cerebral vascular diseases
 No 1069 37,549.80 28.47 796 45,604.09 17.45 0.64(0.58–0.70)*** 0.60(0.55–0.66)***
 Yes 154 2380.45 64.69 113 3137.99 36.01 0.58(0.45–0.74)*** 0.59(0.46–0.76)***
Depression
 No 1144 38,013.07 30.09 861 46,405.65 18.55 0.64(0.59–0.70)*** 0.62(0.56–0.67)***
 Yes 79 1917.19 41.21 48 2336.42 20.54 0.51(0.36–0.73)*** 0.51(0.35–0.73)***
Anxiety
 No 1042 35,170.64 29.63 778 43,056.56 18.07 0.63(0.58–0.69)*** 0.60(0.55–0.66)***
 Yes 181 4759.61 38.03 131 5685.51 23.04 0.62(0.50–0.78)*** 0.62(0.49–0.78)***
Alcoholism
 No 1222 39,865.91 1222 908 48,660.02 908 0.63(0.58–0.69)*** 0.61(0.56–0.66)***
 Yes 1 64.34 1 1 82.05 1 0.58(0.03–9.89)
Tobacco dependence
 No 1222 39,862.01 1222 909 48,661.38 909 0.63(0.58–0.69)*** 0.61(0.56–0.66)***
 Yes 1 68.24 1 0 80.70 0
Obesity
 No 1214 39,697.39 30.58 900 48,463.26 18.57 0.63(0.58–0.69)*** 0.61(0.55–0.66)***
 Yes 9 232.86 38.65 9 278.81 32.28 0.95(0.37–2.42) 0.74(0.26–2.09)
Drug use
Methotrexate
 No 530 10,045.02 52.76 392 13,042.58 30.06 0.60(0.53–0.68)*** 0.61(0.54–0.7)***
 Yes 693 29,885.23 23.19 517 35,699.50 14.48 0.64(0.57–0.72)*** 0.60(0.53–0.67)***
Hydroxychloroquine
 No 257 5500.43 46.72 195 6736.19 28.95 0.65(0.54–0.78)*** 0.62(0.52–0.75)***
 Yes 966 34,429.82 28.06 714 42,005.89 17 0.63(0.57–0.69)*** 0.60(0.54–0.66)***
Sulfasalazine
 No 410 9296.59 44.10 285 11,957.00 23.84 0.57(0.49–0.66)*** 0.54(0.47–0.63)***
 Yes 813 30,633.66 26.54 624 36,785.07 16.96 0.66(0.59–0.73)*** 0.64(0.57–0.71)***
TNF-α inhibitors
 No 1157 34,731.45 33.31 870 42,448.62 20.50 0.64(0.59–0.7)*** 0.62(0.56–0.67)***
 Yes 66 5198.80 12.70 39 6293.45 6.20 0.48(0.32–0.72)*** 0.42(0.28–0.62)***
NSAIDs
 No 0 42.45 0 2 62.67 31.91
 Yes 1223 39,887.80 30.66 907 48,679.41 18.63 0.63(0.58–0.69)*** 0.61(0.56–0.66)***
Oral steroids
 No 68 1442.40 47.14 54 1799.74 30 0.69(0.48–1.00)* 0.74(0.51–1.07)
 Yes 1155 38,487.85 30.01 855 46,942.34 18.21 0.63(0.57–0.69)*** 0.60(0.55–0.65)***
Statins
 No 1020 33,516.65 30.43 762 41,308.09 18.45 0.63(0.58–0.69)*** 0.61(0.55–0.67)***
 Yes 203 6413.60 31.65 147 7433.99 19.77 0.63(0.51–0.78)*** 0.59(0.48–0.73)***

Abbreviations: IR, incidence rates per 1000 person-years; HR, hazard ratio; and CI, confidence interval

Adjusted HR: adjusted for acupuncture use, age, sex, diabetes mellitus, hypertension, hyperlipidemia, congestive heart failure, cerebral vascular diseases, depression, anxiety, alcoholism, tobacco dependence, obesity, Methotrexate, Hydroxychloroquine, Sulfasalazine, TNF-α inhibitors, oral steroids, NSAIDs and statins in Cox proportional hazard regression

*: p < 0.05; **: p < 0.01; and ***: p < 0.001

Fig. 3.

Fig. 3

Cumulative incidence of coronary heart disease (CHD) between the acupuncture and no-acupuncture cohorts. The cumulative incidence of coronary heart disease in the acupuncture cohort (dashed line) is significantly lower than the no-acupuncture cohort (solid line) (log-rank test, p < .001). The zero point indicated the index date

Subgroup analysis was performed to evaluate whether different types of acupuncture had any correlation and difference on preventing CHD in patients with RA (Table 4). Patients receiving only manual acupuncture, only electroacupuncture, or combination of manual acupuncture and electroacupuncture all had a lower incidence of CHD than patients never receiving acupuncture.

Table 4.

Cox model for coronary heart disease in patients with rheumatoid arthritis receiving different types of acupuncture treatment vs. not receiving acupuncture treatment

n CHD
(n = 2132)
Crude HR*(95%CI) Adjusted HR (95%CI)
No-acupuncture 9932 1223 1 (reference) 1 (reference)
Acupuncture
Manual acupuncture of TCM type 8614 819 0.67(0.62–0.74)*** 0.64(0.58–0.69)***
Electroacupuncture 340 19 0.48(0.31–0.76)** 0.46(0.29–0.73)**
Combination of manual acupuncture and electroacupuncture 978 71 0.39(0.30–0.49)*** 0.36(0.28–0.46)***

Abbreviations: HR, hazard ratio; CHD, coronary heart disease; CI, confidence interval; and TCM, traditional Chinese medicine

Crude HR* represents relative hazard ratio. Adjusted HR represents adjusted hazard ratio mutually adjusted for age, sex, diabetes mellitus, hypertension, hyperlipidemia, congestive heart failure, cerebral vascular diseases, depression, anxiety, alcoholism, tobacco dependence, obesity, Methotrexate, Hydroxychloroquine, Sulfasalazine, TNF-α inhibitors, oral steroids, NSAIDs and statins in Cox proportional hazard regression

**: p < 0.01; and ***: p < 0.001

Among the different types of CHD, patients with acupuncture had a significant lower incidence of chronic ischemic heart disease (Table 5). However, there was no significant difference in the incidence of angina pectoris between the two groups. The effect on acute myocardial infarction was not analyzed (data not shown; only 11 patients had acute myocardial infarction in two groups).

Table 5.

Cox model for angina pectoris and chronic ischemic heart disease in patients with rheumatoid arthritis receiving acupuncture treatment vs. not receiving acupuncture treatment

N Angina pectoris Chronic ischemic heart disease
n Crude HR* Adjusted HR n Crude HR* (95%CI) Adjusted HR (95%CI)
Acupuncture
 No 8733 24 1 (reference) 1 (reference) 516 1 (reference) 1 (reference)
 Yes 9044 21 0.79(0.44–1.43) 0.72(0.4–1.3) 358 0.59(0.52–0.68)*** 0.56(0.49 − 0.64)***

Crude HR* represents relative hazard ratio. Adjusted HR represents adjusted hazard ratio mutually adjusted for age, sex, diabetes mellitus, hypertension, hyperlipidemia, congestive heart failure, cerebral vascular diseases, depression, anxiety, alcoholism, tobacco dependence, obesity, Methotrexate, Hydroxychloroquine, Sulfasalazine, TNF-α inhibitors, oral steroids, NSAIDs and statins in Cox proportional hazard regression

***: p < 0.001

Table 6 showed the top five disease categories, which represented the reasons for clinical visits, in the acupuncture cohort. Most patients with RA received acupuncture because of musculoskeletal disease.

Table 6.

The top five disease categories (as the reasons for clinical visits) in acupuncture user cohort

Disease (ICD-9-CM) Acupuncture users
(n = 9932)
n %
Musculoskeletal system and connective tissue (710–739) 6885 69.32
Injury (800–999) 5377 54.14
Symptoms, signs and ill-defined conditions (780–799) 700 7.05
Nervous system (320–389) 463 4.66
Digestive system (520–579) 302 3.04

Maximal of three ICD-9-CM codes could be recorded in one visit

Discussion

To the best of our knowledge, this nationwide population-based study is the first report to reveal that acupuncture decreased the risk of CHD in patients with RA. Our study found that the beneficial effects of acupuncture for developing CHD in patients with RA were independent of sex, age, DM, and hypertension. Moreover, acupuncture decreased the risk of CHD in patients with RA who took statins, which was reported to reduce the risk of cardiovascular diseases [20]. Among different types of CHD, our results demonstrated that patients with acupuncture had a significant lower incidence of chronic ischemic heart disease than patients without receiving acupuncture treatment.

The risk of CHD in patients with RA is highly associated with the conventional risk factors of CHD, such as hypertension, DM, hyperlipidemia, and physical inactivity [6]. Our study also demonstrated that RA patients with comorbidities, including DM, hypertension, or hyperlipidemia, had a higher risk of CHD. Even in RA patients with DM, hypertension, or hyperlipidemia, our study indicated that acupuncture decreased the incidence of CHD. Previous clinical trials indicated that acupuncture improved swelling and range of motion of knee joints, physical activity, and quality of life in patient with RA [14, 21, 22]. Improved physical activity might be one of the possible explanations that acupuncture can prevent CHD in patients with RA. However, we also found that for those RA patients with the comorbidity of congestive heart failure, acupuncture did not reduce the risk of CHD.

The previous study stated that endothelial dysfunction contributed to atherosclerosis and induced cardiovascular disorders [23]. The current conventional drugs for RA treatment, including glucocorticoids [24] and etanercept (a TNF-α inhibitor) [9], have been demonstrated to improve endothelial dysfunction through the anti-inflammatory effect in RA rat models. However, the protective effect of glucocorticoids remains controversial because glucocorticoids also enhance the risk of cardiovascular disorders based on their negative effects on blood pressure, lipid, insulin resistance and obesity [25]. The protective effect of MTX at a low dose against CHD in patients with RA was also demonstrated in a previous meta-analysis study although it did not attenuate hyperlipidemia or insulin resistance [8]. Our study also indicated that patients with RA who took MTX or TNF-α inhibitors had a lower risk of CHD. In addition, regardless of whether patients used TNF-α inhibitors or MTX, acupuncture significantly decreased the risk of CHD in those patients.

Chronic inflammation plays an important role in the pathogenesis of RA and its comorbidities, including DM, hypertension, and CHD [7]. As a result, amelioration of inflammatory activity should be considered in the CHD preventive strategy for patients with RA. The anti-inflammatory mechanisms of acupuncture have been evaluated in murine models of inflammatory disorders [11, 12]. Acupuncture corrected the imbalance between Th17 and Treg cells to attenuate the severity of RA in a rat RA model [26]. IL-17, cytokine secreted by Th17 cells, has been demonstrated to be important in the pathogenesis of RA, and it has been demonstrated to be associated with CHD [27]. Further mechanistic studies are needed to evaluate whether acupuncture can suppress IL-17 in RA to prevent CHD. Previous clinical trials have demonstrated that acupuncture suppressed inflammation in patients with RA [13, 14]. Based on these studies, acupuncture not only had a traditionally recognized analgesic effect, it had an anti-inflammatory effect. Anti-inflammation might be one of the mechanisms by which acupuncture decreased the incidence of CHD in RA patients. In addition, previous studies demonstrated that the combination of acupuncture with current conventional treatment had additive effects on hypertensive patients [28] and diabetic patients [29], which could contribute to how acupuncture decreased the risk of CHD in RA patients with comorbidity or conventional treatment. The mechanistic study is needed in the future. We have established a collagen-induced arthritis murine model and designed a clinical trial to investigate whether acupuncture is effective on the immunological modulation in RA.

The strength of this study was the comprehensive large-scale database, Taiwan’s NHIRD, which provides a large sample size and reduces selection bias while including long-term follow-up data [17]. Although there were a few clinical trials investigating the efficacy of acupuncture for the treatment of patients with RA [13, 14, 21, 22, 3035], none of them addressed the question whether acupuncture treatment reduces the risk of cardiovascular disease among patients with RA.

Some limitations merit attention in our study. First, the detailed information about severity of RA, including pain scores, the severity of inflammation, or the destruction of cartilage and bone were not provided in the database. Because patients with different severities may receive different conventional treatments, we performed a 1:1 propensity score matching to minimize this confounding factor. We found that there were no difference in percentages of patients who used analgesic drugs, including NSAIDs and oral steroids, in the acupuncture and no-acupuncture cohorts. The second limitation is that lifestyle data, such as smoking, alcohol drinking, and body mass index (BMI), were unavailable in the datasets. We were able to acquire information of the illnesses that resulted from these personal habits and lifestyles, including alcoholism, tobacco dependence, and obesity. However, an underestimation of smoking exposure and alcohol drinking in both cohorts still may exist. The third limitation was that the datasets did not provide detailed information of selected acupoints and treatment protocols. Usually, TCM doctors selected acupoints for each patient according to the TCM diagnosis, which may vary during the disease progression. In addition, the duration or visit times of acupuncture in the current study might be underestimated because the NHI program only covers maximal 15 times of acupuncture per month and maximal 2–3 times per week, some patients may use self-pay for additional acupuncture treatment, which could not be identified from the NHIRD database. Future clinical trials should consider comparing the effects of a set of fixed acupoints versus flexible acupoints and different duration/times of acupuncture treatment. Although there are some limitations in the current retrospective cohort study, it’s a “real-world evidence” that can reflect actual use in practice [36] from a comprehensive large-scale database. Evidences gathered from future clinical trials should be able to provide more information regarding the selection of acupoints.

Conclusions

Our study revealed that acupuncture treatment might have beneficial effect on reducing the risk of CHD in patients with RA in Taiwan. This noteworthy finding can provide useful information for further clinical and mechanistic studies.

Acknowledgements

This study was based in part on data from the NHIRD, provided by the NHIA, and the Ministry of Health and Welfare and was managed by National Health Research Institutes. The interpretation and conclusions contained herein do not represent those of the NHIA, Ministry of Health and Welfare, or National Health Research Institutes. The funding agencies had no role in study design, data collection and analyses, and result interpretation and publication of this study.

Funding

This study was supported in part by China Medical University under the Higher Education Sprout Project, Ministry of Education, Taiwan. It was also supported by the Taiwan Ministry of Health and Welfare Clinical Trial and Research Center of Excellence (MOHW107-TDU-B-212-123004), the Health and Welfare surcharge of tobacco product, China Medical University Hospital Cancer Research Center of Excellence (MOHW107-TDU-B-212-114-24) and Department of Medical Research, China Medical University Hospital (DMR-107-004), Taiwan.

Availability of data and materials

All data of this study are deposited in a properly managed public repository. We examined and analysed datasets released from the Taiwan NHIRD (http://nhird.nhri.org.tw/en/index.html), maintained and managed by National Health Research Institutes (http://www.nhri.org.tw/), Taiwan. The use of the datasets is limited to research purposes only. Applicants must follow the Computer-Processed Personal Data Protection Law (http://www.winklerpartners.com/?p=987) and related regulations of National Health Insurance Administration and National Health Research Institutes, and the agreement must be signed by the applicant and his/her supervisor upon application submission. All applications are reviewed for approval of data release.

Abbreviations

95% CI

95% confidence interval

CHD

Coronary heart disease

DM

Diabetes mellitus

DMARDs

Disease-modifying anti-rheumatic drugs

HRs

Hazard ratios

ICD-9-CM

International Classification of Diseases, Ninth Revision, Clinical Modification

MTX

Methotrexate

NHI

National Health Insurance

NHIRD

National Health Insurance Research Database

NSAIDs

Non-steroid anti-inflammatory drugs

RA

Rheumatoid arthritis

TCM

Traditional Chinese medicine

Authors’ contributions

MYW conceived the study design, interpreted the data, and drafted the manuscript. MCH conceived the study design, and interpreted the data. HHL conceived the study design interpreted the data. JHC did the statistical analysis. YCL and CYH interpreted the data. MFS conceived the study design, interpreted the data, supervised the study, and finalized the manuscript. HRY conceived the study design, interpreted the data, supervised the study, and finalized the manuscript. MFS and HRY contributed equally. All authors read and approved the final manuscript.

Ethics approval and consent to participate

This study was approved by the Research Ethics Committee of China Medical University and Hospital, Taichung, Taiwan (CMUH104-REC2–115) and also the National Health Research Institute, the data holder of the NHI database. The patient consent was exempted for the total anonymity of all research data in this study.

Consent for publication

Not applicable in this section.

Competing interests

The funding agencies had no role in study design, data collection, analyses, result interpretation or publication of this study. This study was based in part on data from the NHIRD, provided by the NHIA, and the Ministry of Health and Welfare and was managed by National Health Research Institutes. The interpretation and conclusions contained herein do not represent those of the National Health Insurance Administration, Ministry of Health and Welfare, or National Health Research Institutes. All authors declare that they have no conflicts of interest.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Contributor Information

Mei-Yao Wu, Email: meiyaowu0919@gmail.com.

Ming-Cheng Huang, Email: hmctcm@gmail.com.

Hou-Hsun Liao, Email: a202098@cmu.edu.tw.

Jen-Huai Chiang, Email: zinvii@gmail.com.

Yu-Chen Lee, Email: d5167@mail.cmuh.org.tw.

Chung-Y Hsu, Email: hsuc@mail.cmuh.org.tw.

Mao-Feng Sun, Phone: +886-4-2205-3366, Email: maofeng@mail.cmuh.org.tw.

Hung-Rong Yen, Phone: +886-4-2205-3366, Email: hungrongyen@gmail.com.

References

  • 1.Firestein GS. Evolving concepts of rheumatoid arthritis. Nature. 2003;423(6937):356–361. doi: 10.1038/nature01661. [DOI] [PubMed] [Google Scholar]
  • 2.McInnes IB, Schett G. The pathogenesis of rheumatoid arthritis. N Engl J Med. 2011;365(23):2205–2219. doi: 10.1056/NEJMra1004965. [DOI] [PubMed] [Google Scholar]
  • 3.Smolen JS, Aletaha D, McInnes IB. Rheumatoid arthritis. Lancet. 2016;388(10055):2023–2038. doi: 10.1016/S0140-6736(16)30173-8. [DOI] [PubMed] [Google Scholar]
  • 4.Nossent H. Risk of cardiovascular events and effect on mortality in patients with rheumatoid arthritis. J Rheumatol. 2000;27(9):2282–2283. [PubMed] [Google Scholar]
  • 5.Peters MJ, van Halm VP, Voskuyl AE, Smulders YM, Boers M, Lems WF, Visser M, Stehouwer CD, Dekker JM, Nijpels G, et al. Does rheumatoid arthritis equal diabetes mellitus as an independent risk factor for cardiovascular disease? A prospective study. Arthritis Rheum. 2009;61(11):1571–1579. doi: 10.1002/art.24836. [DOI] [PubMed] [Google Scholar]
  • 6.Baghdadi LR, Woodman RJ, Shanahan EM, Mangoni AA. The impact of traditional cardiovascular risk factors on cardiovascular outcomes in patients with rheumatoid arthritis: a systematic review and meta-analysis. PLoS One. 2015;10(2):e0117952. doi: 10.1371/journal.pone.0117952. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Hollan I, Dessein PH, Ronda N, Wasko MC, Svenungsson E, Agewall S, Cohen-Tervaert JW, Maki-Petaja K, Grundtvig M, Karpouzas GA, et al. Prevention of cardiovascular disease in rheumatoid arthritis. Autoimmun Rev. 2015;14(10):952–969. doi: 10.1016/j.autrev.2015.06.004. [DOI] [PubMed] [Google Scholar]
  • 8.De Vecchis R, Baldi C, Palmisani L. Protective effects of methotrexate against ischemic cardiovascular disorders in patients treated for rheumatoid arthritis or psoriasis: novel therapeutic insights coming from a meta-analysis of the literature data. Anatol J Cardiol. 2016;16(1):2–9. doi: 10.5152/akd.2015.6136. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Totoson P, Maguin-Gate K, Prigent-Tessier A, Monnier A, Verhoeven F, Marie C, Wendling D, Demougeot C. Etanercept improves endothelial function via pleiotropic effects in rat adjuvant-induced arthritis. Rheumatology (Oxford) 2016;55(7):1308–1317. doi: 10.1093/rheumatology/kew062. [DOI] [PubMed] [Google Scholar]
  • 10.Huang MC, Pai FT, Lin CC, Chang CM, Chang HH, Lee YC, Sun MF, Yen HR. Characteristics of traditional Chinese medicine use in patients with rheumatoid arthritis in Taiwan: a nationwide population-based study. J Ethnopharmacol. 2015;176:9–16. doi: 10.1016/j.jep.2015.10.024. [DOI] [PubMed] [Google Scholar]
  • 11.Liao HH, Yeh CC, Lin CC, Chen BC, Yeh MH, Chang KM, Sun MF, Yen HR. Prescription patterns of Chinese herbal products for patients with fractures in Taiwan: a nationwide population-based study. J Ethnopharmacol. 2015;173:11–19. doi: 10.1016/j.jep.2015.07.014. [DOI] [PubMed] [Google Scholar]
  • 12.Torres-Rosas R, Yehia G, Pena G, Mishra P, Del Rocio Thompson-Bonilla M, Moreno-Eutimio MA, Arriaga-Pizano LA, Isibasi A, Ulloa L. dopamine mediates vagal modulation of the immune system by electroacupuncture. Nat Med. 2014;20(3):291–295. doi: 10.1038/nm.3479. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Ouyang BS, Gao J, Che JL, Zhang Y, Li J, Yang HZ, Hu TY, Yang M, Wu YJ, Ji LL. Effect of electro-acupuncture on tumor necrosis factor-alpha and vascular endothelial growth factor in peripheral blood and joint synovia of patients with rheumatoid arthritis. Chin J Integr Med. 2011;17(7):505–509. doi: 10.1007/s11655-011-0783-2. [DOI] [PubMed] [Google Scholar]
  • 14.Sato M, Inubushi M, Shiga T, Hirata K, Okamoto S, Kamibayashi T, Tanimura K, Tamaki N. Therapeutic effects of acupuncture in patients with rheumatoid arthritis: a prospective study using (18)F-FDG-PET. Ann Nucl Med. 2009;23(3):311–316. doi: 10.1007/s12149-009-0238-4. [DOI] [PubMed] [Google Scholar]
  • 15.Chang CC, Lee YC, Lin CC, Chang CH, Chiu CD, Chou LW, Sun MF, Yen HR. Characteristics of traditional Chinese medicine usage in patients with stroke in Taiwan: a nationwide population-based study. J Ethnopharmacol. 2016;186:311–321. doi: 10.1016/j.jep.2016.04.018. [DOI] [PubMed] [Google Scholar]
  • 16.NHIA . National Health Insurance Annu Rep 2015–2016. Taiwan: Taipie: National Health Insurance Administration, Ministry of Health and Welfare; 2015. [Google Scholar]
  • 17.Hsing AW, Ioannidis JP. Nationwide population science: lessons from the Taiwan National Health Insurance Research Database. JAMA Intern Med. 2015;175(9):1527–1529. doi: 10.1001/jamainternmed.2015.3540. [DOI] [PubMed] [Google Scholar]
  • 18.Hung YM, Lin L, Chen CM, Chiou JY, Wang YH, Wang PY, Wei JC. The effect of anti-rheumatic medications for coronary artery diseases risk in patients with rheumatoid arthritis might be changed over time: a nationwide population-based cohort study. PLoS One. 2017;12(6):e0179081. doi: 10.1371/journal.pone.0179081. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Chen YJ, Chang YT, Wang CB, Wu CY. The risk of cancer in patients with rheumatoid arthritis: a nationwide cohort study in Taiwan. Arthritis Rheum. 2011;63(2):352–358. doi: 10.1002/art.30134. [DOI] [PubMed] [Google Scholar]
  • 20.LaRosa JC, Pedersen TR, Somaratne R, Wasserman SM. Safety and effect of very low levels of low-density lipoprotein cholesterol on cardiovascular events. Am J Cardiol. 2013;111(8):1221–1229. doi: 10.1016/j.amjcard.2012.12.052. [DOI] [PubMed] [Google Scholar]
  • 21.Lee H, Lee JY, Kim YJ, Kim S, Yin C, Khil JH, Kwon K, Choi SM, Lee H, Park HJ. Acupuncture for symptom management of rheumatoid arthritis: a pilot study. Clin Rheumatol. 2008;27(5):641–645. doi: 10.1007/s10067-007-0819-3. [DOI] [PubMed] [Google Scholar]
  • 22.Zanette Sde A, Born IG, Brenol JC, Xavier RM. A pilot study of acupuncture as adjunctive treatment of rheumatoid arthritis. Clin Rheumatol. 2008;27(5):627–635. doi: 10.1007/s10067-007-0759-y. [DOI] [PubMed] [Google Scholar]
  • 23.Steyers CM, 3rd, Miller FJ Jr.: Endothelial dysfunction in chronic inflammatory diseases. Int J Mol Sci 2014, 15(7):11324–11349. [DOI] [PMC free article] [PubMed]
  • 24.Verhoeven F, Totoson P, Maguin-Gate K, Prigent-Tessier A, Marie C, Wendling D, Moretto J, Prati C, Demougeot C. Glucocorticoids improve endothelial function in rheumatoid arthritis: a study in rats with adjuvant-induced arthritis. Clin Exp Immunol. 2017;188(2):208–218. doi: 10.1111/cei.12938. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Amaya-Amaya J, Sarmiento-Monroy JC, Mantilla RD, Pineda-Tamayo R, Rojas-Villarraga A, Anaya JM. Novel risk factors for cardiovascular disease in rheumatoid arthritis. Immunol Res. 2013;56(2–3):267–286. doi: 10.1007/s12026-013-8398-7. [DOI] [PubMed] [Google Scholar]
  • 26.Zhu J, Chen XY, Li LB, Yu XT, Zhou Y, Yang WJ, Liu Z, Zhao N, Fu C, Zhang SH, et al. Electroacupuncture attenuates collagen-induced arthritis in rats through vasoactive intestinal peptide signalling-dependent re-establishment of the regulatory T cell/T-helper 17 cell balance. Acupunct Med. 2015;33(4):305–311. doi: 10.1136/acupmed-2014-010732. [DOI] [PubMed] [Google Scholar]
  • 27.Qi Y, Feng W, Song A, Song H, Yan S, Sun Q, Yang P. Role of serum IL-23/IL-17 axis in the relationship between periodontitis and coronary heart disease. Int J Periodontics Restorative Dent. 2013;33(2):185–191. doi: 10.11607/prd.1327. [DOI] [PubMed] [Google Scholar]
  • 28.Leem J. Acupuncture to treat hypertension: a recent systematic review and implications for subsequent research. Integr Med Res. 2016;5(1):69–71. doi: 10.1016/j.imr.2015.12.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Firouzjaei A, Li GC, Wang N, Liu WX, Zhu BM. Comparative evaluation of the therapeutic effect of metformin monotherapy with metformin and acupuncture combined therapy on weight loss and insulin sensitivity in diabetic patients. Nutr Diabetes. 2016;6:e209. doi: 10.1038/nutd.2016.16. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Kim ST, Kim YJ, Lee JY, Lee H, Yin CS, Choi SM, Chae Y, Lee H, Park HJ. Acupuncture enhances superoxide dismutase (SOD) activity in the serum of rheumatoid arthritis patients. Clin Exp Rheumatol. 2009;27(2):384. [PubMed] [Google Scholar]
  • 31.Bernateck M, Becker M, Schwake C, Hoy L, Passie T, Parlesak A, Fischer MJ, Fink M, Karst M. Adjuvant auricular electroacupuncture and autogenic training in rheumatoid arthritis: a randomized controlled trial. Auricular acupuncture and autogenic training in rheumatoid arthritis. Forsch Komplementmed. 2008;15(4):187–193. doi: 10.1159/000141929. [DOI] [PubMed] [Google Scholar]
  • 32.Tam LS, Leung PC, Li TK, Zhang L, Li EK. Acupuncture in the treatment of rheumatoid arthritis: a double-blind controlled pilot study. BMC Complement Altern Med. 2007;7:35. doi: 10.1186/1472-6882-7-35. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Wang R, Jiang C, Lei Z, Yin K. The role of different therapeutic courses in treating 47 cases of rheumatoid arthritis with acupuncture. J Tradit Chin Med. 2007;27(2):103–105. [PubMed] [Google Scholar]
  • 34.David J, Townsend S, Sathanathan R, Kriss S, Dore CJ. The effect of acupuncture on patients with rheumatoid arthritis: a randomized, placebo-controlled cross-over study. Rheumatology (Oxford) 1999;38(9):864–869. doi: 10.1093/rheumatology/38.9.864. [DOI] [PubMed] [Google Scholar]
  • 35.Man SC, Baragar FD. Preliminary clinical study of acupuncture in rheumatoid arthritis. J Rheumatol. 1974;1(1):126–129. [PubMed] [Google Scholar]
  • 36.Sherman RE, Anderson SA, Dal Pan GJ, Gray GW, Gross T, Hunter NL, LaVange L, Marinac-Dabic D, Marks PW, Robb MA, et al. Real-world evidence - what is it and what can it tell us? N Engl J Med. 2016;375(23):2293–2297. doi: 10.1056/NEJMsb1609216. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Data Availability Statement

All data of this study are deposited in a properly managed public repository. We examined and analysed datasets released from the Taiwan NHIRD (http://nhird.nhri.org.tw/en/index.html), maintained and managed by National Health Research Institutes (http://www.nhri.org.tw/), Taiwan. The use of the datasets is limited to research purposes only. Applicants must follow the Computer-Processed Personal Data Protection Law (http://www.winklerpartners.com/?p=987) and related regulations of National Health Insurance Administration and National Health Research Institutes, and the agreement must be signed by the applicant and his/her supervisor upon application submission. All applications are reviewed for approval of data release.


Articles from BMC Complementary and Alternative Medicine are provided here courtesy of BMC

RESOURCES