Cumulative Meta‐Analysis of the Soy Effect Over Time

Abstract

Background

Soy protein foods have attracted attention as useful plant protein foods with mild cholesterol‐lowering effects that are suitable for inclusion in therapeutic diets. But on the basis of the lack of consistency in significant cholesterol reduction by soy in 46 randomized controlled trials, the US Food and Drug Administration (FDA) is reassessing whether the 1999 heart health claim for soy protein should be revoked.

Methods and Results

We have, therefore, performed a cumulative meta‐analysis on the 46 soy trials identified by the FDA to determine if at any time, since the 1999 FDA final rule that established the soy heart health claim, the soy effect on serum cholesterol lost significance. The cumulative meta‐analysis for both total cholesterol and low‐density lipoprotein cholesterol demonstrated preservation of the small, but significant, reductions seen both before and during the subsequent 14 years since the health claim was originally approved. For low‐density lipoprotein cholesterol, the mean reduction in 1999 was −6.3 mg/dL (95% CI, −8.7 to −3.9 mg/dL; P=0.00001) and remained in the range of −4.2 to −6.7 mg/dL (P=0.0006 to P=0.0002, respectively) in the years after 1999. At no time point did the total cholesterol or low‐density lipoprotein cholesterol reductions lose significance or were the differences at individual time points in the cumulative meta‐analysis significantly different from those seen in 1999 when the health claim was approved.

Conclusions

A cumulative meta‐analysis of the data selected by the FDA indicates continued significance of total cholesterol and low‐density lipoprotein cholesterol reduction after soy consumption and supports the rationale behind the original soy FDA heart health claim.

Short abstract

See Editorial Petersen and Kris‐Etherton

Clinical Perspective

What Is New?

• The 1999 US Food and Drug Administration heart health claim for soy is now challenged despite the fact that similar heart health claims are allowed for nuts and viscous fibers (ie, oats, barley, and psyllium), with equally modest cholesterol‐lowering ability, and also despite the fact that all these foods have entered dietary guidelines for cholesterol control in other jurisdictions.

What Are the Clinical Implications?

• We, therefore, assessed, using a cumulative meta‐analysis, whether at any time point since 1999 had soy foods failed to lower serum cholesterol and found that LDL cholesterol reductions for soy protein have consistently been between −4.2 and −6.7 mg/dL (P<0.006), with no loss of significance at any time point, so justifying the continued use of soy for health and therapeutic purposes as part of cholesterol‐lowering diets.

Introduction

The US Food and Drug Administration (FDA) has proposed to revoke the heart health claim status for soy protein foods1 that it originally granted in 1999.2 For almost 40 years, there has been interest in soy protein as a food source with cholesterol‐lowering properties, starting with the large reductions in serum cholesterol, as seen in the early feeding studies of hypercholesterolemic patients by Sirtori et al.3 Later, this cholesterol‐lowering effect of soy was explored in a meta‐analysis by Anderson and colleagues in 1995 that demonstrated a dramatic 13% reduction in low‐density lipoprotein cholesterol (LDL‐C).4 Subsequent meta‐analyses have shown more moderate LDL‐C reductions, but overall the reductions have remained significant.5, 6, 7 Concern has been expressed that the effects of soy on LDL‐C are too modest to be clinically significant.8 However, trials that have combined several cholesterol‐lowering foods that included soy in a “portfolio” provided under metabolically controlled conditions have achieved statin‐like LDL‐C reductions.9

Nevertheless, the FDA, without a formal meta‐analysis, concluded that the variability in the results of soy trials was too great.1 Further trials demonstrating a significant LDL‐C reduction were too few. As a result, the FDA proposed that, subject to comment, the health claim should be revoked.1

We have, therefore, performed a cumulative meta‐analysis to demonstrate whether, and if so when, a significant effect of soy on cholesterol was lost.

Methods

This cumulative meta‐analysis was conducted according to the Cochrane Handbook for Systematic Reviews and Interventions.10 Standard meta‐analysis, heterogeneity, sensitivity, and risk of bias analysis are reported elsewhere.11 This report focuses on the cumulative meta‐analysis12 to answer the question of if and when the soy effect on total cholesterol (TC) and LDL‐C was lost. Results have been reported in accordance with the Preferred Reporting Items for Systematic Reviews and Meta‐Analyses13 guidelines. The authors declare that all supporting data are available within the article or within the online supplementary files of the standard meta‐analysis referred to above.11

Data Sources and Study Selection

We included all 46 trials14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59 that the FDA selected1 for the reanalysis.

Data Extraction

Data were extracted by at least 2 independent reviewers (S.B.M., L.C., E.V., or S.S.L.) and entered on a standard proforma. The outcome data included the mean difference between test and control in LDL‐C and TC. Plot Digitizer, version 2.6.8,60 was used to extract data from graphs, where numerical data were not available and we were unable to obtain original data from authors. Discrepancies in data extraction were resolved through consensus.

Statistical Analysis

A cumulative meta‐analysis was performed to monitor the evidence over time and to detect whether the results were influenced by a particular study.12, 61 We used Review Manager, version 5.3, for analyses. Pooled estimates of the treatment effect were updated every time the result of a new study was published. We tracked the progression of evidence on the effect of soy protein intake on lipid markers to pinpoint if there was a change in outcome since 1999 when the FDA published the final rule for the soy protein heart health claim. The principal effect measure was the mean pairwise difference in change from baseline (or, when not available, the posttreatment value) between the soy intervention arm and the comparator arm. We extracted the mean differences and corresponding 95% CIs for each outcome. Change from baseline differences was preferred over end differences, and paired analyses were applied to all crossover trials with the use of a within‐individual correlation coefficient between treatments of 0.5, as described by Elbourne and colleagues.62 Data were pooled in a cumulative manner using the generic inverse variance method with a random‐effects model and expressed as cumulative mean differences with 95% CIs.

Results

Of the 46 FDA‐selected studies, 2 reported data on neither TC nor LDL‐C17, 24 and 1 was a substudy of a larger study already included,53 leaving 43 trials (Figure S1). The median age of the 2607 participants included in the 43 trials for which TC or LDL‐C was available was 55 years; 37% were men, and 49% had hypercholesterolemia. Hypercholesterolemia was defined by the FDA as >240 mg/L for TC or >160 mg/L for LDL‐C.1 The median soy protein dose was 25 g/d, with a median follow‐up of 6 weeks.

The cumulative forest plots for both TC and LDL‐C showed a similar pattern. A significant reduction in TC with soy protein was seen in 1999, at the time of the FDA heart health claim, of −4.5 mg/dL (95% CI, −8.1 to −1.0 mg/dL; P=0.01). Significance was maintained over the following 14 years and ranged from a minimum reduction of −4.0 mg/dL (95% CI, −6.7 to −1.3 mg/dL; P=0.004) in 2001 to a maximum reduction of −7.7 mg/dL (95% CI, −11.2 to −4.3 mg/dL; P<0.0008) in 2006 (Figure 1). The corresponding effect estimate for LDL‐C was −6.3 mg/dL (95% CI, −8.7 to −3.9 mg/dL; P<0.00001) in 1999; and in the following 14 years, the estimates ranged from a minimum reduction of −4.2 mg/dL (95% CI, −6.6 to −1.8 mg/dL; P=0.0006) in 2006 to a maximum reduction of −6.7 mg/dL (95% CI, −10.2 to −3.2 mg/dL; P=0.0002) in 2002 (Figure 2). At no point over the 14 years since the initial FDA heart health claim did the significance level for either TC or LDL‐C fall below P=0.002 (Figures 1 and 2), nor was there a significant deviation at any time from the values of 1999, at the time when the FDA soy heart health claim was granted (Figures 1 and 2).

Figure 1.

Cumulative forest plot for the effect of soy protein intake on low‐density lipoprotein cholesterol (LDL‐C). Cumulative effect is represented by red diamonds. *Total n=37, but reported n=71 for soy protein diets and n=72 for milk protein diets. ^†Twelve subjects were excluded in data analysis in test 1 and 7 in test 2, but it was not specified from which arm; and a 50% dropout rate was taken from each. Data are expressed as cumulative mean differences with 95% CIs.

Figure 2.

Cumulative forest plot for the effect of soy protein intake on total cholesterol (TC). Cumulative effect is represented by red diamonds. *Total n=37, but reported n=71 for soy protein diets and n=72 for milk protein diets. ^†Twelve subjects were excluded in data analysis in test 1 and 7 in test 2, but it was not specified from which arm; and a 50% dropout rate was taken from each. Data are expressed as cumulative mean differences with 95% CIs.

Discussion

The FDA proposes to revoke the heart health claim status granted for soy in 19992; however, the cumulative meta‐analyses of the same data that the FDA is basing this decision on show no inflections that would suggest a significant departure from the effect present at the time when the FDA granted the original health claim for soy in 1999.2

A major concern is that if the FDA is now to use this same approach (on the basis of its 2009 ruling)63 for the remaining heart health claims, then the claims for nuts, viscous fibers (ie, oats, barley, and psyllium), and plant sterols could also be revoked because these effects on serum cholesterol are also modest and variable in terms of individual study statistical significance.

Such a move will reduce public awareness of useful foods for cholesterol control. We have shown that in combination under metabolically controlled conditions, as a dietary portfolio, these foods may reduce LDL‐C and CRP (C‐reactive protein) similarly to a statin (lovastatin) by ~30%.9 This dietary portfolio8, 64, 65 that specifically includes the FDA heart health claim approved foods has now entered the dietary guidelines of the Canadian Cardiovascular Society,66 Heart UK,67 and The European Atherosclerosis Society Guidelines for the Treatment of Statin Associated Muscle Symptoms68 and was mentioned originally in the 2004 National Cholesterol Education Program Adult Treatment Panel‐III update.69

The FDA has, in fact, led the field internationally in providing cardiovascular disease risk reduction health claims. Agencies in other jurisdictions (eg, Health Canada and European Food Safety Authority) have followed the FDA's lead in initiating health claims for various foods or food components. Health Canada approved a cholesterol‐lowering claim for soy as recently as 2016.70

In taking its current action, the FDA found 238 intervention studies, of which it considered 58 to be well‐designed studies, 12 that mentioned blood pressure and 46 that mentioned blood TC or LDL‐C.1 However, studies were eliminated if they had no control group, there was no statistical comparison between the test and control group, total fat intakes were different between treatments, or the saturated fat, dietary cholesterol, and fiber were not balanced between the test and control arms. The level of tolerance for differences between treatments in these nutrients was not defined, and it may be that otherwise reasonable studies with small treatment differences in fiber or dietary cholesterol might still have been acceptable and have strengthened the conclusions. Furthermore, some negative studies were included, such as the one by Jenkins et al,35 in which soy flour was added to high‐temperature extruded breakfast cereal, that may have resulted in possible damage to soy protein structure and the formation of browning reaction products between the starch and soy amino acids. These interactions may have reduced the effectiveness of soy protein. Nevertheless, despite inclusion of such studies, the overall soy effect persisted over time.

The soy health claims may be particularly important at a time when government agencies worldwide are suggesting more plant foods,71, 72, 73, 74 and especially plant protein foods,75 should be consumed. Soy provides an important plant protein source.

The weakness of this study is that it is not a systematic review (followed by a meta‐analysis) because no systematic review of the literature was undertaken, but rather the 46 studies identified by the FDA for their determination were used without prior selection and comprehensive review of the literature by the authors. It may also be questioned how easy it is to consume 25 g of soy protein daily. A total of 7 g of soy protein can be obtained by a cup of most soy milks (some, such as Eden Soy, provide 12 g soy protein/cup), soy yogurts contain up to 9 g/cup and Greek‐style soy yogurts even more, the average soy burger provides ≈12 g soy protein/patty, and extrafirm tofu has just >14 g soy protein in 3 oz, so the daily dose could be obtained from a cup of soy milk, half a cup of yogurt, and 3 oz of extrafirm tofu as a meat replacement.

The strength of this study is that it used the exact data on which the FDA is basing its judgment. Furthermore, a cumulative meta‐analysis was undertaken to determine when, or if, the TC or LDL‐C reductions lost significance.

We conclude that soy continues to have a significant, if modest, effect in reducing serum LDL‐C as a cardiovascular disease risk factor. The effect of soy alone is modest, but it may produce a clinically meaningful reduction when combined in the diet with other FDA‐approved cholesterol‐lowering foods. Furthermore, at a time when plant protein sources are required, soy protein provides a useful plant protein source for the food industry, with a range of applications and with the production of heart healthy foods being one of them.

Sources of Funding

Dr Jenkins was funded by the Canada Research Chairs Program. Dr Sievenpiper was funded by a PSI Graham Farquharson Knowledge Translation Fellowship, Diabetes Canada Clinician Scientist Award, and Banting & Best Diabetes Centre (BBDC) Sun Life Financial New Investigator Award for Diabetes Research.

Disclosures

Dr Jenkins has received research grants from Saskatchewan Pulse Growers, the Agricultural Bioproducts Innovation Program through the Pulse Research Network, the Advanced Foods and Material Network, Loblaw Companies Ltd, Unilever Canada and Netherlands, Barilla, the Almond Board of California, Agriculture and Agri‐food Canada, Pulse Canada, Kellogg's Company, Canada, Quaker Oats, Canada, Procter & Gamble Technical Centre Ltd, Bayer Consumer Care, Springfield, NJ, Pepsi/Quaker, International Nut & Dried Fruit (INC), Soy Foods Association of North America, the Coca‐Cola Company (investigator initiated, unrestricted grant), Solae, Haine Celestial, the Sanitarium Company, Orafti, the International Tree Nut Council Nutrition Research and Education Foundation, the Peanut Institute, Soy Nutrition Institute (SNI), the Canola and Flax Councils of Canada, the Calorie Control Council, the Canadian Institutes of Health Research (CIHR), the Canada Foundation for Innovation (CFI) and the Ontario Research Fund (ORF). He has received in‐kind supplies for trials as a research support from the Almond board of California, Walnut Council of California, American Peanut Council, Barilla, Unilever, Unico, Primo, Loblaw Companies, Quaker (Pepsico), Pristine Gourmet, Bunge Limited, Kellogg Canada, WhiteWave Foods. He has been on the speaker's panel, served on the scientific advisory board and/or received travel support and/or honoraria from the Almond Board of California, Canadian Agriculture Policy Institute, Loblaw Companies Ltd, the Griffin Hospital (for the development of the NuVal scoring system), the Coca‐Cola Company, EPICURE, Danone, Diet Quality Photo Navigation (DQPN), Better Therapeutics (FareWell), Verywell, True Health Initiative (THI), Institute of Food Technologists (IFT), Soy Nutrition Institute (SNI), Herbalife Nutrition Institute (HNI), Saskatchewan Pulse Growers, Sanitarium Company, Orafti, the American Peanut Council, the International Tree Nut Council Nutrition Research and Education Foundation, the Peanut Institute, Herbalife International, Pacific Health Laboratories, Nutritional Fundamentals for Health (NFH), Barilla, Metagenics, Bayer Consumer Care, Unilever Canada and Netherlands, Solae, Kellogg, Quaker Oats, Procter & Gamble, Abbott Laboratories, Dean Foods, the California Strawberry Commission, Haine Celestial, PepsiCo, the Alpro Foundation, Pioneer Hi‐Bred International, DuPont Nutrition and Health, Spherix Consulting and WhiteWave Foods, the Advanced Foods and Material Network, the Canola and Flax Councils of Canada, Agri‐Culture and Agri‐Food Canada, the Canadian Agri‐Food Policy Institute, Pulse Canada, the Soy Foods Association of North America, the Nutrition Foundation of Italy (NFI), Nutra‐Source Diagnostics, the McDougall Program, the Toronto Knowledge Translation Group (St. Michael's Hospital), the Canadian College of Naturopathic Medicine, The Hospital for Sick Children, the Canadian Nutrition Society (CNS), the American Society of Nutrition (ASN), Arizona State University, Paolo Sorbini Foundation and the Institute of Nutrition, Metabolism and Diabetes. He received an honorarium from the United States Department of Agriculture to present the 2013 W.O. Atwater Memorial Lecture. He received the 2013 Award for Excellence in Research from the International Nut and Dried Fruit Council. He received funding and travel support from the Canadian Society of Endocrinology and Metabolism to produce mini cases for the Canadian Diabetes Association (CDA). He is a member of the International Carbohydrate Quality Consortium (ICQC). His wife, Alexandra L Jenkins, is a director and partner of Glycemic Index Laboratories, Inc, and his sister, Caroline Brydson, received funding through a grant from the St. Michael's Hospital Foundation to develop a cookbook for one of his studies. Dr Sievenpiper has received research support from the Canadian Foundation for Innovation, Ontario Research Fund, Province of Ontario Ministry of Research and Innovation and Science, Canadian Institutes of health Research (CIHR), Diabetes Canada, PSI Foundation, Banting and Best Diabetes Centre (BBDC), American Society for Nutrition (ASN), INC International Nut and Dried Fruit Council Foundation, National Dried Fruit Trade Association, The Tate and Lyle Nutritional Research Fund at the University of Toronto, The Glycemic Control and Cardiovascular Disease in Type 2 Diabetes Fund at the University of Toronto (a fund established by the Alberta Pulse Growers), and the Nutrition Trialists Fund at the University of Toronto (a fund established by an inaugural donation from the Calorie Control Council). He has received in‐kind food donations to support a randomized controlled trial from the Almond Board of California, California Walnut Commission, American Peanut Council, Barilla, Unilever, Unico/Primo, Loblaw Companies, Quaker, Kellogg Canada, and WhiteWave Foods. He has received travel support, speaker fees and/or honoraria from Diabetes Canada, Mott's LLP, Dairy Farmers of Canada, FoodMinds LLC, International Sweeteners Association, Nestlé, Pulse Canada, Canadian Society for Endocrinology and Metabolism (CSEM), GI Foundation, Abbott, Biofortis, ASN, Northern Ontario School of Medicine, INC Nutrition Research & Education Foundation, European Food Safety Authority (EFSA), and Physicians Committee for Responsible Medicine. He has or has had ad hoc consulting arrangements with Perkins Coie LLP, Tate & Lyle, and Wirtschaftliche Vereinigung Zucker e.V. He is a member of the European Fruit Juice Association Scientific Expert Panel. He is on the Clinical Practice Guidelines Expert Committees of Diabetes Canada, European Association for the study of Diabetes (EASD), Canadian Cardiovascular Society (CCS), and Obesity Canada. He serves or has served as an unpaid scientific advisor for the Food, Nutrition, and Safety Program (FNSP) and the Technical Committee on Carbohydrates of the International Life Science Institute (ILSI) North America. He is a member of the International Carbohydrate Quality Consortium (ICQC), Executive Board Member of the Diabetes and Nutrition Study Group (DNSG) of the EASD, and Director of the Toronto 3D Knowledge Synthesis and Clinical Trials foundation. His wife is an employee of Sobeys Inc. Dr Kendall has received grants or research support from the Advanced Food Materials Network, Agriculture and Agri‐Foods Canada, Almond Board of California, American Pistachio Growers, Barilla, Calorie Control Council, Canadian Institutes of Health Research, Canola Council of Canada, International Nut and Dried Fruit Council, International Tree Nut Council Research and Education Foundation, Loblaw Brands Ltd, Pulse Canada, Saskatchewan Pulse Growers, and Unilever. He has received inkind research support from the Almond Board of California, American Peanut Council, Barilla, California Walnut Commission, Kellogg Canada, Loblaw Companies, Quaker (Pepsico), Primo, Unico, Unilever, and WhiteWave Foods. He has received travel support and/or honoraria from the American Peanut Council, American Pistachio Growers, Barilla, California Walnut Commission, Canola Council of Canada, General Mills, International Nut and Dried Fruit Council, International Pasta Organization, Loblaw Brands Ltd, Nutrition Foundation of Italy, Oldways Preservation Trust, Paramount Farms, Peanut Institute, Pulse Canada, Sabra Dipping Co, Saskatchewan Pulse Growers, Sun‐Maid, Tate & Lyle, Unilever, and White Wave Foods. He has served on the scientific advisory board for the International Tree Nut Council, International Pasta Organization, McCormick Science Institute, Oldways Preservation Trust, Paramount Farms, and Pulse Canada. He is a member of the International Carbohydrate Quality Consortium and Executive Board Member of the Diabetes and Nutrition Study Group of the EASD, is on the Clinical Practice Guidelines Expert Committee for Nutrition Therapy of the EASD, and is a Director of the Toronto 3D Knowledge Synthesis and Clinical Trials Foundation. Dr Vuksan holds the Canadian (2 410 556) and American (7 326 404) patent on the medical use of viscous fiber blend for reducing blood glucose for treatment of diabetes mellitus, increasing insulin sensitivity, and reducing systolic blood pressure and blood lipids. E. Viguiliouk serves as a scientific advisor for New Era Nutrition. The remaining authors have no 11 disclosures to report.

Supporting information

Figure S1. Study selection indicating the number of studies identified by the FDA and the number of studies included in the meta‐analysis.

(J Am Heart Assoc. 2019;8:e012458 DOI: 10.1161/JAHA.119.012458.)