Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2020 Oct 1.
Published in final edited form as: Acta Diabetol. 2019 Apr 15;56(10):1091–1102. doi: 10.1007/s00592-019-01334-5

Systematic Review: Associations of Calcium Intake, Vitamin D Intake, and Physical Activity on Skeletal Outcomes in People with Type 1 Diabetes Mellitus

María Cristina Gil-Díaz 1, Jennifer Raynor 2, Kimberly O O’Brien 3, George J Schwartz 4, David R Weber 5
PMCID: PMC6751023  NIHMSID: NIHMS1527080  PMID: 30989378

Abstract

The skeletal complications of type 1 diabetes (T1D) include low bone density, poor bone quality and fractures. Greater calcium intake, vitamin D intake, and physical activity are commonly recommended to improve bone health in patients with T1D. However, it is not clear whether these factors are affected by T1D or improve clinical outcomes. The objective of this study was to systematically review the literature for evidence of associations between calcium intake, vitamin D intake, and physical activity and skeletal outcomes in T1D. In accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, twenty-two studies were included in this review. The prevalence of calcium deficiency was high and encompassed greater than 50% of participants in the majority of studies. Despite this finding, there was no clear association between calcium intake and bone density in any study. Calcitriol use was associated with gains in bone density in one study but was not associated with changes in bone turnover markers in a second report. No studies specifically investigated the impact of vitamin D2 or D3 supplementation on bone health. Two studies reported a beneficial effect of physical activity interventions on bone accrual in children. The findings from observational studies of physical activity were mixed. In conclusion, there are insufficient data to determine if deficient calcium intake, vitamin D intake, or physical activity contribute to the skeletal complications of T1D. Future studies specifically designed to assess the impact of these interventions on the skeleton in T1D participants are needed.

Keywords: Type 1 diabetes, calcium, vitamin D, physical activity, exercise, bone density

Introduction

Type 1 diabetes (T1D) is a disease of absolute insulin deficiency caused by autoimmune mediated destruction of the insulin producing pancreatic beta cells. This incurable condition typically develops during childhood and requires lifelong treatment with insulin to control hyperglycemia. Over time, inadequately controlled hyperglycemia contributes to the development of well-known systemic complications of diabetes including retinopathy, neuropathy, and nephropathy. More recently, it has become apparent that the skeleton is also adversely affected by T1D. Deficits in bone density have been identified in both pediatric and adult populations [1, 2]. Patients with T1D have an increased risk of fracture that is apparent early in the disease course and persists across the entire lifespan [3, 4]. Of particular concern are findings that adults with T1D have up to a seven-fold greater risk of suffering a hip fracture [5], a health outcome that is associated with significant morbidity, mortality, and societal cost [6].

There remains uncertainty regarding the appropriate clinical interventions that should be undertaken to prevent or treat T1D related skeletal disease. The absence of robust practice guidelines in this area can be attributed to an incomplete understanding of the mechanisms underlying the skeletal fragility in T1D and a lack of evidence-based data to support specific treatment approaches [7]. As a general principle, optimization of dietary factors (especially calcium and vitamin D) and weight-bearing physical activity are typically recommended for patients at increased risk of fracture [8]. Vitamin D is an essential regulator of calcium absorption, which in turn is critical for maintaining adequate bone mineralization and strength. Numerous studies have illustrated the importance of calcium and vitamin D to bone health in both pediatric and adult populations [9, 10] however there remains lack of consensus as to what constitutes optimal intake of either nutrient and expert committee recommendations have varied [11, 12]. Weight-bearing physical activity promotes skeletal health via an effect of loading to increase bone formation and exercise interventions have been shown to improve bone accrual during growth [13] and to prevent bone loss in older age [14].

Despite the potential importance of diet and exercise in this population at high risk for fractures, there may be disease related factors that that make it difficult for people with T1D to achieve adequate calcium intake and/or participate in physical activity. Optimal diabetes management dictates that mealtime insulin doses are in part determined by the amount of carbohydrates consumed. While carbohydrate consumption is typically not specifically restricted, individuals may nevertheless alter their dietary habits and food selection in an attempt to lower insulin requirements. This hypothesis is supported by studies in children which have shown that milk consumption is reduced following T1D diagnosis in favor of carbohydrate free beverages [15, 16]. Studies of nutrient intake in T1D populations have reported that fewer than 60% of teenagers met the recommended dietary allowance (RDA) for calcium [17, 18]. The reported prevalence of insufficient calcium intake in healthy children has ranged from 35–85% [19], although direct comparisons of calcium intake between T1D and healthy youth have not been widely reported. Participation in physical activity has been found to be lower in patients with T1D compared to healthy peers [20] and may be related to factors including fear of hypoglycemia or presence of diabetic complications including neuropathy or retinopathy [21].

The objective of this study was to systematically review the published literature for evidence of an effect between calcium and vitamin D intake and/or physical activity on skeletal outcomes in children and adults with T1D.

Methods

A systematic search of the literature was conducted by a medical librarian in accordance with Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines [22] to identify studies related to the effects of calcium intake, vitamin D intake, or physical activity on bone health in people with T1D. Search terms were developed by an expert in T1D and bone health in consultation with a medical librarian. The search strategy combined both keywords and controlled vocabulary in the following databases from inception until the date the search was performed (listed in parenthesis): PubMed (2/23/2018), Embase (2/23/2018), Cochrane Library (2/26/2018), Cumulative Index to Nursing and Allied Health Literature (2/26/2018), Web of Science (3/12/2018), Clinicaltrials.gov (3/19/2018), CenterWatch (3/19/2018), Metaregister of Controlled Trials (3/19/2018), and select grey literature sources (3/19/2018). Full search details are provided in Supplemental Table 1.

The initial search yielded 601 citations. Abstracts were independently screened for eligibility by two reviewers who were blinded to each other using the Rayyan QCRI web-based review platform [23]. Eligibility criteria for inclusion in the final review were agreed upon prior to abstract screening and required that studies were conducted in humans with T1D, were primary research, had a primary outcome related to bone health, and measured and reported associations between exposures (calcium intake/supplementation, vitamin D intake/supplementation, physical activity) and bone health outcomes (fractures, bone density/mass, biochemical markers of bone and mineral metabolism) of interest. Both interventional and observational study designs were considered. Citations that referred to abstract-only or non-English language publications were excluded. Studies that were limited to biochemical measures of calcium and/or vitamin D (i.e. serum calcium or 25-OH vitamin D level) but did not directly assess calcium or vitamin D intake or evaluate the response to calcium or vitamin D supplementation were excluded and have been reviewed elsewhere [24, 25].

Conflicts related to eligibility criteria were resolved after both reviewers had completed their initial reviews via face-to-face discussion. If the conflict could not be resolved, the full text was requested for review. Full text articles for citations meeting the eligibility criteria and with outstanding conflicts were independently assessed by the reviewers according to pre-determined criteria. The strength and significance of associations between exposures of interest and bone health outcomes were abstracted from individual studies and summarized in tables and narrative form. Secondary data items were additionally abstracted and included comparisons of calcium intake and physical activity between T1D vs control participants and the percentage of study participants that were meeting the RDA for calcium [11]. Data from secondary abstraction were combined and analyzed using standard summary statistics. The risk of bias was assessed at the study level for each included study.

Results

A total of 22 studies were selected for final inclusion into the systematic review. Details of the study selection process are provided in Figure 1. Of the 22 included studies: eight reported results relevant to the associations between calcium/vitamin D intake and bone health; eight reported on associations between physical activity and bone health; and six on associations between both calcium/vitamin D and physical activity.

Figure 1: PRISMA flow diagram for literature search and article inclusion.

Figure 1:

1 Exclusion criteria included non-human subjects, not primary research, did not have a primary outcome related to bone health, abstract-only publication, or non-English language

2 Additional exclusion criteria for full texts included failure to measure and/or report associations between exposures (calcium intake/supplementation, vitamin D intake/supplementation, physical activity) and bone health outcomes (fractures, bone density/mass, biochemical markers of bone and mineral metabolism) of interest

Calcium and vitamin D intake studies

The characteristics and key findings of the calcium/vitamin D intake studies are summarized in Table 1. This included ten studies conducted in children or adolescents and four in adult populations. There was a single clinical trial that included an assessment of calcium and vitamin D supplementation in a T1D cohort. In a study designed to assess the effect of alendronate on areal bone mineral density (aBMD), 20 elderly women were assigned to an active comparator arm where they received 1000 mg elemental calcium (as calcium carbonate) and 800 IU vitamin D3 daily. There was no change in spine aBMD over 24 months in those that received only calcium and vitamin D3 supplementation, whereas it significantly increased by 8.5% in those that additionally received alendronate [26]. The average calcium intake of participants prior to starting the study was not reported. There were two interventional studies that assessed the effects of calcitriol (active vitamin D) on skeletal outcomes in T1D participants, with mixed results. In a secondary analyses of a randomized, controlled trial (RCT) to assess the effect of calcitriol on beta-cell function in 27 young adults, Napoli et al. found no effect of calcitriol on markers of bone turnover [27]. By contrast, Al-Qadreh et al. found that BMD increased over 12 months in response to daily calcitriol in an unblinded, uncontrolled study performed in 12 T1D children with low baseline BMD [28]. No studies of vitamin D supplementation with either cholecalciferol or ergocalciferol were identified.

Table 1:

Characteristics and findings of studies relating dietary calcium or vitamin D intake to bone health outcomes

Authors, year (country) Study Design Study participants Method of dietary assessment / intervention Bone health outcome(s) Relevant findings
Studies showing a beneficial association between calcium and/or vitamin D and bone health outcomes (n=2)
Al-Qadreh, et al., 1996 (Greece) Longitudinal open label interventional T1D: 8M, 4F; age 12 ± 1.91 Oral 1α-OHD3 0.05 μg/kg/d for 12 months SPA: Radius aBMD aBMD significantly increased over 12 months on treatment, p=0.02
Weber, et al., 2017 (USA) Cross-sectional observational T1D: 20F; age 14.2 (range 9.1–16.5) Ca intake by 3-day diet record, analyzed by NDSR Estimated calcium retention using a dual-stable Ca isotope method - Ca intake positively correlated with calcium retention (Spearman’s rho 0.49, p=0.03)
Studies showing no association between calcium and/or vitamin D and bone health outcomes (n=12)
Maugeri, et al., 2002 (Italy) Longitudinal intervention2 T1D: 40F; age 76 ± 3 Ca carbonate (1 gm/d) + vitamin D3 (800 IU/d) vs Ca + vitamin D3 + alendronate for 24 months DXA: LS aBMD Z-scores aBMD did not significantly change after 24 months of Ca carbonate + vitamin D3 alone
Heap, et al., 2003 (USA) Cross-sectional observational T1D: 30M, age 14.6 ± 1.7; 25F, age 14.7 ± 1.9
Controls: 42M, age 14.5 ± 1.9; 53F, age 14.8 ± 1.5
Ca intake by questionnaire (unnamed) DXA: TB, LS, hip aBMD
pQCT: tibia
- Ca intake did not differ significantly in T1D vs controls
- Ca intake not significantly associated with DXA or pQCT outcomes
Moyer-Mileur et al., 2004 (USA) Longitudinal observational T1D: 26M, age 14.9 ± 1.8; 16F, age 14.1 ± 1.8
Controls: 90M, age 15.0 ± 2.1; 109F, age 15.1 ± 1.0
Ca intake by questionnaire (unnamed) DXA: TB, LS, hip aBMD
pQCT: tibia
- Ca intake did not differ significantly in T1D vs controls
- Ca intake not significantly associated with changes in DXA or pQCT
Valerio, et al., 2004 (Italy) Cross-sectional observational T1D: 46 M, age 11.9 ± 4.1; 40F, age 11.8 ± 4.3 Ca intake by FFQ (unnamed) QUS: first phalynx speed of sound (SoS) Z-score - Ca intake not significantly correlated with SOS
Janghorbani, et al., 2006 (USA) Retrospective longitudinal T1D: 292F; age 54.7 ± 9.3
Controls: 101,343F, age (55.9 ± 9.4)
Ca and Vitamin D intake by FFQ (unnamed) Fracture - Ca intake not significantly different in T1D vs controls
- Ca intake did not alter association between T1D and increased fracture risk
Brandao, et al., 2007 (Brazil) Cross-sectional observational T1D: 44 M/F; age 15.5 ± 2.4
Controls: 22 M/F; age 14.6 ± 2.8
Ca intake by questionnaire (unnamed) DXA: LS BMC and aBMD Z-scores - Ca intake not significantly different in T1D vs controls
- Ca intake not significantly associated with BMC or aBMD
Heilman, et al., 2009 (Estonia) Cross-sectional observational T1D: 19M, 11F; age 13.1 ± 3.6
Controls: 19M, 11F, age 13.2 ± 3.9
Ca intake by questionnaire (unnamed) DXA: TB, LS BMC and aBMD3 - Ca intake not significantly different in T1D and controls
- aBMD did not differ significantly among those with low, moderate, or high Ca intake
Janner, et al., 2010 (Switzerland) Cross-sectional observational T1D: 129 M/F; age 11.6 (95% CI 11.012.3) Ca intake by FFQ (unnamed) Vitamin D status based on serum 25- OH vitamin D level - Dietary Ca intake not significantly associated with vitamin D status
Maggio, et al., 2010 (Switzerland) Cross-sectional observational T1D: 13M, 14F; age 10.5 ± 2.4
Controls: 16M, 16F; age 10.5 ± 2.5
Ca intake by FFQ (unnamed) DXA: TB, LS, FN aBMD Z-scores - Dietary Ca intake significantly lower in T1D vs controls (p=0.01)
- Dietary Ca intake not significantly associated with aBMD Z-scores in T1D or controls
Simmons, et al., 2011 (USA) Cross-sectional observational T1D - good control: 29M, 11 F; age 15.2 (IQR: 14.3–16.1)
T1D - poor control: 11M, 16F; age 16.4 (IQR:15.1,17.8)
Ca and vitamin D by 3-day diet record, analyzed by NDSR DXA: TB and LS BMD Z-scores Serum markers of bone turnover - Ca and vitamin D intake not significantly associated with glycemic control
- Association between Ca/vitamin D intake and BMD not reported
Napoli, et al., 2013 (Italy) Longitudinal randomized placebo controlled intervention T1D -placebo: 8M, 7F; age 22.0 ± 2.3
T1D-drug: 8M, 4F age 22.8 ± 2.1
Oral calcitriol 0.25 μg daily for 12 months Serum markers of bone turnover (OCN, β-CTX) - No effect of calcitriol intervention on 12-month change in OCN or β-CTX
Kujath, et al., 2015 (USA) Cross-sectional observational T1D: 89F; age 27.9 ± 6.8
Controls: 76F age 28.4 ± 8.2
Ca intake by FFQ (unnamed) DXA: Heel, forearm aBMD
Serum markers of bone and mineral metabolism
- Ca intake not significantly different between T1D vs controls
- Ca intake not significantly associated with aBMD or any skeletal outcome in either group

Abbreviations: 1α-OHD3, 1alpha-hydroxy vitamin D3; aBMD, areal bone mineral density; BMC, bone mineral content; β-CTX, beta cross-laps; F, female; FFQ, food frequency questionnaire; IQR, inter-quartile range; LS, lumbar spine; M, male; MET, metabolic equivalent time; NDSR, Nutrition Data System for Research program; OCN, osteocalcin; SPA, single photon absorptiometry; T1D, type 1 diabetes; TB, total body;

1

Age expressed in years, all such values

2

Not noted if subjects were randomized or blinded

3

Adjusted for age and height

There were eleven observational studies that assessed dietary calcium intake, nine by a food-frequency questionnaire [2937] and two by three-day diet recall [38, 39]. Dietary calcium intake was not associated with bone density (seven studies [3137, 39]), fractures (one study [32]) or other markers of bone mineral metabolism (two studies [29, 30]). A study that investigated calcium metabolism using stable isotopes found that calcium intake was positively associated with estimated calcium retention in adolescent girls with T1D but did not measure BMD [38]. A comparison between the average dietary calcium intake in T1D participants and healthy controls is shown in Figure 2. Mean calcium intake among T1D participants varied widely, from less than 500 mg/day to more than 1300 mg/day. Calcium intake did not differ statistically between T1D and healthy participants in any of the seven studies that included a control population [2933, 36]. Of the seven studies that reported dietary calcium intake in T1D participants relative to the RDA, the majority (86%) found that at least half of T1D participants were not consuming the recommended amount of calcium (Figure 3). One additional study reported that the calcium intake of T1D participants was “adequate” but did not quantify this statement relative to the RDA [30]. Physical activity studies

Figure 2:

Figure 2:

Comparison of dietary calcium intake in participants with type 1 diabetes (T1D) versus healthy controls assessed by food questionnaire (n=8) and dietary recall (n=2). Error bars represent standard deviation and were taken directly from manuscripts. None of the studies reported a significant difference in calcium intake in T1D vs control participants. Studies by Simmons et al., Janner et al., and Weber et al. did not have a control group. Data for Valerio, Moyer-Mileur, and Janner studies represent average of reported sex-specific values, in all cases there was no difference by sex. Values below the X-axis represent mean calcium intake for T1D and control participants. Age of study participants is reported as range, if available, otherwise as mean ± standard deviation

Figure 3:

Figure 3:

Percentage of participants with type 1 diabetes (T1D) not meeting the recommended dietary allowance (RDA) for calcium intake. RDA for calcium intake: 4–8 years old, 1,000 mg/day; 9–18 years old, 1300 mg/d. Solid bars represent studies done in children, dashed bars represent studies done in adults.

The characteristics and key findings of the physical activity studies are summarized in Table 2. This included seven studies conducted in children or adolescents, six in adults, and one in a combined population. Two interventional studies investigated the effects of physical activity on skeletal outcomes in children with T1D. Elhabashy et al. reported that femoral neck aBMD increased by 3% over three months in 11 T1D adolescents that completed a thrice weekly aerobic exercise program [40]. There was no control group and 54% of enrolled participants did not complete the intervention. In an RCT, Maggio et al. reported that 15 children and young adolescents who completed a twice weekly weight-bearing physical activity program demonstrated greater gains in whole body and spine aBMD over nine months compared to 12 T1D similarly aged controls [41]. The effect size in T1D subjects was similar to what was seen in an additional arm of healthy children of similar age.

Table 2:

Characteristics and findings of studies relating physical activity to bone health outcomes

Authors, year (country) Study Design Study participants Method of physical activity assessment / intervention Bone health outcome(s) Relevant findings
Studies showing a beneficial association between physical activity and bone health outcomes (n=6)
Salvatoni, et al., 2004 (Italy) Cross-sectional observational T1D: 36M, 21F; age 11.1 ± 4.51 Time spent in PA (unnamed questionnaire) DXA: TB and LS BMC and aBMD, adjusted for confounders - Time spent in PA positively correlated with TB BMC (R = 0.32, P<0.001) and inversely correlated with LS aBMD (R = 0.18, P< 0.05)
Maser, et al., 2009 (USA) Cross-sectional observational T1D: 39M, age 45 ± 11; 27F, age 39 ±9 Energy expenditure/d (accelerometer) DXA: TB, hip BMC and aBMD - PA positively correlated with TB BMC (P=0.4, p<0.01) and hip BMC (P=0.01, p<0.01)
Elhabashy, et al., 2011 (Egypt) Longitudinal open label interventional T1D: 14M, 10 F; age 17.2 ± 2.0 70 min stretching and non-weight bearing PA (ergometer); 3 sessions/wk for 3 months DXA: FN aBMD Serum markers of bone turnover - FN aBMD increased over study (3.4 ± 2.9%, p<0.01)
- P1NP increased over study (40.9 ± 31.7%, p<0.01)
Joshi, et al., 2012 (India) Cross-sectional observational T1D: 53M, 22F
Controls: 100M, 40F Age: 27.2 ± 11.2
MET (international PA questionnaire) DXA: TB and LS aBMD Z-scores - MET positively correlated with BMD Z-score at TB (R=0.58, p<0.05) and LS (R=0.54, p<0.05) in T1D
Maggio, et al., 2012 (Switzerland) Longitudinal randomized interventional trial T1D: 13M, 14F; age 10.5 ± 2.4
Controls: 16M, 16F; age 10.5 ± 2.5
- Weight bearing PA sessions; 90 min 2 times/wk for 9 months
- Time spent in PA (Modifiable Activity Questionnaire for Adolescents)
DXA: TB, LS, FN aBMD - PA at baseline not different in T1D vs controls
- PA intervention resulted in greater increases in TB and LS aBMD in T1D and control participants, pooled effect size 0.169 (TB) and 0.101 (LS)
- Effect size of PA intervention did not differ in T1D vs controls
Kujath, et al., 2015 (USA) Cross-sectional observational T1D: 89F; age 27.9 ± 6.8
Controls: age 28.4 ± 8.2
Total energy expenditure/d from vigorous PA (Five- City Project questionnaire) - DXA: Heel, forearm aBMD
- Serum markers of bone and mineral metabolism
- Vigorous PA not different in T1D vs controls
- Vigorous PA positively correlated with heel aBMD in T1D only (P=0.039, p<0.0001)
Studies showing no association between physical activity and bone health outcomes (n=8)
Rosenbloom, et al., 1977 (USA) Cross-sectional observational T1D: 196 M/F Controls: 124 M/F Age: 6–26 Subjective PA rating scale Bone mineral analyzer: distal forearm aBMD - No significant association between PA and aBMD in T1D (R = −0.11, p=0.19)
Rix, et al., 1999 (Denmark) Cross-sectional observational T1D PN: 21 M/F; age 57 ± 6 T1D no PN: 21 M/F; age 56 ± 5 Controls: 21 M/F; age 58 ± 6 Categorical PA questionnaire (high vs low PA) DXA: TB, LS, forearm aBMD Z- scores - PA level did not contribute to differences in aBMD by T1D or PN status
Heap, et al., 2003 (USA) Cross-sectional observational T1D: 30M, age 14.6 ± 1.7; 25F, age 14.7 ± 1.9
Controls: 42M, age 14.5 ± 1.9; 53F, age 14.8 ± 1.5
MET hrs/wk (unnamed questionnaire) DXA: TB, LS, hip aBMD
pQCT: tibia
- PA not significantly different in T1D vs controls
- PA not significantly associated with DXA or pQCT outcomes
Moyer-Mileur et al., 2004 (USA) Longitudinal observational T1D: 26M, age 14.9 ± 1.8; 16F, age 14.1 ± 1.8
Controls: 90M, age 15.0 ± 2.1; 109F, age 15.1 ± 1.0
MET hrs/wk (unnamed questionnaire) DXA: TB, LS, hip aBMD pQCT: tibia - PA not significantly different in T1D vs controls
- PA not significantly associated with changes in DXA or pQCT outcomes
Janghorbani, et al., 2006 (USA) Retrospective longitudinal T1D: 292F; age 54.7 ± 9.3
Controls: 101,343F, age (55.9 ± 9.4)
Hrs/wk in PA (unnamed questionnaire) Fracture - PA not significantly different in T1D vs controls
- PA did not alter association between T1D and fracture risk
Heilman, et al., 2009 (Estonia) Cross-sectional observational T1D: 19M, 11F; age 13.1 ± 3.6
Controls: 19M, 11F, age 13.2 ± 3.9
Counts/hr (accelerometer) DXA: TB, LS BMC and aBMD2 - PA significantly lower in T1D vs control boys (p=0.04); not different in girls
- PA not significantly associated with BMC or aBMD
Maggio, et al., 2010 (Switzerland) Cross-sectional observational T1D: 13M, 14F; age 10.5 ± 2.4
Controls: 16M, 16F; age 10.5 ± 2.5
Counts/min (accelerometer)
Time spent in PA (Modifiable Activity Questionnaire for Adolescents)
DXA: TB, LS, FN aBMD Z-scores - PA significantly lower in T1D vs controls by accelerometer (p=0.04), not by questionnaire
- No association between PA and aBMD Z-scores in T1D or controls
Sayarifard, et al., 2017 (Iran) Cross-sectional observational T1D: 55M, 57F; age 12.5 (IQR 4–14) Hrs/d in moderate to vigorous PA (unnamed questionnaire) DXA: LS aBMD Z-score; categorized as low, low normal, normal PA did not differ signficantly across BMD categories

Abbreviations: aBMD, areal bone mineral density; BMC, bone mineral content; DXA, dual energy X-ray absorptiometry; F, female; FN, femoral neck; IQR, inter-quartile range; LS, lumbar spine; M, male; MET, metabolic equivalent time; P1NP, procollagen type 1 N-terminal propeptide; PA, physical activity; PN, Peripheral neuropathy; QUS, quantitative ultrasound; T1D, type 1 diabetes; TB, total body

1

Expressed in years, all such values

2

Effect size expressed as eta-squared (small effect, 0.01; moderate effect, 0.06; large effect, 0.14; as reported by authors)

Twelve observational (ten cross-sectional and two longitudinal) studies reported associations between physical activity and skeletal outcomes. Of these, four (three in adults and one in children) reported a significant positive association with a measure of activity and a skeletal outcome. Maser et al., found that objectively measured energy expenditure by accelerometry was associated with whole body and hip bone mineral content (BMC) in 66 young T1D adults [42]. Kujath et al. [37], Salvatoni et al. [43], and Joshi et al. [44] all reported positive associations between subjective measures of activity and aBMD in T1D participants. Eight studies did not find a relationship between physical activity and skeletal outcomes [29, 30, 32, 34, 36, 4547]. There were no reports of a negative relationship between physical activity and a skeletal outcome in any of the included studies.

The method of physical activity assessment varied across studies. Three studies [34, 36, 42] reported objective measures of activity recorded by accelerometer, including activity counts (two studies [34, 41]) and energy expenditure (one study [42]). Ten studies [29, 30, 32, 37, 41, 4347] reported subjective measures of activity determined by questionnaire, only three of these provided the name of the questionnaire that was used. Subjective activity outcomes included time spent in physical activity (four studies [32, 41, 43, 47]), metabolic equivalent time (three studies [29, 30, 44]), categorization of relative activity (two studies[45, 46]) and energy expenditure (one study [37]). The time spent in physical activity by T1D participants across the studies is shown in Figure 4. In the eight studies that compared physical activity in T1D compared to control participants, only one showed a significant difference, with lower physical activity in the T1D group [36].

Figure 4:

Figure 4:

Comparison of subjectively assessed physical activity in participants with type 1 diabetes versus healthy controls reported as A. time spent in physical activity (hrs/wk) and B. metabolic equivalents (MET, hrs/wk). Values below X-axis represent group means, error measurements were taken directly from manuscripts. Error bars represent standard deviation except for Maggio et al. (standard error). *Statistical difference between T1D and controls, p<0.05

Risk of bias

The majority (17/22) of included studies were determined to be at risk of selection bias as a result of observational or retrospective study design and/or reliance on convenience samples of T1D participants. These studies were also at risk of reporting bias, as in many cases calcium intake, vitamin D intake, and/or physical activity were not the primary exposures of interest which may have contributed to under or incomplete reporting of associations between exposures and outcomes of interest. Two of the five interventional studies were randomized controlled trials [27, 41] including one that was double-blinded [27] and one in which only investigators were blinded [41].

Therefore, four out of five interventional studies were at risk of performance and detection bias. It should be noted, however, that in all cases the primary outcomes were objectively measured (BMD by DXA in four out of five, biochemical markers in one) which mitigates the risk of these forms of bias to some degree.

Discussion

Low bone mass and skeletal fragility are now widely recognized as clinically relevant complications of T1D [48]. Despite the growing interest in this topic, there is little consensus on how best to prevent or treat impaired bone health in the T1D population. Recommendations often center on increasing calcium intake and physical activity, but data to support these interventions in T1D patients are scarce. In this systematic review, we identified a substantial number of publications that were related to bone health in T1D (n=127). However, very few (n=22) assessed the relationships between the intake or supplementation of calcium and vitamin D and/or physical activity and bone health outcomes in T1D participants.

None of the included studies reported a significant association between calcium intake and bone density in T1D participants. There was only one calcium interventional study, which found no effect of calcium and vitamin D supplementation on aBMD [26]. The interpretation of the findings of this study is limited by a small sample size, the fact that calcium/vitamin D supplementation was not the primary exposure and that there was no placebo group. Greater calcium intake was associated with greater calcium retention in a small study conducted in growing adolescent females with T1D [38]. Additionally, a quarter of the study participants were found to have negative estimated calcium retention (i.e. calcium losses from urine and stool were estimated to exceed calcium absorption from the gastrointestinal tract). While preliminary, these findings support the need for additional studies to determine if impaired calcium retention is a contributor to impaired bone accrual in T1D.

The effect of calcitriol (active vitamin D) on bone health was assessed in two studies. The first reported a beneficial effect on aBMD children [28], however the impact of the findings are limited by small sample size and lack of a control group. The second study found no effect of calcitriol on bone formation as assessed by bone turnover markers [27]. The clinical significance of these findings is unclear in the absence of aBMD or other measure of bone mass. A potential drawback to the use of calcitriol in the T1D population is the propensity of this medication to increase urinary calcium excretion. Patients with T1D may already have excess urinary calcium excretion at baseline [38, 49] and indeed one subject had to be withdrawn from the Al-Qadreh study due to the development of hypercalciuria [28].

There were no studies that investigated the effect of vitamin D supplementation on bone health outcomes in T1D participants. Vitamin D deficiency is common in T1D populations, with many [24], but not all [50], studies reporting that vitamin D levels were lower in T1D participants compared to healthy individuals. We note that there has been considerable interest in the investigation of potential off-target effects of vitamin D in the development of T1D. The findings of these studies have been reviewed elsewhere [51, 52] and were excluded from the present analyses if they did not report a relevant bone health outcome.

With respect to physical activity, there were two small interventional studies in children with T1D that reported a positive effect of exercise on aBMD [40, 41]. The nature and duration of the interventions differed, with one assessing the effect of a non-weight bearing ergometer intervention over three months [40] and the other a mixture of weight-bearing activities over nine months [41]. It is interesting to note that aBMD increased in response to the non-weight bearing activity, as bone formation in response to mechanosensation is the primary mechanism by which bone density is thought to increase with exercise [53]. The interpretation of this study is limited by the use of raw aBMD values (rather than Z-scores) in a population where aBMD would be expected to increase over time, by lack of a control group, and a high drop-out rate. There were no adverse events related to hypoglycemia in either study, supporting the safety of conducting supervised exercise interventions in the pediatric T1D population. The findings of studies that reported cross-sectional associations between physical activity and bone health outcomes were mixed and difficult to synthesize given the variety of ways in which physical activity was assessed.

This systematic review has limitations. A major limitation of the published literature was the use of a wide variety of different measures to assess both calcium intake and physical activity. Previous studies have found poor correlations between different subjective measures of calcium intake [54], for example, which speak to the limitations of combining results from different tools for analysis. Additionally, the majority of the studies employed observational cross-sectional study designs which do not all allow for assessment of causality. Finally, the study is at risk of publication bias, particularly as non-significant findings may be less likely to be published.

Conclusion

In summary, there is insufficient evidence to determine if deficiencies in calcium intake, vitamin D intake and physical activity are important contributors to the abnormal skeletal phenotype of T1D or whether prescription of calcium, vitamin D, or an exercise regimen will improve bone health in this population. Dietary calcium deficiency was common in T1D research participants, but was no more prevalent than what was observed in healthy comparison groups. Properly designed prospective, interventional studies that evaluate the effect of dietary modification or calcium supplementation on bone accrual in calcium deficient T1D participants are needed. The findings of increased bone accrual following exercise interventions in two small studies in children are promising but need to be evaluated in larger cohorts. Studies that directly compare the effects of weight bearing versus non-weight bearing interventions would be of interest, and may yield insight into the mechanisms that underlie impaired bone accrual in T1D. Finally, the assessment and reporting of calcium intake and physical activity in clinical studies of T1D would benefit from greater standardization to allow for more accurate comparisons across studies.

Supplementary Material

592_2019_1334_MOESM1_ESM

Acknowledgements

DRW was supported by K23 DK114477 from the National Institutes of Health.

Footnotes

Conflicts of interest

The authors declare that they have no conflicts of interest.

Contributor Information

María Cristina Gil-Díaz, Golisano Children’s Hospital, University of Rochester, 601 Elwood Ave, Box 690, Rochester NY, 14642, USA. orcid: 0000-0002-4580-3240.

Jennifer Raynor, Edward G. Miner Library, University of Rochester Medical Center, 601 Elmwood Ave., Rochester, NY 14642.

Kimberly O. O’Brien, College of Human Ecology, Cornell University, 230 Savage Hall, Ithaca, New York, 14850, USA. orcid: 0000-0002-0954-1706

George J. Schwartz, Golisano Children’s Hospital, University of Rochester, 601 Elwood Ave, Box 690, Rochester NY, 14642, USA.

David R. Weber, Golisano Children’s Hospital, University of Rochester, 601 Elwood Ave, Box 690, Rochester NY, 14642, USA. orcid: 0000-0001-6895-2372

References

  • 1.Kaur H, Joshee P, Franquemont S, Baumgartner A, Thurston J, Pyle L, Nadeau KJ, Shah VN (2018) Bone mineral content and bone density is lower in adolescents with type 1 diabetes: A brief report from the RESISTANT and EMERALD studies. J Diabetes Complications 32:931–933 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Zhukouskaya VV, Eller-Vainicher C, Vadzianava VV, Shepelkevich AP, Zhurava IV, Korolenko GG, Salko OB, Cairoli E, Beck-Peccoz P, Chiodini I (2013) Prevalence of morphometric vertebral fractures in patients with type 1 diabetes. . Diabetes Care 36,6:1635–1640 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Weber D, Haynes K, Leonard M, Willi S, Denburg M (2015) Type 1 diabetes is associated with an increased risk of fracture across the life span: a population-based cohort study using The Health Improvement Network (THIN). Diabetes Care 38:1913–1920 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Dhaliwal R, Foster NC, Boyle C, Al Mukaddam M, Weinstock RS, Rickels MR, Shah VN, DiMeglio LA (2018) Determinants of fracture in adults with type 1 diabetes in the USA: Results from the T1D Exchange Clinic Registry. Journal of Diabetes and its Complications 32:1006–1011 [DOI] [PubMed] [Google Scholar]
  • 5.Vestergaard P (2007) Discrepancies in bone mineral density and fracture risk in patients with type 1 and type 2 diabetes a meta-analysis. Osteoporos Int 18:427–444 [DOI] [PubMed] [Google Scholar]
  • 6.Blume S, Curtis J (2011) Medical costs of osteoporosis in the elderly Medicare population. Osteoporos Int 22:1835–1844 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Zhukouskaya VV, Eller-Vainicher C, Shepelkevich AP, Dydyshko Y, Cairoli E, Chiodini I (2015) Bone health in type 1 diabetes: focus on evaluation and treatment in clinical practice. J Endocrinol Invest 38:941–950 [DOI] [PubMed] [Google Scholar]
  • 8.Cosman F, de Beur S, LeBoff M, Lewiecki E, Tanner B, Randall S, Lindsay R (2014) Clinician’s Guide to Prevention and Treatment of Osteoporosis. Osteoporos Int 25:2359–2381 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Welten D, Kemper H, Post G, van Staveren W (1995) A meta-analysis of the effect of calcium intake on bone mass in young and middle aged females and males. J Nutr 125:2802–2813 [DOI] [PubMed] [Google Scholar]
  • 10.Johnston CJ, Miller J, Slemenda C, Reister T, Hui S, Christian J, Peacock M (1992) Calcium supplementation and increases in bone mineral density in children. N Engl J Med 327:82–87 [DOI] [PubMed] [Google Scholar]
  • 11.Institute of Medicine (US) Committee to Review Dietary Reference Intakes for Vitamin D and Calcium, Ross A, Taylor C, Yaktine A, et al. editors. (2011) Dietary Reference Intakes for Calcium and Vitamin D. National Academies Press; [PubMed] [Google Scholar]
  • 12.Holick M, Binkley N, Bischoff-Ferrari H, Gordon C, Hanley D, Heaney R, Murad M, Weaver C (2011) Evaluation, treatment, and prevention of vitamin D deficiency: an Endocrine Society clinical practice guideline. J Clin Endocrinol Metab 96:1911–1930 [DOI] [PubMed] [Google Scholar]
  • 13.Specker B, Thiex N, Sudhagoni R (2015) Does Exercise Influence Pediatric Bone? A Systematic Review. Clin Orthop Relat Res 473:3658–3672 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Zhao R, Zhang M, Zhang Q (2017) The Effectiveness of Combined Exercise Interventions for Preventing Postmenopausal Bone Loss: A Systematic Review and Meta-analysis. J Orthop Sports Phys Ther 47:241–251 [DOI] [PubMed] [Google Scholar]
  • 15.Blouin V, Bouchard I, Galibois I (2011) Body Mass Index and Food and Nutrient Intake of Children with Type 1 Diabetes and a Carbohydrate Counting Meal Plan. Can J Diabetes 35:254–261 [Google Scholar]
  • 16.Patton S, Clements M, George K, Goggin K (2016) “I Don’t Want Them to Feel Different”: A Mixed Methods Study of Parents’ Beliefs and Dietary Management Strategies for Their Young Children with Type 1 Diabetes Mellitus. J Acad Nutr Diet 116:272–282 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Gilbertson H, Reed K, Clark S, Francis K, Cameron F (2018) An audit of the dietary intake of Australian children with type 1 diabetes. Nutr Diabetes 8:10. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Mayer-Davis E, Nichols M, Liese A, Bell R, Dabelea D, Johansen J, Pihoker C, Rodriguez B (2006) Thomas J, Williams D: Dietary intake among youth with diabetes: the SEARCH for Diabetes in Youth Study. J Am Diet Assoc 106:689–697 [DOI] [PubMed] [Google Scholar]
  • 19.Bailey RL, Dodd KW, Goldman JA, Gahche JJ, Dwyer JT, Moshfegh AJ, Sempos CT, Picciano MF (2010) Estimation of total usual calcium and vitamin D intakes in the United States. J Nutr 140:817–822 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Sundberg F, Forsander G, Fasth A, Ekelund U (2012) Children younger than 7 years with type 1 diabetes are less physically active than healthy controls. Acta Paediatr 101:1164–1169 [DOI] [PubMed] [Google Scholar]
  • 21.Keshawarz A, Piropato A, Brown T, Duca L, Sippl R, Wadwa R, Snell-Bergeon J (2018) Lower objectively measured physical activity is linked with perceived risk of hypoglycemia in type 1 diabetes. J Diabetes Complications 32:975–981 [DOI] [PubMed] [Google Scholar]
  • 22.Moher D, Liberati A, Tetzlaff J, Altman D (2009) Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Bmj 339:b2535. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A (2016) Rayyan-a web and mobile app for systematic reviews. Syst Rev 5:210.-016-0384-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Feng R, Li Y, Li G, Li Z, Zhang Y, Li Q, Sun C (2015) Lower serum 25 (OH) D concentrations in type 1 diabetes: A meta-analysis. Diabetes Res Clin Pract 108:E71–E75 [DOI] [PubMed] [Google Scholar]
  • 25.Hygum K, Starup-Linde J, Harslof T, Vestergaard P, Langdahl B (2017) MECHANISMS IN ENDOCRINOLOGY: Diabetes mellitus, a state of low bone turnover - a systematic review and meta-analysis. Eur J Endocrinol 176:R137–r157 [DOI] [PubMed] [Google Scholar]
  • 26.Maugeri D, Panebianco P, Rosso D, Calanna A, Speciale S, Santangelo A, Rizza I, Motta M, Lentini A, Malaguarnera M (2002) Alendronate reduces the daily consumption of insulin (DCI) in patients with senile type I diabetes and osteoporosis. . Arch Gerontol Geriatr 34:117–122 [DOI] [PubMed] [Google Scholar]
  • 27.Napoli N, Strollo R, Pitocco D, Bizzarri C, Maddaloni E, Maggi D, Manfrini S, Schwartz A, Pozzilli P (2013) Effect of Calcitriol on Bone Turnover and Osteocalcin in Recent-Onset Type 1 Diabetes. PLoS ONE 8(2):e56488. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Al-Qadreh A, Voskaki I, Kassiou C, Athanasopoulou H, Sarafidou E, Bartsocas C (1996) Treatment of osteopenia in children with insulin-dependent diabetes mellitus: the effect of 1 alpha-hydroxyvitamin D3. Eur J Pediatr 155:15–17 [DOI] [PubMed] [Google Scholar]
  • 29.Heap J, Murray M, Miller S, Jalili T, Moyer-Mileur L (2004) Alterations in bone characteristics associated with glycemic control in adolescents with type 1 diabetes mellitus. J Pediatr 144:56–62 [DOI] [PubMed] [Google Scholar]
  • 30.Moyer-Mileur LJ, Slater H, Jordan KC, Murray MA (2008) IGF-1 and IGF-binding proteins and bone mass, geometry, and strength: relation to metabolic control in adolescent girls with type 1 diabetes. J Bone Miner Res 23:1884–1891 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Valerio G, del Puente A, Buono P, Esposito A, Zanatta M, Mozzillo E, Moretto E, Mastidoro L, Franzese A (2004) Quantitative ultrasound of proximal phalanxes in patients with type 1 diabetes mellitus. Diabetes Res Clin Pract 64:161–166 [DOI] [PubMed] [Google Scholar]
  • 32.Janghorbani M, Feskanich D, Willett W, Hu F (2006) Prospective study of diabetes and risk of hip fracture: the Nurses’ Health Study. Diabetes care 29:1573–1578 [DOI] [PubMed] [Google Scholar]
  • 33.Brandao F, Vicente E, Daltro C, Sacramento M, Moreira A, Adan L (2006) Bone metabolism is linked to disease duration andmetabolic control in type 1 diabetes mellitus. Diabetes Res Clin Pract 78:334–339 [DOI] [PubMed] [Google Scholar]
  • 34.Heilman K, Zilmer M, Zilmer K, Tillmann V (2009) Lower bone mineral density in children with type 1 diabetes is associated with poor glycemic control and higher serum ICAM-1 and urinary isoprostane levels. . J Bone Miner Metab 27:598–604 [DOI] [PubMed] [Google Scholar]
  • 35.Janner M, Ballinari P, Mullis PE, Fluck CE (2010) High prevalence of vitamin D deficiency in children and adolescents with type 1 diabetes. Swiss Med Wkly 140:w13091. [DOI] [PubMed] [Google Scholar]
  • 36.Maggio AB, Ferrari S, Kraenzlin M, Marchand LM, Schwitzgebel V, Beghetti M, Rizzoli R, Farpour-Lambert NJ (2010) Decreased bone turnover in children and adolescents with well controlled type 1 diabetes. J Pediatr Endocrinol Metab 23:697–707 [DOI] [PubMed] [Google Scholar]
  • 37.Kujath A, Quinn L, Elliott M, LeCaire T, Binkley N, Molino A, Danielson K (2015) Different health behaviours and clinical factors associated with bone mineral density and bone turnover in premenopausal women with and without type 1 diabetes. Diabetes Metab Res Rev 31:421–432 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Weber D, O’Brien K, Schwartz G (2017) Evidence of disordered calcium metabolism in adolescent girls with type 1 diabetes: An observational study using a dual-stable calcium isotope technique. Bone 105:184–190 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Simmons JH, Raines M, Ness KD, Hall R, Gebretsadik T, Mohan S, Spagnoli A (2011) Metabolic control and bone health in adolescents with type 1 diabetes. Int J Pediatr Endocrinol 2011:13.-9856-2011-13 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.Elhabashy SA, Said O, Agaiby M, Abdelrazek AA, Abdelhamid S (2011) Effect of physical exercise on bone density and remodeling in Egyptian type 1 diabetic osteopenic adolescents. Diabetol Metab Syndr 3 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Maggio A, Rizzoli R, Marchand L, Ferrari S, Beghetti M, Farpour-Lambert N (2012) Physical activity increases bone mineral density in children with type 1 diabetes. Med Sci Sports Exerc 44:1206–1211 [DOI] [PubMed] [Google Scholar]
  • 42.Maser RE, Stabley JN, Lenhard MJ, Provost-Craig MA (2009) Autonomic nerve fiber function and bone mineral density in individuals with type 1 diabetes: a cross-sectional study. Diabetes Res Clin Pract 84:252–258 [DOI] [PubMed] [Google Scholar]
  • 43.Salvatoni A, Mancassola G, Biasoli R, Cardani R, Salvatore S, Broggini M, Nespoli L (2004) Bone mineral density in diabetic children and adolescents: A follow-up study. . Bone 34:900–904 [DOI] [PubMed] [Google Scholar]
  • 44.Joshi A, Varthakavi P, Chadha M, Bhagwat N (2013) A study of bone mineral density and its determinants in type 1 diabetes mellitus. J Osteoporos 2013:397814. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 45.Rosenbloom A, Lezotte D, Weber F, Gudat J, Heller D, Weber M, Klein S, Kennedy B (1977) Diminution of bone mass in childhood diabetes. Diabetes 26:1052–1055 [DOI] [PubMed] [Google Scholar]
  • 46.Rix M, Andreassen H, Eskildsen P (1999) Impact of peripheral neuropathy on bone density in patients with type 1 diabetes. Diabetes Care 22:827–831 [DOI] [PubMed] [Google Scholar]
  • 47.Sayarifard F, Safarirad M, Rabbani A, Sayarifard A, Ziaee V, Setoodeh A, Rostami P (2017) Status of bone mineral density in children with type 1 diabetes mellitus and its related factors. Iran J Pediatr 27 [Google Scholar]
  • 48.Napoli N, Chandran M, Pierroz DD, Abrahamsen B, Schwartz AV, Ferrari SL, IOF Bone and Diabetes Working Group (2017) Mechanisms of diabetes mellitus-induced bone fragility. Nat Rev Endocrinol 13:208–219 [DOI] [PubMed] [Google Scholar]
  • 49.Schneider D, Gauthier B, and Trachtman H (1992) Hypercalciuria in children with renal glycosuria: evidence of dual renal tubular reabsorptive defects. . J Pediatr 121:715–719. [DOI] [PubMed] [Google Scholar]
  • 50.Wood J, Bacha F, Willi SM, Cengiz E, Tamborlane WV, Gregg B, Klingensmith GJ, Schatz DA, Ruedy K, Beck RW, Kollman C, Connor C, Haro H (2014) Youth with type 2 diabetes (T2D) have more vitamin D deficiency than youth with type 1 diabetes (T1D): The pediatric diabetes consortium (PDC). Diabetes 63:A389–A390 [Google Scholar]
  • 51.Mathieu C (2015) Vitamin D and diabetes: Where do we stand? Diabetes Res Clin Pract 108:201–209 [DOI] [PubMed] [Google Scholar]
  • 52.Gregoriou E, Mamais I, Tzanetakou I, Lavranos G, Chrysostomou S (2017) The Effects of Vitamin D Supplementation in Newly Diagnosed Type 1 Diabetes Patients: Systematic Review of Randomized Controlled Trials. Rev Diabet Stud 14:260–268 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 53.Deere K, Sayers A, Rittweger J, and Tobias JH (2012) Habitual levels of high, but not moderate or low, impact activity are positively related to hip BMD and geometry: results from a population-based study of adolescents. J Bone Miner Res 27:1887–1895. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 54.Liese A, Crandell J, Tooze J, Fangman M, Couch S, Merchant A, Bell R, Mayer-Davis E (2015) Relative validity and reliability of an FFQ in youth with type 1 diabetes. Public Health Nutr 18:428–437 [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

592_2019_1334_MOESM1_ESM

RESOURCES