Skip to main content
Heliyon logoLink to Heliyon
. 2024 Mar 10;10(6):e27520. doi: 10.1016/j.heliyon.2024.e27520

Association between weight-adjusted waist index and serum total testosterone in males aged 6–19 years in the United States: Data from NHANES 2013–2016

Zhifei Wu a,1, Lingling Bao b,1, Haiyan Wang a, Jiajing Zheng a, Yu Chen a, Wenjuan Wang b,, Dongkai Qiu a,⁎⁎
PMCID: PMC10944222  PMID: 38496858

Abstract

Objective

This study aimed to estimate the association between weight-adjusted waist index and serum total testosterone (sTT) in males aged 6–19 years in the United States.

Methods

A cross-sectional study was conducted using data from the National Health and Nutrition Examination Survey (NHANES) 2013–2016. sTT was considered as the response variable, and weight-adjusted waist circumference index (WWI) as the independent variable. Multiple linear regression was performed to estimate the association between the two variables, and subgroup analysis was performed to identify sensitive cohorts. Smoothing curve fitting and threshold effects analysis was carried out to assess possible nonlinear relationships between WWI and sTT.

Results

The study included 4207 participants. The mean value of sTT (117.93 ng/dl) was used as the grouping basis, with 1066 participants having serum total testosterone levels above the mean. A negative association was observed between WWI and sTT [beta coefficient (β) = −72.50, 95% confidence interval (CI): -79.45, −65.55], which decreased as WWI increased (P for trend<0.05). Subgroup analysis indicated a stronger negative correlation in late adolescent (16–19 years) males (β = −128.94, 95% CI: −146.75, −111.13). The smoothing curve fit analysis revealed a U-shaped curve relationship for the negative correlation between WWI and sTT. Threshold effect analysis suggested a significant change when WWI exceeded 10.09 (β = −15.82, 95% CI: −24.11, −7.54), and stepwise threshold effect analysis indicated that this negative correlation became less stable when WWI exceeded 11.45 (β = −0.80, 95% CI: −9.15, 7.56).

Conclusions

Participants with higher WWI exhibited lower total testosterone levels, and a negative association was found between WWI and total testosterone, particularly in late adolescent males aged 16–19 years. Among males aged 6–19 years, caution should be exercised regarding the risk of lower testosterone levels associated with elevated WWI, particularly when WWI is below 10.09.

Keywords: Obesity, Weight-adjusted waist index, Testosterone, Cross-sectional study, NHANES

1. Introduction

Testosterone is a steroid hormone testosterone [1], secreted primarily by the testes, that plays an important physiologic role in normal physiologic processes at all life stages. In males, testosterone is the primary sex hormone for the maintenance of secondary sexual characteristics [2,3]. In both genders, testosterone contributes to increased muscle mass and bone density, as well as relieving anxiety [4]. For females, proper testosterone levels are key to maintaining ovarian growth and development [5,6]. In addition to this, testosterone imbalance is a major cause of reproductive dysfunction in both sexes at multiple life stages. In adults, low testosterone levels are associated with decreased semen quality in men [7,8], as well as an increased risk of genital malformations [9]. In female, high testosterone levels are associated with polycystic ovary syndrome (PCOS) [10] and altered pubertal development [11]. Previous studies have shown that androgen deficiency-induced hypogonadism or high testosterone levels-induced PCOS actually occurs as early as adolescence [12,13]. In summary, based on the results of previous studies we found that abnormal changes in testosterone before adulthood are generally recognized as harmful. Therefore, it is clinically important to explore the relationship between specific factors with serum testosterone in children and adolescents.

With the rise in economic prosperity, an increasing number of children and adolescents are experiencing obesity [14]. Obesity carries a significant socioeconomic burden and is associated with various adverse health outcomes, including cardiovascular disease, sleep apnea, osteoarthritis, an increased risk of certain cancers [15]. Earlier studies evaluated the association between testosterone levels and body mass index (BMI), demonstrating that obese men tend to have lower testosterone levels than those with a lower body mass index [16]. It is well known that BMI is the classic index used to assess obesity [16,17]. However, the inability of BMI to differentiate between lean body mass and fat body mass has led to its accuracy being questioned [[18], [19], [20]]. As the understanding of obesity continues to evolve, recent studies have shown that visceral fat, especially abdominal fat, responds to a more realistic obesity [21]. In order to better reflect the real situation of abdominal obesity, Park et al. first proposed a new obesity index, the weight-adjusted waist circumference index (WWI). By adjusting for body weight, the WWI optimizes the benefits of waist circumference (WC) while attenuating the correlation between WC and overall body weight to more accurately measure weight-independent centripetal obesity [22]. A prospective cohort study demonstrated that WWI has better accuracy than BMI, WC, and waist-to-height ratio (WHtR) in predicting the risk of cardiovascular disease mortality [22]. In another cohort study, researchers examined participants' fat and muscle mass using Dual Energy X-ray Absorptiometry (DXA), which showed that WWI was more responsive to participants' adiposity than BMI and WC [23]. The above study suggested the potential advantages of WWI as an indicator of obesity. However, the association between WWI and testosterone levels has not yet been completely conclusive.

To gain a more accurate understanding of the association between obesity and testosterone, this study aimed to estimate the association between WWI and serum total testosterone (sTT) in males aged 6–19 years in the United States. Data from the National Health and Nutrition Examination Survey (NHANES) were utilized for this analysis. The findings of this study will contribute evidence supporting the prevention of obesity-induced testosterone decrease in children and adolescents.

2. Methods

2.1. Data source

The National Health and Nutrition Examination Survey (NHANES), conducted by the National Center for Health Statistics (NCHS), is a public service survey that provides estimates of the nutrition and health status of the United States population. It collects data on demographics, diet, physical examinations, questionnaires, and laboratory tests. Participants in the NAHENS survey provided written consent, and the NCHS research ethics review committee approved the study. Since the NHANES database is publicly available, no additional research ethics review was required for this study.

2.2. Participants

Complete serum total testosterone information was available in the NHANES 2013–2016 dataset, and only participants aged ≥6 years were eligible for testing. A total of 20,146 participants were enrolled in NHANES 2013–2016. Participants younger than 6 years (n = 3207) and older than 19 years (n = 11,488) were excluded. Additionally, participants without complete weight (n = 245) and waist circumference information (n = 167) were excluded to calculate the weight-adjusted waist index (WWI). Participants with incomplete sTT information (n = 826) were also excluded. In addition, we also excluded participants with missing information on education level (n = 2), diabetes (n = 3), and prescription medications (n = 1) due to the small sample size of these participants. Finally, a total of 4207 participants were included in this study (Fig. 1). We examined all participants included in the study to exclude the possibility of current treatment with sex hormone products.

Fig. 1.

Fig. 1

Flowchart of participants. NHANES: National Health and Nutrition Examination Survey.

2.3. Dependent and independent variables

sTT was used as the dependent variable and was obtained from laboratory tests. Total testosterone levels were measured using the isotope dilution liquid chromatography-tandem mass spectrometry (ID-LC-MS/MS) method developed by the Center for Disease Control and Prevention (CDC) for routine analysis. Testosterone was isolated from 100 μL of serum using two consecutive liquid-liquid extraction steps and quantified using [13C] stable isotope-labeled testosterone as an internal standard. This method certified by the CDC Hormone Standardization Project (HoSt), has demonstrated high accuracy and precision over many years of practice [24]. The experimental conditions for total testosterone testing remained consistent throughout NHANES 2013–2016.

WWI was used as the independent variable and was calculated as the waist circumference (cm) divided by the square root of body weight (kg) [25]. Both waist circumference and weight information were obtained from the physical examination section of NHANES 2013–2016, and there were no changes in the measurement methods for waist circumference and weight.

2.4. Covariates

Theoretically, in cross-sectional studies, factors that are potentially related to the independent and dependent variables should be controlled for as covariates. However, many covariates are still unknown or difficult to obtain from existing research data. To estimate the independent effect between WWI and sTT, we adjusted for potential confounders by including them as covariates in the model, as suggested by previous studies [[26], [27], [28]]. Age, gender, race (black, white, and other), educational level (<6th grade, 6–9th grade, and >9th grade), and ratio of family income to poverty (PIR) were derived from the Demographic Variables and Sample Weights questionnaire. Based on previous research [29], we grouped PIR (<1.35,1.35–3.45 and > 3.45) and adjusted them in the models as categorical variables. Dietary information was obtained from the 24-h dietary questionnaire, and in this study the mean value of the intake for specific dietary elements was calculated for the first and second 24-h participants as the final dietary information. Considering the presence of missing values for some of the specific dietary elements, we grouped each specific dietary element based on the median value and set the missing values as a group, which were finally adjusted in the models as categorical variables. Dietary elements included Energy (kcal) (<1800, ≥1800 and Unclear), Sugar (gm) (<100, ≥100 and Unclear), Fat (gm) (<68, ≥68 and Unclear), Cholesterol (mg) (<200, ≥200 and Unclear), Caffeine (mg) (<6, ≥6 and Unclear) and Alcohol (gm) (<0.566 and ≥ 0.566). Due to the strict age requirement for questionnaire information, we did not have access to additional self-reported information regarding disease health and lifestyle habits. Nevertheless, we attempted to obtain the results of the diabetes questionnaire and the prescription medications questionnaire as covariates. Diabetes was determined from subjects' responses to “Ever been told by a doctor that you have diabetes?", with responses included “Yes”, “Borderline”, and “No”. Prescription medications use was defined as the subject's response to the question “Have you used prescription drugs in the past 1 month?", with “Yes" indicating an affirmative response to this question and “No" indicating a negative response. Serum cotinine is an important indicator of exposure to cigarette smoke [30]. Information on serum cotinine was obtained from laboratory tests and measured by isotope dilution high-performance liquid chromatography/atmospheric pressure chemical ionization tandem mass spectrometry (ID HPLC-APCI MS/MS).

2.5. Statistical analysis

The need to use sample weights for cross-sectional studies in complex multistage sampling designs remains controversial [[31], [32], [33]]. However, the NHANES dataset incorporates sample weights that account for complex survey designs, including oversampling, survey nonresponsiveness, and poststratification to ensure that calculated estimates are representative of the civilian noninstitutionalized population in the United States. The official NHANES principle is to choose sample weights with the “lowest common denominator" [34]. In this study, since total testosterone data were derived from Mobile Examination Center (MEC) examination data, we selected sub weights corresponding to serum total testosterone (WTSAF4YR).

All data analysis procedures were performed using R (http://www.R-project.org) and EmpowerStats (http://www.empowerstats.com). Normally distributed continuous variables were expressed as mean value ± standard deviation (SD), otherwise the median (Q1, Q3) was used, and categorical variables were expressed as ratios or percentages. Missing values in covariates were handled using appropriate methods [29]. If the percentage of missing values in a continuous covariate was within 5% of the total sample, it was supplemented with the mean value. Otherwise, the covariates were grouped according to specific criteria, and the missing values were designated as “Unclear" groups. Categorical covariates with no more than 3 missing values were directly removed, while those with more missing values were designated as “Unclear" groups.

Multiple linear regression analysis was performed to assess the association between WWI and sTT. Three models were generated, adjusting for different covariates: Model 1 (no covariates adjusted), Model 2 (adjustment for age and race), and Model 3 (adjustment for all covariates listed in Table 1, except for WWI). Additionally, quartile groupings of WWI were performed to evaluate the trend between WWI and sTT. Age grouping [26] (6–11 years as children; 12–15 years as adolescents, and 16–19 years as late adolescents) was used for subgroup analysis to identify sensitive individuals. A smoothed curve fit was used to assess the presence of a nonlinear relationship between WWI and sTT. Furthermore, a threshold effect analysis was conducted to identify the optimal inflection point (K) if a nonlinear association was observed. In order to determine at which level of WWI the negative association became unstable, a stepwise threshold effect analysis was performed based on the optimal inflection point. The likelihood ratio (LLR) < 0.05 was used as the criterion for selecting the nonlinear model in the threshold effects analysis.

Table 1.

Characteristics of participants.

Characteristics sTT (<117.93) sTT (≥117.93) P-value
Sample size 3141 1066
Age (years) 10.00 (8.00–14.00) 16.00 (14.00–17.00) <0.001
Stratified by age (years) (%) <0.001
6–11 51.38 0.47
12–15 31.30 48.41
16–19 17.32 51.13
Gender (%) <0.001
Male 34.96 99.81
Female 65.04 0.19
Race (%) 0.64
White 62.59 60.98
Black 22.89 23.73
Other races 14.52 15.29
Education level (%) <0.001
<6th grade 69.37 13.70
6–9th grade 14.90 42.50
>9th grade 15.73 43.81
PIR 1.59 (0.82–2.73) 1.66 (0.88–2.74) 0.54
Stratified by PIR (%) 0.29
<1.35 42.82 40.15
1.35–3.45 38.81 41.09
>3.45 18.37 18.76
Diabetes (%) 0.70
Yes 0.48 0.28
Borderline 0.48 0.47
No 99.04 99.25
Prescription medications (%) 0.02
Yes 20.85 17.64
No 79.15 82.36
Serum cotinine (ng/ml) 0.03 (0.01–0.13) 0.04 (0.01–0.63) <0.001
Stratified by serum cotinine (ng/ml) (%) <0.001
<4.923 96.82 86.68
≥4.923 3.18 13.32
Alcohol intake (gm) (%) <0.001
<0.566 88.25 93.06
≥0.566 11.75 6.94
Energy intake (kcal) (%) <0.001
<1800 39.67 27.95
≥1800 35.56 52.53
Unclear 24.77 19.51
Sugar intake (gm) (%) <0.001
<100 37.89 34.24
≥100 37.34 46.25
Unclear 24.77 19.51
Fat intake (gm) (%) <0.001
<68 40.40 29.83
≥68 34.83 50.66
Unclear 24.77 19.51
Cholesterol intake (mg) (%) <0.001
<200 40.69 30.02
≥200 34.54 50.47
Unclear 24.77 19.51
Caffeine (mg) (%) <0.001
<6 38.78 32.08
≥6 36.45 48.41
Unclear 24.77 19.51
WWI 10.91 ± 0.83 9.91 ± 0.74 <0.001

Mean ± SD for continuous variables: P value was calculated by weighted linear regression model. % For Categorical variables: P value as calculated by weighted chi-square test. sTT: serum total testosterone; PIR: ratio of family income to poverty; WWI: weight-adjusted waist index; SD: standard deviation.

3. Results

3.1. Characteristics of participants

A total of 4027 participants were included in this study. Grouping in cross-sectional studies is often based on either the independent variable or the dependent variable. However, in the present study, both independent and dependent variables were continuous variables. In order to better observe the differences of all variables in different sTT groups, we grouped participants according to the mean value of sTT (117.93 ng/dl). The 1066 participants above this mean value had higher age, proportion of males, education level, serum cotinine levels, and obesity-inducing food intake (energy, sugar, fat, cholesterol, caffeine), whereas WWI was lower. The results of the comparison between the two groups are shown in Table 1.

3.2. Association between WWI and sTT

In all models, the results showed a negative association between WWI and sTT level. In the fully adjusted model, the independent effect between the two was (β = −72.50, 95% CI: −79.45, −65.55), indicating that for every 1 unit increase in WWI, sTT decreased by 72.50 ng/dl. This negative association persisted across quartiles of WWI (P for trend < 0.05) and was most pronounced in the Q1 group (β = −266, 95% CI: −310.03, −223.05). Detailed results were presented in Table 2.

Table 2.

Association between WWI and sTT (ng/ml).

Characteristics
  • Model 1

  • β, (95% CI)

  • Model 2

  • β, (95% CI)

  • Model 3

  • β, (95% CI)

WWI −111.84 (−117.37, −106.31) −76.14 (−82.94, −69.35) −72.50 (−79.45, −65.55)
Quintiles of WWI
Q1 (8.24–9.97) −302.54 (−345.20, −259.87) −284.20 (−327.86, −240.53) −266.54 (−310.03, −223.05)
Q2 (9.98–10.62) −87.65 (−143.11, −32.18) −53.35 (−105.70, −1.00) −54.02 (−106.77, −1.28)
Q3 (10.63–11.31) −58.37 (−91.98, −24.77) −16.02 (−46.19, 14.15) −14.99 (−45.15, 15.17)
Q4 (11.32–15.3) −6.26 (−14.79, 2.26) 0.98 (−6.45, 8.41) 0.15 (−7.07, 7.37)
P for trend <0.01 <0.01 <0.01
Stratified by age (years)
6–10 −4.02 (−5.33, −2.70) −4.52 (−5.94, −3.11) −4.11 (−5.45, −2.77)
11–15 −104.24 (−115.65, −92.84) −110.37 (−122.18, −98.56) −95.42 (−107.78, −83.07)
16–19 −140.99 (−157.88, −124.10) −148.06 (−165.52, −130.59) −128.94 (−146.75, −111.13)
P for trend <0.01 <0.01 <0.01

*Model 1: No covariates were adjusted. Model 2: Age, race were adjusted. Model 3: All covariates presented in Table 1 were adjusted except for WWI.

*In the subgroup analysis stratified by each covariate, the model is not adjusted for the stratification variable itself. WWI: weight-adjusted waist index; sTT: serum total testosterone; β: Beta coefficient; CI: confidence interval.

3.3. Results of subgroup analysis

A subgroup analysis was conducted to identify sensitive individuals in the negative association between WWI and sTT and to verify the stability of this association across different subgroups. The results showed that the negative association between WWI and sTT remained stable in all subgroups, except for participants with confirmed diabetes mellitus. The sensitive subgroups included participants with 16–19 years old, male, a family income to poverty ratio (PIR) of 1.35–3.45, no prescription drug use, alcohol intake <0.566 gm, energy intake ≥1800 kcal, sugar intake ≥100 gm, fat intake ≥68 gm, cholesterol intake ≥200 mg, and caffeine intake ≥6 mg. The subgroup analysis results were presented in Table 2, Table 3.

Table 3.

Subgroup analysis of the association between WWI and sTT.

Characteristics
  • Model 1

  • β, (95% CI)

  • Model 2

  • β, (95% CI)

  • Model 3

  • β, (95% CI)

Stratified by gender
Male −162.76 (−170.46, −155.07) −87.80 (−96.16, −79.45) −85.38 (−92.09, −78.68)
Female −7.78 (−8.49, −7.08) −0.47 (−1.19, 0.25) −0.36 (−1.10, 0.37)
Race
White −112.69 (−119.64, −105.73) −74.97 (−83.59, −66.34) −70.80 (−79.56, −62.04)
Black −128.95 (−141.53, −116.36) −90.86 (−105.63, −76.10) −86.79 (−101.70, −71.87)
Other races −125.64 (−140.83, −110.45) −92.60 (−112.24, −72.96) −89.14 (−109.02, −69.25)
Education level
<6th grade −33.84 (−37.85, −29.84) −14.80 (−19.58, −10.02) −14.37 (−19.23, −9.52)
6–9th grade −119.46 (−136.95, −101.97) −122.17 (−139.99, −104.35) −123.57 (−141.99, −105.16)
>9th grade −146.47 (−164.56, −128.37) −155.01 (−173.59, −136.44) −135.58 (−154.39, −116.78)
Stratified by PIR
<1.35 −106.63 (−115.00, −98.25) −74.87 (−85.10, −64.63) −69.66 (−80.03, −59.29)
1.35–3.45 −116.33 (−125.07, −107.60) −82.15 (−92.99, −71.31) −77.09 (−88.06, −66.13)
>3.45 −118.88 (−132.92, −104.84) −69.88 (−87.89, −51.86) −65.85 (−84.46, −47.24)
Diabetes
Yes −86.86 (−169.09, −4.63) −152.92 (−266.63, −39.20) 64.98 (−1234.12, 1364.07)
Borderline 12.83 (−53.33, 79.00) 20.81 (−47.27, 88.90) −87.87 (−155.95, −19.80)
No −112.34 (−117.90, −106.78) −76.09 (−82.94, −69.24) −72.83 (−79.82, −65.84)
Prescription medications
Yes −84.86 (−97.35, −72.37) −58.32 (−73.17, −43.46) −49.00 (−64.04, −33.95)
No −118.16 (−124.31, −112.01) −79.49 (−87.12, −71.86) −78.91 (−86.73, −71.09)
Stratified by serum cotinine (ng/ml)
<4.923 −103.68 (−109.14, −98.22) −71.32 (−78.04, −64.61) −67.45 (−74.35, −60.56)
≥4.923 −165.72 (−199.63, −131.82) −153.80 (−191.20, −116.40) −150.78 (−191.02, −110.53)
Alcohol intake (gm)
<0.566 −111.81 (−117.73, −105.89) −78.13 (−85.36, −70.90) −73.89 (−81.32, −66.45)
≥0.566 −110.01 (−125.69, −94.34) −55.58 (−75.28, −35.88) −50.15 (−69.36, −30.95)
Energy intake (kcal)
<1800 −83.37 (−92.11, −74.63) −63.36 (−73.96, −52.76) −62.33 (−73.11, −51.55)
≥1800 −128.29 (−137.64, −118.94) −83.45 (−94.51, −72.39) −80.31 (−91.80, −68.82)
Unclear −119.52 (−130.31, −108.73) −68.03 (−82.19, −53.87) −68.52 (−82.82, −54.22)
Sugar intake (gm)
<100 −91.74 (−100.82, −82.67) −69.16 (−80.10, −58.23) −68.27 (−79.55, −56.98)
≥100 −124.79 (−133.88, −115.70) −81.15 (−92.08, −70.23) −78.34 (−89.53, −67.14)
Unclear −119.52 (−130.31, −108.73) −68.03 (−82.19, −53.87) −68.52 (−82.82, −54.22)
Fat intake (gm)
<68 −85.55 (−94.26, −76.84) −62.92 (−73.50, −52.35) −61.21 (−72.03, −50.39)
≥68 −128.76 (−138.18, −119.34) −88.21 (−99.39, −77.04) −84.46 (−95.98, −72.93)
Unclear −119.52 (−130.31, −108.73) −68.03 (−82.19, −53.87) −68.52 (−82.82, −54.22)
Cholesterol intake (mg)
<200 −88.73 (−97.34, −80.11) −61.38 (−71.96, −50.81) −61.56 (−72.41, −50.72)
≥200 −126.00 (−135.42, −116.57) −90.81 (−101.87, −79.74) −86.16 (−97.59, −74.73)
Unclear −119.52 (−130.31, −108.73) −68.03 (−82.19, −53.87) −68.52 (−82.82, −54.22)
Caffeine (mg)
<6 −104.35 (−112.64, −96.05) <0.0001 −67.45 (−78.21, −56.70) <0.0001 −68.24 (−79.01, −57.47) <0.0001
≥6 −112.49 (−122.34, −102.64) −88.25 (−99.41, −77.10) −84.36 (−95.91, −72.82)
Unclear −119.52 (−130.31, −108.73) −68.03 (−82.19, −53.87) −68.52 (−82.82, −54.22)

*Model 1: No covariates were adjusted. Model 2: Age, race were adjusted. Model 3: All covariates presented in Table 1 were adjusted except for WWI.

*In the subgroup analysis stratified by each covariate, the model is not adjusted for the stratification variable itself. sTT: serum total testosterone; PIR: ratio of family income to poverty; WWI: weight-adjusted waist index; β: Beta coefficient; CI: confidence interval.

3.4. Smoothed curve fitting and threshold effect analysis

To assess whether the negative association between WWI and sTT was nonlinear, we performed further smoothed curve fitting. The results confirmed that participants with higher WWI had lower sTT level, indicating a negative association between the two. However, the negative association significantly decreased after reaching a certain threshold of WWI, suggesting a U-shaped curve relationship (Fig. 2a and b). We identified some outliers in the smoothing curve, but the nonlinear relationship remained stable when these outliers were removed (Supplementary Fig. 2). Consequently, we conducted a threshold effect analysis, which revealed that the negative association between WWI and sTT was significantly reduced when WWI exceeded 10.09 (β = −15.82, 95% CI: −24.11, −7.54) (Table 4). Subsequently, a stepwise threshold effect analysis was performed, indicating that the negative association between WWI and sTT was no longer stable when WWI exceeded 11.45 (Fig. 3a and b, Supplementary Tables 1–3). In the age subgroups, smoothed fitted curves suggested that although late adolescent males aged 16–19 years had higher sTT level, the decreasing trend in sTT was also most pronounced when WWI increased within a certain WWI range (Fig. 2c). Subsequently, we further performed a threshold effect analysis in the age subgroups (Fig. 3c). The results suggested that the negative association between WWI and sTT had effective inflection point scores of 10.93, 10.14, and 10.02 in the three subgroups of children (6–11 years), the adolescent (12–15 years), and the late adolescent (16–19 years), respectively (Table 5).

Fig. 2.

Fig. 2

The association between WWI and sTT (ng/dl). (a) Each black point represents a sample. (b) Solid rad line represents the smooth curve fit between variables. Blue bands represent the 95% of confidence interval from the fit. (c) The association between WWI and sTT (ng/dl) stratified by age (years). All covariates presented in Table 1 were adjusted except for WWI. sTT: serum total testosterone; WWI: weight-adjusted waist index.

Table 4.

Threshold effect analysis of the association between WWI and sTT (ng/dl).

Outcome: sTT (ng/dl) P-value
Linear effect model
β, (95%CI) −72.50 (−79.45, −65.55) <0.0001
Non-linear model
Inflection point (K) 10.09
β, (95%CI)
<K −271.84 (−289.41, −254.28) <0.0001
≥K −15.82 (−24.11, −7.54) <0.001
LLR <0.001

aAll covariates presented in Table 1 were adjusted except for WWI. sTT: serum total testosterone; WWI: weight-adjusted waist index; β: Beta coefficient; CI: confidence interval; LLR: Likelihood ratio; K: inflection point.

Fig. 3.

Fig. 3

Smoothed curve fit for stepwise threshold effect analysis. (a) WWI ≥10.09; (b) WWI ≥10.24; (c) WWI ≥11.45. *Solid rad line represents the smooth curve fit between variables. Blue bands represent the 95% of confidence interval from the fit. All covariates presented in Table 1 were adjusted except for WWI. sTT: serum total testosterone; WWI: weight-adjusted waist index.

Table 5.

Threshold effect analysis of the association between WWI and sTT stratified by age (years).

Age (years) 6–10 11–15 16–19
Linear effect model
β, (95%CI) −4.11 (−5.45, −2.77) −95.42 (−107.78, −83.07) −128.94 (−146.75, −111.13)
Non-linear model
Inflection point (K) 10.93 10.14 10.02
β, (95%CI)
<K −11.66 (−15.30, −8.02) −262.80 (−291.67, −233.93) −284.83 (−325.30, −244.35)
≥K −1.26 (−3.11, 0.58) −17.83 (−34.79, −0.87) −42.09 (−68.81, −15.36)
LLR <0.001 <0.001 <0.001

*All covariates presented in Table 1 were adjusted except for WWI.

*In the subgroup analysis stratified by each covariate, the model is not adjusted for the stratification variable itself. sTT: serum total testosterone; WWI: weight-adjusted waist index; β: Beta coefficient; CI: confidence interval; LLR: Likelihood ratio; K: inflection point.

4. Discussion

In this cross-sectional study, which included sufficiently large sample size, we found a negative association between WWI and sTT in males aged 6–19 years. Notably, late adolescent males were at a greater risk of reduced sTT due to increased WWI. This negative association remained stable in all subgroups after adjusting for potential confounders. Smoothing curve fit analysis revealed a nonlinear relationship, and threshold effect analysis indicated that the negative association was most significant when WWI did not exceed 10.09. Additionally, we demonstrated that this negative correlation between WWI and sTT was not indefinite and became less stable when WWI exceeded 11.45.

Previous studies have shown that sTT decreases with increasing BMI [35]. While we acknowledge the plausibility of such results, it is important to note that BMI as a traditional index, does not accurately reflect obesity. WWI, a newly developed index for assessing obesity, has been studied in various fields and has shown advantages over BMI [23,[36], [37], [38]]. Therefore, we believe that our results, which utilized WWI, are more reliable. Furthermore, based on the supplementary materials, we also found that WWI was a more suitable predictor for sTT than BMI (Supplementary Fig. 1).

sTT levels have been shown to correlate closely with age, with adult testosterone levels decreasing with increasing age [39,40]. However, males experience several peak periods before reaching adulthood, such as the first few months of life, adolescence, and the post-pubertal months [39]. Consistent with previous studies, we observed a higher proportion of participants with higher sTT in the older age subgroup in our study. However, we also discovered that the negative association between WWI and sTT was more pronounced in adolescent and late-adolescent males. This finding may be attributed to the higher prevalence of obesity among males in these age groups [41].

Although our study demonstrated that participants with higher WWI had lower sTT, we observed that this negative association was not linear. The independent effect of the negative correlation diminished when WWI exceeded 10.09. Additionally, the negative correlation was not indefinite, and the relationship became less stable when WWI exceeded 11.45. Notably, no relevant cross-sectional study has reported a similar pattern, even when considering BMI as the independent variable [35]. Previous studies have suggested that sTT would be reduced by approximately 40%–50% in men with a BMI above 35 kg/m2 [42]. When reviewing the outcome of the smoothed curve fit, we found that the rate of decrease in total testosterone corresponding to a WWI of 10.09 was consistent with the findings of previous studies (using sTT of 400 ng/dl as a reference). While there is no exact standard value for sTT at optimal physiological levels [43], we considered the sample size corresponding to an sTT of 400 ng/dl to be adequate, justifying the results of our study. Testosterone is primarily secreted by the testes in men, but it is also produced in the adrenal cortex, liver, kidney, adipose, and muscle tissues [44]. Therefore, we speculate that when WWI reaches a certain value, it may promote compensatory production of testosterone in other tissues, leading to the above phenomenon.

We observed a negative association between WWI and sTT and identified several subgroups where this association was particularly significant. Notably, participants with higher serum cotinine levels, which serve as indicators of exposure to cigarette smoke, showed a stronger association. However, previous studies have yielded conflicting results regarding the impact of smoking on testosterone levels. Wang W [45] and Svartberg J [46] et al. found significantly higher sTT in smokers compared to nonsmokers [45], while Halmenschlager G [47] and Shaarawy M [48] et al. did not find such a difference. Given that our study focused on participants aged 6–19 years, resulting in a lower proportion of participants with higher serum cotinine levels further research is necessary to validate our findings. Additionally, we observed a more pronounced negative association between WWI with sTT in the subgroup with higher consumption of energy, sugar, fat, cholesterol, and caffeine. Since these dietary factors can contribute to higher WWI [49,50], it is reasonable to draw conclusions based on these findings.

However, there were still some limitations in this study. Firstly, there are numerous potential factors affecting WWI and sTT, and some of these factors may require further exploration by researchers. As we were unable to adjust for all potential confounders using the NHANES database, the conclusions of this study should be validated by subsequent research. Secondly, Although we attempted to obtain information on participants' use of prescription medications and adjusted for this, as well as checking that current participants were not treated with exogenous sex hormones, reliance on patient self-reporting is biased, leading us to remain unable to guarantee with certainty the possibility that exogenous hormone use confounded the study's conclusions. Also, the medical questionnaire in NHANES did not contain more information about endocrine disorders, so we were unable to clarify whether any of the participants were at risk for underlying endocrine disorders. Thirdly, due to the characteristics of participants aged 6–19 years, they were unable to participate in certain self-report questionnaires regarding smoking and alcohol consumption. However, we made efforts to obtain information on alcohol intake from dietary questionnaires and serum cotinine levels from laboratory tests, supporting the reliability of our conclusions. Finally, while an adequate sample and the use of sample weights ensure the reliability of the results, cross-sectional studies cannot establish causality, and the generalizability of our findings to other regions and populations requires validation in subsequent studies.

5. Conclusions

In this study, we observed a negative association between n WWI and sTT in males aged 6–19 years in the United States. As WWI increased, sTT levels decreased, although this negative correlation diminished as WWI further increased. Specifically, the negative association between WWI and sTT was most pronounced when WWI exceeded 10.09, and it became unstable when WWI exceeded 11.45. Furthermore, we found that with increasing age, adolescent and late adolescent participants were more susceptible to a decrease in sTT due to higher WWI. Additionally, participants with male, a PIR of 1.35–3.45, no prescription drug use, alcohol intake <0.566 gm, serum cotinine ≥4.923 ng/ml, energy intake ≥1800 kcal, sugar intake ≥100 gm, fat intake ≥68 gm, cholesterol intake ≥200 mg, and caffeine intake ≥6 mg should be particularly cautious about the potential reduction in sTT induced by higher WWI.

Ethics approval and consent to participate

The NCHS Research Ethics Review Committee approved the NHANES survey protocol (https://www.cdc.gov/nchs/nhanes/irba98.htm), and all participants of the study provided informed written consent. The NHANES database is open to the public and therefore the ethical review of this study was exempt.

Funding

Not applicable.

Data availability

Data included in article/supp. material/referenced in article.

Consent for publication

Not applicable.

CRediT authorship contribution statement

Zhifei Wu: Writing – original draft, Data curation, Conceptualization. Lingling Bao: Writing – review & editing, Project administration, Data curation, Conceptualization. Haiyan Wang: Software. Jiajing Zheng: Methodology. Yu Chen: Visualization, Data curation. Wenjuan Wang: Writing – review & editing, Supervision, Project administration. Dongkai Qiu: Writing – review & editing, Project administration.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgements

We expressed the gratitude of all the participants and staff of NHANES.

Footnotes

Appendix A

Supplementary data to this article can be found online at https://doi.org/10.1016/j.heliyon.2024.e27520.

Contributor Information

Wenjuan Wang, Email: wwjwjbwsy@163.com.

Dongkai Qiu, Email: ap2005ap@163.com.

Appendix A. Supplementary data

The following is the Supplementary data to this article.

Multimedia component 1
mmc1.doc (13.5MB, doc)

References

  • 1.Handelsman D.J., Hirschberg A.L., Bermon S. Circulating testosterone as the hormonal basis of sex differences in athletic performance. Endocr. Rev. 2018;39:803–829. doi: 10.1210/er.2018-00020. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Surampudi P.N., Wang C., Swerdloff R. Hypogonadism in the aging male diagnosis, potential benefits, and risks of testosterone replacement therapy. Internet J. Endocrinol. 2012;2012 doi: 10.1155/2012/625434. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Bassil N., Alkaade S., Morley J.E. The benefits and risks of testosterone replacement therapy: a review. Therapeut. Clin. Risk Manag. 2009;5:427–448. doi: 10.2147/tcrm.s3025. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Goodale T., Sadhu A., Petak S., Robbins R. Testosterone and the heart. Methodist. Debakey. Cardiovasc. J. 2017;13:68–72. doi: 10.14797/mdcj-13-2-68. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Pluchino N., Carmignani A., Cubeddu A., Santoro A., Cela V., Errasti T. Androgen therapy in women: for whom and when. Arch. Gynecol. Obstet. 2013;288:731–737. doi: 10.1007/s00404-013-2969-7. [DOI] [PubMed] [Google Scholar]
  • 6.Davis S.R., Wahlin-Jacobsen S. Testosterone in women--the clinical significance. Lancet Diabetes Endocrinol. 2015;3:980–992. doi: 10.1016/S2213-8587(15)00284-3. [DOI] [PubMed] [Google Scholar]
  • 7.Carlsen E., Giwercman A., Keiding N., Skakkebaek N.E. Evidence for decreasing quality of semen during past 50 years. BMJ. 1992;305:609–613. doi: 10.1136/bmj.305.6854.609. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Rolland M., Le Moal J., Wagner V., Royère D., De Mouzon J. Decline in semen concentration and morphology in a sample of 26,609 men close to general population between 1989 and 2005 in France. Hum. Reprod. 2013;28:462–470. doi: 10.1093/humrep/des415. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Main K.M., Skakkebaek N.E., Virtanen H.E., Toppari J. Genital anomalies in boys and the environment. Best Pract. Res. Clin. Endocrinol. Metabol. 2010;24:279–289. doi: 10.1016/j.beem.2009.10.003. [DOI] [PubMed] [Google Scholar]
  • 10.Escobar-Morreale H.F. Polycystic ovary syndrome: definition, aetiology, diagnosis and treatment. Nat. Rev. Endocrinol. 2018;14:270–284. doi: 10.1038/nrendo.2018.24. [DOI] [PubMed] [Google Scholar]
  • 11.Cole T.J., Ahmed M.L., Preece M.A., Hindmarsh P., Dunger D.B. The relationship between Insulin-like Growth Factor 1, sex steroids and timing of the pubertal growth spurt. Clin. Endocrinol. 2015;82:862–869. doi: 10.1111/cen.12682. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Flannigan R., Patel P., Paduch D.A. Klinefelter syndrome. The effects of early androgen therapy on competence and behavioral phenotype. Sex Med. Rev. 2018;6:595–606. doi: 10.1016/j.sxmr.2018.02.008. [DOI] [PubMed] [Google Scholar]
  • 13.American association of clinical endocrinologists position statement on metabolic and cardiovascular consequences of polycystic ovary syndrome. Endocr. Pract. 2005;11:126–134. doi: 10.4158/EP.11.2.125. [DOI] [PubMed] [Google Scholar]
  • 14.Hills A.P., Andersen L.B., Byrne N.M. Physical activity and obesity in children. Br. J. Sports Med. 2011;45:866–870. doi: 10.1136/bjsports-2011-090199. [DOI] [PubMed] [Google Scholar]
  • 15.Fui M.N., Dupuis P., Grossmann M. Lowered testosterone in male obesity: mechanisms, morbidity and management. Asian J. Androl. 2014;16:223–231. doi: 10.4103/1008-682X.122365. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Glass A.R., Swerdloff R.S., Bray G.A., Dahms W.T., Atkinson R.L. Low serum testosterone and sex-hormone-binding-globulin in massively obese men. J. Clin. Endocrinol. Metab. 1977;45:1211–1219. doi: 10.1210/jcem-45-6-1211. [DOI] [PubMed] [Google Scholar]
  • 17.Allan C.A., McLachlan R.I. Androgens and obesity. Curr. Opin. Endocrinol. Diabetes Obes. 2010;17:224–232. doi: 10.1097/MED.0b013e3283398ee2. [DOI] [PubMed] [Google Scholar]
  • 18.Flegal K.M., Kit B.K., Orpana H., Graubard B.I. Association of all-cause mortality with overweight and obesity using standard body mass index categories: a systematic review and meta-analysis. JAMA. 2013;309:71–82. doi: 10.1001/jama.2012.113905. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Heymsfield S.B., Cefalu W.T. Does body mass index adequately convey a patient's mortality risk. JAMA. 2013;309 doi: 10.1001/jama.2012.185445. 87–8. [DOI] [PubMed] [Google Scholar]
  • 20.Javed A., Jumean M., Murad M.H., Okorodudu D., Kumar S., Somers V.K., et al. Diagnostic performance of body mass index to identify obesity as defined by body adiposity in children and adolescents: a systematic review and meta-analysis. Pediatr. Obes. 2015;10:234–244. doi: 10.1111/ijpo.242. [DOI] [PubMed] [Google Scholar]
  • 21.Thomas E.L., Frost G., Taylor-Robinson S.D., Bell J.D. Excess body fat in obese and normal-weight subjects. Nutr. Res. Rev. 2012;25:150–161. doi: 10.1017/S0954422412000054. [DOI] [PubMed] [Google Scholar]
  • 22.Park Y., Kim N.H., Kwon T.Y., Kim S.G. A novel adiposity index as an integrated predictor of cardiometabolic disease morbidity and mortality. Sci. Rep. 2018;8 doi: 10.1038/s41598-018-35073-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Kim N.H., Park Y., Kim N.H., Kim S.G. Weight-adjusted waist index reflects fat and muscle mass in the opposite direction in older adults. Age Ageing. 2021;50:780–786. doi: 10.1093/ageing/afaa208. [DOI] [PubMed] [Google Scholar]
  • 24.Määttänen I., Gluschkoff K., Komulainen K., Airaksinen J., Savelieva K., García-Velázquez R., et al. Testosterone and specific symptoms of depression: evidence from NHANES 2011-2016. Compr. Psychoneuroendocrinol. 2021;6 doi: 10.1016/j.cpnec.2021.100044. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Qin Z., Chang K., Yang Q., Yu Q., Liao R., Su B. The association between weight-adjusted-waist index and increased urinary albumin excretion in adults: a population-based study. Front. Nutr. 2022;9 doi: 10.3389/fnut.2022.941926. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Tao C., Fan Y., Niu R., Li Z., Qian H., Yu H., et al. Urinary polycyclic aromatic hydrocarbons and sex hormones in children and adolescents: evidence from NHANES. Ecotoxicol. Environ. Saf. 2021;216 doi: 10.1016/j.ecoenv.2021.112215. [DOI] [PubMed] [Google Scholar]
  • 27.Luo K., Liu J., Wang Y., Aimuzi R., Luo F., Ao J., et al. Associations between organophosphate esters and sex hormones among 6-19-year old children and adolescents in NHANES 2013-2014. Environ. Int. 2020;136 doi: 10.1016/j.envint.2020.105461. [DOI] [PubMed] [Google Scholar]
  • 28.Su M., Wei H., Chen L., Guan Y., Dong W., Zhao M. The impact of visceral adiposity on testosterone levels in American adult men: a cross-sectional analysis. Med. Sci. Monit. 2023;29 doi: 10.12659/MSM.941394. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Han K., Tan K., Shen J., Gu Y., Wang Z., He J., et al. Machine learning models including insulin resistance indexes for predicting liver stiffness in United States population: data from NHANES. Front. Public Health. 2022;10 doi: 10.3389/fpubh.2022.1008794. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Kim S. Overview of cotinine cutoff values for smoking status classification. Int. J. Environ. Res. Publ. Health. 2016:13. doi: 10.3390/ijerph13121236. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Korn E.L., Graubard B.I. Epidemiologic studies utilizing surveys: accounting for the sampling design. Am. J. Publ. Health. 1991;81:1166–1173. doi: 10.2105/ajph.81.9.1166. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.James-Todd T., Stahlhut R., Meeker J.D., Powell S.G., Hauser R., Huang T., et al. Urinary phthalate metabolite concentrations and diabetes among women in the National Health and Nutrition Examination Survey (NHANES) 2001-2008. Environ. Health Perspect. 2012;120:1307–1313. doi: 10.1289/ehp.1104717. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Lewis R.C., Meeker J.D. Biomarkers of exposure to molybdenum and other metals in relation to testosterone among men from the United States National Health and Nutrition Examination Survey 2011-2012. Fertil. Steril. 2015;103:172–178. doi: 10.1016/j.fertnstert.2014.09.020. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Johnson C.L., Paulose-Ram R., Ogden C.L., Carroll M.D., Kruszon-Moran D., Dohrmann S.M., et al. National health and nutrition examination survey: analytic guidelines, 1999–2010. Vital Health Stat. 2013;2:1–24. [PubMed] [Google Scholar]
  • 35.Lokeshwar S.D., Patel P., Fantus R.J., Halpern J., Chang C., Kargi A.Y., et al. Decline in serum testosterone levels among adolescent and young adult men in the USA. Eur. Urol. Focus. 2021;7:886–889. doi: 10.1016/j.euf.2020.02.006. [DOI] [PubMed] [Google Scholar]
  • 36.Li Q., Qie R., Qin P., Zhang D., Guo C., Zhou Q., et al. Association of weight-adjusted-waist index with incident hypertension: the Rural Chinese Cohort Study. Nutr. Metabol. Cardiovasc. Dis. 2020;30:1732–1741. doi: 10.1016/j.numecd.2020.05.033. [DOI] [PubMed] [Google Scholar]
  • 37.Kim J.Y., Choi J., Vella C.A., Criqui M.H., Allison M.A., Kim N.H. Associations between weight-adjusted waist index and abdominal fat and muscle mass: multi-ethnic study of atherosclerosis. Diabetes Metab. J. 2022;46:747–755. doi: 10.4093/dmj.2021.0294. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Cai S., Zhou L., Zhang Y., Cheng B., Zhang A., Sun J., et al. Association of the weight-adjusted-waist index with risk of all-cause mortality: a 10-year follow-up study. Front. Nutr. 2022;9 doi: 10.3389/fnut.2022.894686. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Kyriakopoulou L., Yazdanpanah M., Colantonio D.A., Chan M.K., Daly C.H., Adeli K. A sensitive and rapid mass spectrometric method for the simultaneous measurement of eight steroid hormones and CALIPER pediatric reference intervals. Clin. Biochem. 2013;46:642–651. doi: 10.1016/j.clinbiochem.2013.01.002. [DOI] [PubMed] [Google Scholar]
  • 40.Deutschbein T., Mann K., Petersenn S. Total testosterone and calculated estimates for free and bioavailable testosterone: influence of age and body mass index and establishment of sex-specific reference ranges. Horm. Metab. Res. 2015;47:846–854. doi: 10.1055/s-0034-1395569. [DOI] [PubMed] [Google Scholar]
  • 41.Zwiauer K.F. Prevention and treatment of overweight and obesity in children and adolescents. Eur. J. Pediatr. 2000;159(Suppl 1):S56–S68. doi: 10.1007/pl00014367. [DOI] [PubMed] [Google Scholar]
  • 42.Dhindsa S., Miller M.G., McWhirter C.L., Mager D.E., Ghanim H., Chaudhuri A., et al. Testosterone concentrations in diabetic and nondiabetic obese men. Diabetes Care. 2010;33:1186–1192. doi: 10.2337/dc09-1649. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43.Aversa A., Morgentaler A. The practical management of testosterone deficiency in men. Nat. Rev. Urol. 2015;12:641–650. doi: 10.1038/nrurol.2015.238. [DOI] [PubMed] [Google Scholar]
  • 44.Schmidtová E. [Testosterone--effects, metabolism and genetic determination] Cesk. Fysiol. 2008;57:61–75. [PubMed] [Google Scholar]
  • 45.Wang W., Yang X., Liang J., Liao M., Zhang H., Qin X., et al. Cigarette smoking has a positive and independent effect on testosterone levels. Hormones. 2013;12:567–577. doi: 10.14310/horm.2002.1445. [DOI] [PubMed] [Google Scholar]
  • 46.Svartberg J., Jorde R. Endogenous testosterone levels and smoking in men. The fifth Tromsø study. Int. J. Androl. 2007;30:137–143. doi: 10.1111/j.1365-2605.2006.00720.x. [DOI] [PubMed] [Google Scholar]
  • 47.Halmenschlager G., Rossetto S., Lara G.M., Rhoden E.L. Evaluation of the effects of cigarette smoking on testosterone levels in adult men. J. Sex. Med. 2009;6:1763–1772. doi: 10.1111/j.1743-6109.2009.01227.x. [DOI] [PubMed] [Google Scholar]
  • 48.Shaarawy M., Mahmoud K.Z. Endocrine profile and semen characteristics in male smokers. Fertil. Steril. 1982;38:255–257. doi: 10.1016/s0015-0282(16)46470-8. [DOI] [PubMed] [Google Scholar]
  • 49.Jääskeläinen A., Kaila-Kangas L., Leino-Arjas P., Lindbohm M.L., Nevanperä N., Remes J., et al. Association between occupational psychosocial factors and waist circumference is modified by diet among men. Eur. J. Clin. Nutr. 2015;69:1053–1059. doi: 10.1038/ejcn.2015.59. [DOI] [PubMed] [Google Scholar]
  • 50.Bowman S.A., Gortmaker S.L., Ebbeling C.B., Pereira M.A., Ludwig D.S. Effects of fast-food consumption on energy intake and diet quality among children in a national household survey. Pediatrics. 2004;113:112–118. doi: 10.1542/peds.113.1.112. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Multimedia component 1
mmc1.doc (13.5MB, doc)

Data Availability Statement

Data included in article/supp. material/referenced in article.


Articles from Heliyon are provided here courtesy of Elsevier

RESOURCES