Skip to main content
International Journal of Environmental Research and Public Health logoLink to International Journal of Environmental Research and Public Health
. 2021 May 2;18(9):4862. doi: 10.3390/ijerph18094862

The Effects of the Combination of High-Intensity Interval Training with 3D-Multiple Object Tracking Task on Perceptual-Cognitive Performance: A Randomized Controlled Intervention Trial

Soo-Yong Park 1,*, Thomas Jürgen Klotzbier 1, Nadja Schott 1
Editors: Pantelis T Nikolaidis1, Cristina Cortis1
PMCID: PMC8125741  PMID: 34063304

Abstract

The ability to process goal-related visual information while ignoring goal-irrelevant information is essential for the human attention system. The study aimed to investigate how perceptual–cognitive performance was affected during high-intensity interval training (HIIT) using a 3D-multiple object tracking (3D-MOT) task called Neurotracker (NT). In an experimental design, 42 healthy adults (age M = 23.3 SD = 2.94, VO2max 52.8 ± 5.66 mL·kg−1·min−1) were randomly assigned to an intervention (HIIT + NT, NT, HIIT) or control group. NT performance (20 trials per session) was measured pre-and post-test (at 5, 15, and 25 min while running on the treadmill). The participants trained twice a week for a 4-week intervention period. There was a significant interaction effect between pre/post-test and groups regarding perceptual-cognitive performance, indicating similar enhancements in the HIIT + NT and the NT group during exercise. HIIT influences physical fitness but did not show any impact on perceptual–cognitive performance. Due to the specific NT task characteristics, improved physical abilities may not directly impact sport-specific perceptual-cognitive performance. Our findings suggest that training resulted in substantial task-specific gains. Therefore, combination training may be proposed as a training program to improve perceptual–cognitive, and physical performance in a time-efficient way.

Keywords: 3D-multiple object tracking, cognitive training, perceptual–cognitive skills, working memory, high-intensity interval training

1. Introduction

A flexible allocation of attention resources seems essential (e.g., during a soccer match) to process goal-oriented visual information while selecting irrelevant information. The human attention system must adapt to perceptual–cognitive demands in various sports situations [1]. Perceptual–cognitive performance in sports contexts is considered to play a crucial role for competitive athletes [2]. This is especially true for ball games where athletes are frequently confronted with cognitively demanding situations that require the simultaneous processing of different pieces of information with varying exercise intensities and durations [3]. Intermittent practice with repetitions of short and/or long and high and/or moderate intensities is required to meet the demands of team sports such as soccer, handball, basketball, and hockey.

While perceptual–cognitive skills are typically developed through regular, on-field practice, several training techniques can enhance those skills outside of or in conjunction with regular training (e.g., perceptual–cognitive training (computer projection, Virtual Reality, Quiet eye training, and 3D-MOT task)) [4]. Sports training (and the experience gained through training) is a form of cognitive training that leads to more efficient brain networks and plasticity, leading to cognitive process improvements [2,5,6]. Based on these studies, Faubert and Sidebottom [7] introduced a perceptual–cognitive training program in the form of a 3D-multiple object tracking task called Neurotracker (NT; CogniSens Inc.; Montreal, QC, Canada). NT has been suggested as a cognitive training program to isolate and improve specific cognitive abilities such as selective visual attention or working memory while tracking multiple objects moving in 3D-space. [8]. This training program, which stimulates many brain networks that work together during exercise, including complex motion integration, dynamic, sustained, distributed attention processing, and working memory [9], is of great importance in a dynamic sport context [7]. In most cognitive tasks, response time and correct response rate are assessed for a given number of items at a given fixed speed [10]. Since the NT-system can use the speed threshold as a dependent variable indicating the difference between individuals, the degree of difficulty can be individualized during NT-training, and performance changes can be continuously observed [7]. Previous studies have already demonstrated the training effects of cognitive performance through NT-training, suggesting that optimal processing of sports-related visual scenes and perceptual–cognitive abilities can be trained in athletes [7,11,12,13]. However, perceptual–cognitive performance using NT training has not been studied during physical exercise.

Only a limited number of longitudinal studies evaluate the long-term effects of different exercise interventions on cognitive performance. Most studies show an improvement in various attention components, neurophysiological measures, or both. A recent systematic review by Fernandes et al. [14] indicates that most longitudinal studies are limited to older adults, especially women. Only two studies have been conducted with children [15] and young adult men [16]. While most studies with older adults and children [15,17,18] exhibited positive results, the only study with young adults [16] showed no positive long-term improvements with training during the performance of an attention-demanding task. Fernandes et al. [14] justify this finding by stating that the study by Gutmann et al. [16] used a very low exercise intensity (50% of peak power). Overall, it appears that aerobic exercise, given an adequate training stimulus, is an effective method for improving cognitive performance on both behavioral [17,19,20] and neuroelectric indices [17]. Other potentially effective aerobic training methods, such as HIIT, have not been investigated to the best of our knowledge. Therefore, this study uses a HIIT protocol to examine its effect on cognitive performance.

The current literature shows that the relationship appears to depend on the type of exercise, duration [21], intensity [22,23], physical fitness and demographics [22,24,25], as well as timing and type of cognitive tasks assessed [2,22,23,26]. In the past few years, most studies have been dedicated to examining the effect of different exercise training modalities on cognitive and physical functioning and have measured cognitive performance mainly after exercise and at rest, but rarely during exercise. However, these situations could be of great importance, especially in sports games. Few researchers have studied cognitive performance during a High-Intensity Interval Training (HIIT) protocol, and it is rarely discussed how cognitive performance is influenced during physical exercise. For example, Smith et al. [27] reported that the decision error rate deteriorates during high-intensity exercise. Casanova et al. [27] demonstrated decreased accuracy in a decision-making task due to a prolonged intermittent exercise protocol. HIIT has been shown to produce similar cardiovascular and metabolic health benefits more time-efficient than continuous moderate-intensity exercise. More importantly, it leads to more sustained improvements in prefrontal cortex-dependent executive function during recovery from exercise [28,29]. One of the fundamental mechanisms responsible for high-intensity exercise having a positive effect on cognitive function is the higher brain-derived neurotrophic factor (BDNF) and catecholamine concentrations induced by high-intensity exercise. However, this requires further investigation [30]. It is essential to investigate the longitudinal effects of HIIT on perceptual–cognitive performance to make context-specific and individual recommendations for action concerning exercise load. Thus, we examine the intervention effect of exercise on cognitive performance during physical exercise.

The purpose of this study was to investigate a four-week-intervention effect of HIIT in combination with and without a 3D-multiple object tracking task (NT). Systematic reviews have found that the combination of physical and cognitive training could improve cognition in a more specific manner [31,32]. We thus hypothesized that a combination of HIIT and NT-training would be particularly associated with enhanced long-term effects on perceptual–cognitive performance during physical exercise. This intervention effect may positively impact many team sports by compensating for the decline in cognitive performance during high-intensity exercise.

2. Materials and Methods

2.1. Participants

Forty-two healthy young adults (M = 23.3, SD = 2.94, age-range 19–30, 13 women) participants volunteered to participate in the experiment. They were recruited from the University Sports Center of the University of Stuttgart, Germany. A questionnaire for sports biography was used to determine what type of sport the participants did and how often they exercised per week to participate in the intervention study. Since sport type determines differences in executive functions in elite athletes [33,34], the sports biography was categorized into static sports (self-paced activities in highly consistent circumstances, e.g., taekwondo, athletics, and dancing, etc.), strategic sports (requiring the adaption to highly varying situations considering teammates, opponents, positions or objects, e.g., soccer, basketball, handball, and volleyball, etc.) or interceptive sports (requiring dynamic coordination between athlete′s body and an implement or environment, e.g., table tennis and tennis, etc.) as well as sport-unspecific training such as strength and endurance training (see [6]). The different categories′ participants were randomly assigned (random number generation function in Excel, [35]) to the three experimental conditions and the control group. All participants received verbal and written information about the study′s nature and aim and signed a consent form. All procedures were in accordance with the Declaration of Helsinki with ethical standards, legal requirements, and international norms. An internal ethics committee of the university approved the study.

2.2. Experimental Design and Procedure

To analyze the effects of HIIT on perceptual–cognitive performance during exercise and to examine differences in these changes across interventions, we employed a 4 (intervention: high-intensity interval training + Neurotracker [HIIT + NT] vs. Neurotracker [NT] vs. high-intensity interval training [HIIT] vs. control) × 6 (during HIIT protocol at 5 min, 15 min, and 25 min in pre-and post-test) mixed factorial design. Participants were randomly assigned to one of the four intervention protocols (n = 10/11 per intervention group). They were tested twice before and after a 4-week intervention period.

Baseline assessments: On the first day, the participants were informed about the study′s procedures and requirements. After that, they gave their written consent and completed the questionnaires. Then, all subjects were asked to perform two sessions (20 trials per session) as a practice version to familiarize themselves with Neurotracker. After a short break, VO2max was assessed using the Bruce protocol. Then, participants were randomly divided into four groups (HIIT + NT, NT, HIIT, control). Pre-test: Approximately 48 h to 72 h later, the participants returned to the lab and performed the 3D-MOT task (NT) during the individualized HIIT. Each NT assessment lasted approximately 6 min and was performed at 5 min, 15 min, and 25 min during HIIT, respectively. The HIIT protocol was controlled and administered using the h/p/cosmos para control® 4.1.0. Part II of the Mental and Physical State and Trait Energy and Fatigue Scales Questionnaire (MPSTEFS; [36]) was administered before and after testing. Intervention: All participants in the active groups (HIIT + NT, NT, HIIT) completed their respective intervention program twice a week, with approximately two days in-between for four weeks in our laboratory. Post-test: After the 4-week training program, all participants were reassessed on three NT sessions during HIIT. Again two days later, the cardiovascular fitness assessment (Bruce protocol) was performed (Figure 1).

Figure 1.

Figure 1

Study design and procedure. The study employed a 4 (four groups) × 6 (5, 15, and 25 min in pre-and post-test) mixed factorial design. HIIT: High-intensity interval training; NT: Neurotracker.

2.3. Intervention

The HIIT treadmill protocol used in the present study consisted of eleven intervals of 30 s at 90% VO2max, interspersed by 2 min active recovery periods at 50% VO2max. The duration of each HIIT session was approximately 30 min, not including the warm-up phase (5 min) and the cool-down phase (3 min). The participants of the HIIT group performed only the HIIT protocol. The HIIT + NT group performed three NT-sessions with 20 trials at three-time points (at 5 min, 15 min, and 25 min) during the HIIT protocol. The NT group also completed three NT-sessions with 20 trials and similar breaks between the NT-sessions as the HIIT + NT experimental group. The control group received no treatment (Figure 1).

2.4. Demographic Information, BMI, Physical Activity, and Mental/Physical Fatigue

Participants′ demographic information was recorded, participants′ height and weight were measured, and body mass index (BMI, kg/m2) was calculated. Additionally, the questionnaire recorded the type of sport practiced by the participants in sports clubs and the number of training sessions they executed per week, the duration of the training sessions, the level of athletic performance, the age at which the participants began regular training, and the athletic success achieved in participation in competitions (local, regional, national, or international tournaments). Participants also completed Part II of MPSTEFS [36] to detect changes in perceived feelings of energy and fatigue before and after the pre-and post-test experiment. A total of 12 separate ratings were obtained for Physical Energy, Physical Fatigue, Mental Energy, and Mental Fatigue.

2.5. 3D-Multiple Object Tracking Task

The 3D-MOT (3D Multiple Object Tracking) is a perceptual–cognitive training program under the Neurotracker (NT) system licensed by the University of Montreal (CogniSens Inc.; Montreal, QC, Canada). It is a commercial equivalent of the 3D multiple object tracking [8] for measuring visual tracking speed (VTS) [12]. The VTS score (i.e., NT score or performance) has proven to be a valid indicator of high-order cognitive function [9,37]. The transfer effect onto the sports field for athletes was also demonstrated [9].

Participants stood or ran on a treadmill with 3D-glasses at a 46-degree angle in front of a 3D television (Samsung, 65 inches; Figure 2). In the first phase, eight balls were presented, then four of these eight balls were highlighted in a different color for one second and then returned to their original color. In the next phase, the balls moved at a defined speed in 3D-space for eight seconds. The participant′s task was to track the four initially color-marked balls and then, when they came to a stop, to give a verbal response that identified the four balls illuminated originally at the beginning of the experiment by their number. This was followed by a feedback phase in which the participant named the balls′ numbers, and the subsequent trial began. Participants had to keep their gaze centered and peripherally follow the balls. The five phases of the NT task that occurred within a trial can be seen in Figure 3.

Figure 2.

Figure 2

Participant performing the NT task during running on the treadmill.

Figure 3.

Figure 3

3D-MOT (Multiple object tracking) task. (a) Presentation of randomly positioned spheres in a virtual volumetric space; (b) Identification of the spheres to be tracked during the trial; (c) Removal of identification and movement of all spheres with dynamic interactions; (d) Observer′s response by identifying the spheres; (e) Feedback is given to the observer Reprinted with permission from ref. [8]. Copyright 2013 Jocelyn Faubert.

A session consists of 20 trials, lasting a total of approximately 6–7 min. The balls′ speed is adjusted from trial to trial, depending on the accuracy of the response. If all balls are correctly identified, the speed at which the balls move through 3D-space is increased by 0.05 log in the subsequent trial [9,38]. However, if the answers are partially or entirely incorrect, the balls′ speed is reduced by the same proportion, making the task easier. This method is used to calculate a session-specific speed threshold (staircase method; [39]), which is then used as a performance measure (NT performance). The speed at which a new session starts is also adjusted to the performance of the previous session. Overall, this results in a highly adaptive and individualized procedure within and across sessions. VTS has been defined as the fastest speed (in centimeters per second) at which the participant can correctly identify all four illuminated balls with 100% accuracy [40,41].

2.6. Maximum Oxygen Uptake (VO2max) and Running Speed

In the present study, the Bruce protocol [42] was performed on a treadmill (model: h/p/cosmos pulsar® 3p, Nussdorf-Traunstein, Germany) to determine maximum oxygen uptake (VO2max). The Bruce protocol starts with an initial speed of 2.7 km·h−1 and an incline of 10%. Every 3 min, the incline increases by 2% per level, and the speed increases by 1.3 km·h−1 per level. This testing protocol was terminated when a participant could no longer run due to fatigue. Heart rate (HR) was measured during the protocol using a Polar H1 Heart Rate Sensor (Polar Electro Europe AG, Steinhausen, Switzerland). Maximum HR (HRmax) was determined at the end of the protocol. Thus, an individualized training protocol was applied for all participants in the present study.

2.7. Data Analysis

Statistical analyses were implemented on SPSS v.27 (SPSS, Chicago, IL, USA). We first explored dependent variables to examine missing data points, normality of distributions (tested by Kolmogorov–Smirnov tests), homogeneity of variance using Levene′s tests, and outliers′ presence. An alpha level of 0.05 was used for all statistical tests. Characteristics of the study population: potential group differences for continuous variables (i.e., age, height, weight, BMI, physical activity) were assessed using ANOVAs, and categorical demographic variables (i.e., sex) were compared by chi-square test. For ANOVAs, the partial eta2 (ηp2) was calculated as an effect strength measure. Repeated measures sphericity issues were addressed with the Greenhouse Geisser correction. When ANCOVAs were statistically significant, post hoc comparisons were performed using the Bonferroni correction. The classification of the ηp2 follows the conventions of Cohen [43]: 0.01 small effect; 0.06 medium effect; 0.14 strong effect.

Physical and mental energy/fatigue: Concerning the MPSTEFS, a 2 (pre-/post-exercise) × 4 (group) repeated measures ANCOVA controlled for sex and HRmax was used for all variables by calculating the rate of change between pre-and post-exercise (physical and mental energy/fatigue). Maximum oxygen uptake (VO2max): In relation to post-intervention changes in VO2max, a 2 (time: basic assessments and post-test) × 4 (group) repeated measures ANCOVA, controlled for sex and VO2max of basic assessments was applied.

Effects of HIIT on the NT performance in pre-test: A 3 (time: NT score during exercise at 5, 15, and 25 min) × 4 (group) repeated measures ANCOVA, controlled for sex VO2max of basic assessments was performed. NT performance during the intervention period: A 24 (time: all NT score of eight training sessions) × 2 (group) repeated measures ANCOVA, controlled for sex and the percentage change of VO2max was applied to compare the differences of NT performance in the HIIT + NT and NT group during the intervention period. A coefficient of determination (R2) of HIIT + NT and NT group was calculated by linear regression between the NT performance and the intervention within four weeks. Long-term effects on NT performance during HIIT after Intervention: A 3 (time: NT score during exercise at 5, 15, and 25 min) × 2 (pre-/post-exercise) × 4 (group) repeated measures ANCOVA, controlled for sex and the percentage change of VO2max was used. A 3 (time: the percentage change of NT at 5, 15, and 25 min) × 4 (group) ANCOVA with repeated measures controlled for sex and the percentage change of VO2max was performed. For the repeated measures ANCOVA used above, the within-subjects factor time and pre-/post-exercise were used, and the between-subjects factor was used with the group.

3. Results

3.1. Characteristics of the Study Population

Table 1 summarizes the mean age, height, weight, BMI, and physical activity duration by group. At baseline, there were no significant differences between the four groups regarding demographic information. Homogeneity by Levene′s tests was observed equally for the data of participant′s characteristics.

Table 1.

Participant characteristics.

HIIT + NT
(n = 11)
M (SD)
NT
(n = 10)
M (SD)
HIIT
(n = 11)
M (SD)
Control
(n = 10)
M (SD)
p-Value
Sex (male:female) 8:3 6:4 8:3 7:3 0.913
Age (years) 22.9 (2.81) 22.7 (2.79) 24.9 (3.86) 22.6 (1.43) 0.218
Height (cm) 180 (0.70) 175 (0.12) 176 (0.74) 177 (0.10) 0.600
Weight (kg) 73.6 (8.23) 71.7 (14.0) 67.8 (8.27) 70.2 (13.1) 0.662
BMI (kg·m−2) 22.7 (1.49) 23.3 (2.16) 21.7 (1.22) 22.10 (2.58) 0.290
Physical activity (min/week) 244 (111) 258 (94.0) 205 (42.0) 171 (69.8) 0.095

M = mean, SD = standard deviation.

3.2. Change Rate of MPSTEFS in Pre-and Post-Test

For the rate of change of all variables (physical and mental energy/fatigue), a 2 (pre-/post-exercise) × 4 (group) repeated measures ANCOVA, controlled for sex and HRmax, showed no significant effect between pre-and post-test. Levene′s tests also indicated equal variances for each variable.

3.3. Maximum Oxygen Uptake (VO2max)

A 2 (time: basic assessments and after post-test) × 4 (group) repeated measures ANCOVA, controlled for sex and VO2max of basic assessments revealed a significant interaction between time and group, F(3,36) = 6.04, p = 0.002, ηp2 = 0.335, indicating increases in VO2max only for the HIIT and the HIIT + NT groups (Table 2). Post hoc analysis demonstrated that the HIIT group significantly differed from NT (p = 0.02) and the control group (p = 0.07).

Table 2.

VO2max at before pre-test and after post-test stratified by group.

HIIT + NT
(n = 11)
M (SD)
NT
(n = 10)
M (SD)
HIIT
(n = 11)
M (SD)
Control
(n = 10)
M (SD)
p-Value
HRmax (bpm)
 Basic assessments 190 (7.65) 191 (4.84) 189 (8.15) 192 (4.25) 0.833
 After post-test 190 (7.31) 190 (4.97) 188 (7.39) 192 (5.51) 0.783
VO2max (mL·kg−1·min−1)
 Basic assessments 53.6 (6.09) 52.2 (4.91) 51.6 (6.76) 53.8 (5.07) 0.644
 After post-test 55.7 (5.91) 52.1 (6.06) 54.8 (6.42) 53.0 (4.25) 0.557

M = mean, SD = standard deviation.

3.4. NT Performance during HIIT in Pre-Test

It can be seen that during the HIIT protocol, NT scores increase with increasing time (at 5, 15, and 25 min) in all participants (Figure 4). A 3 (time: during exercise at 5, 15 and 25 min) × 4 (groups) repeated measures ANCOVA, controlled for sex and VO2max of baseline assessments revealed a significant effect of time, F(2,74) = 3.33, p = 0.041, ηp2 = 0.082. No significant interaction was found between time and groups. Post hoc analysis found a significant difference between 5 min and 25 min for the NT Score for all participants (p = 0.01).

Figure 4.

Figure 4

Neurotracker scores over time stratified by group (T1–T8: Training units).

3.5. Long-Term Effects on NT Performance during HIIT after Intervention

A 3 (time; during exercise at 5, 15 and 25 min) × 2 (pre-/post-exercise) × 4 (group) repeated measures ANCOVA, controlled for sex and the percentage change of VO2max revealed for the NT score a significant effect of pre-/post-exercise, F(1,37) = 15.5, p < 0.001, ηp2 = 0.296, and interaction between pre-/post-exercise and group, F(3,37) = 8.37, p < 0.001, ηp2 = 0.404. In addition, there was a significant effect of time, F(2,74) = 4.62, p = 0.013, ηp2 = 0.111. For the percentage change of NT score, a 3 (time: at 5, 15, and 25 min) × 4 (group) repeated measures ANCOVA controlled for sex and the percentage change of VO2max did not demonstrate any significant value, but significant differences were found between groups. At 5 min, there was only a significant difference between NT and HIIT (p = 0.004), and at 15 min, no significant difference was found. Finally, as presented in Figure 5, at 25 min, a significant difference was observed between HIIT and HIIT + NT (p = 0.047), and the NT group was significantly different from HIIT (p = 0.002) and control group (p = 0.048).

Figure 5.

Figure 5

Percentage change of NT scores between pre-and post-test stratified by group. * p < 0.05; ** p < 0.01.

4. Discussion

The present study aimed to examine the effect of HIIT in combination with and without a 3D-MOT (NT) training on the development of perceptual–cognitive performance after a four-week intervention period. We hypothesized that the combination of HIIT and NT-training would enhance the long-term effects on perceptual–cognitive performance during physical exercise. It is well known that exercise has positive effects on cognition [44]. However, few studies investigate the effects of combined physical and cognitive training on perceptual–cognitive performance during exercise. Our study examined the effects of interventions on perceptual–cognitive performance during the HIIT protocol, focusing on long-term effects during physical exercise in young adults.

In the present study, HIIT combined with NT training improved perceptual–cognitive performance. However, a significant improvement of perceptual–cognitive performance during the HIIT protocol was not found. As expected, an improvement was observed during the NT intervention, showing that NT training itself has a high training effect irrespective of the intervention condition.

Amateur athletes′ cognitive abilities reached a certain level with increasing intensity and decreased at the high-intensity level [3]. In contrast, professional athletes′ performance increased despite the high-intensity, which seems to contradict the inverted U-theory [45]. This shows a dependency on the fitness level of the participants [22]. Notably, there were no differences between the groups in terms of NT performance in the pre-test in the present study. This may be attributed to the fact that all participants were having similar VO2max levels and high fitness levels according to the VO2max normative data (Heywood [46] ((Table 2). Additionally, no significant changes were found at the subjective level for physical and mental fatigue. For this reason, the HIIT protocol used in the present study may not induce fatigue and may have resulted in optimal arousal levels.

Our study examined perceptual–cognitive performance using the NT task during a HIIT protocol for the first time. It showed a high increase in NT performance for the intervention period under two conditions while standing and exercising/running on the treadmill. We observed a significant time effect on perceptual–cognitive performance improvement during the intervention period (p < 0.001), but no group differences were found, indicating a high training effect of NT on performance in both groups (HIIT + NT: R2 = 0.73; NT: R2 = 0.80). Similarly, Faubert and Sidebottom [7] examined perceptual–cognitive performance measured with the NT task under different conditions (sitting vs. standing) with 30 NT sessions. Their results demonstrated that performance during standing was not so dramatically increased compared to performance during sitting, which can be attributed to the relationship between postural control mechanisms and perceptual–cognitive demands.

4.1. The Importance of Combining Physical Exercise with Perceptual–Cognitive Training

Many studies suggested that either physical exercise [44] or cognitive training [47,48] has positive effects on the brain′s functional or structural changes. Physical exercise can also strongly stimulate brain function [44,49,50], and its positive effects can potentially be enhanced with combined cognitive training [51]. Combining physical exercise with cognitive tasks may be particularly effective in producing positive, supportive influences on brain structure and function [52]. Nevertheless, the effect of combining two types of intervention is largely unexplored. Few studies have examined the combined effects of exercise and cognitive training, and these have been limited to older adults [31,53,54] or patients with neurological impairment [55]. One study found that specific cognitive training effects improving cognitive function for spatial learning were more pronounced than the effects of physical training [56]. These findings provide evidence that physical exercise alone may not be sufficient to produce improvement in specific cognitive abilities.

For this reason, it could be suggested that specific cognitive training targets relevant functional networks more precisely than a nonspecific physical exercise intervention [32]. In this regard, the significant improvement of NT scores during HIIT protocol was observed only in the HIIT + NT and NT groups in the post-test (interaction between pre-and post-test groups, p < 0.001). No improvement in performance was observed in the HIIT and control groups. However, it is somewhat surprising that four weeks of HIIT did not improve perceptual–cognitive performance in the HIIT group. A possible reason for the absence of an effect for HIIT may be due to the short intervention period. Griffin et al. [57] demonstrated that the improvement in cognitive functions concerning memory performance was observed after five weeks of 60 min training at an intensity of 60% of VO2max on a cycle ergometer, but not after three weeks. For this reason, it should be investigated whether extending the intervention duration can improve perceptual–cognitive performance in the HIIT group.

Moreover, no studies examine cognitive abilities during physical exercise after interventions, while many studies have investigated cognitive performance at rest after an intervention [14]. However, specific sport-related cognitive training may be required to improve performance during exercise. Interestingly, a similar enhancement in perceptual–cognitive abilities was observed with cognitive training alone compared to the combination of HIIT with NT training. The NT training′s continuous cognitive stimulation during the intervention period could improve the more specific perceptual–cognitive function required for the 3D-MOT task better than the HIIT intervention. Physical exercise could contribute to an overall facilitating effect on learning, such as increased angiogenesis and the availability of specific neurotrophins, which are prerequisites for neuroplasticity [32,51]. Physical exercise can prepare the brain to respond well to cognitive stimulation [58], and cognitive training induces neuronal changes in a specific brain network related to the trained skills [32]. Suppose the combination of physical exercise and cognitive training is continued. In that case, the improvement in task-related cognitive performance could be more significant than if only the cognitive task were trained. Of course, this is just an assumption as we did not measure neural correlates. However, there is a relationship between exercise and cognition on behavioral as well as neural levels [32].

Furthermore, as a mediator that alters brain structure and function, physical exercise could eventually lead to cognitive performance improvements at the behavioral level [58]. However, in the present study, we only examined behavioral data and not neural correlates. Therefore, studies on the NT task′s neural correlates during physical exercise in participants with different fitness levels are needed. It is also necessary to identify the differences between the groups after the intervention period using different cognitive tasks and the NT task (transfer task) and extending the intervention period.

We would like to mention some limitations. First, the relatively small number of participants should be noted. No a-priori power analysis was performed. The primary outcome measure (the Neurotracker Score) was used as part of the two experimental groups′ interventions. The learning effect of the task used for training and assessment makes it difficult to determine whether the improvements were due to training or learning. It is also unclear how this would translate to on-field performance as no transfer test was conducted. Given the short duration of the intervention and the high fitness level at baseline, it is unclear whether the intervention would have affected cognitive performance as a function of improved fitness. HIIT could have a very different physiological effect than high-intensity steady-state exercise (although high intensity), so this must be considered when interpreting study results on high-intensity exercise. For example, in a 30 s HIIT protocol such as this, cerebral blood flow would likely increase in the first 3 s–4 s before decreasing to a level nearly equivalent to rest over the following 25 s. We would expect blood flow velocity to increase and peak during recovery, but we would also expect blood flow to generally decreasing throughout the training session. In short, HIIT affects the brain very uniquely, and this needs to be considered in future studies.

4.2. Implications for Future Research and Practice

The combination of physical and cognitive training (as in the HIIT + NT group) can be proposed as an athletic training program to improve overall cognitive and physical performance in a time-efficient way. In particular, a concept of intermittent training or high-intensity interval training for certain team sports and the study of specific perceptual–cognitive performance in such situations would be valuable for all athletes who can train both physical and cognitive skills during real sports.

5. Conclusions

In conclusion, perceptual–cognitive training may play a crucial role in improving an athlete′s ability by using sports-specific visual information to facilitate sports-specific skills such as prediction and decision making [4]. The study aimed to investigate how perceptual–cognitive performance was affected during high-intensity interval training (HIIT) using a 3D-multiple object tracking task. Our findings suggest that training resulted in substantial, task-specific gains (physical fitness, perceptual–cognitive performance), but there was no effect of HIIT on perceptual–cognitive performance. Due to the specific 3D-MOT task characteristics, improved physical abilities may not directly affect perceptual–cognitive performance related to sport-specific tasks. The absence of an additional training effect of the HIIT or HIIT + NT compared to the NT group is only somewhat surprising and maybe (a) due to the nature of the HIIT protocol, (b) the short intervention period, and/or (c) the difference between this specific 3D-MOT task and the usually tested cognitive abilities.

Acknowledgments

We are grateful to the participating subjects from the University Sports Center of the University of Stuttgart, Germany. Special thanks are given to Sinje Rüsch at the University Sports Center of the University of Stuttgart, who offered great help in recruiting the subjects.

Author Contributions

Conceptualization: S.-Y.P., N.S.; data curation: S.-Y.P., T.J.K.; formal analysis: S.-Y.P., N.S.; investigation: S.-Y.P., T.J.K., N.S.; methodology: S.-Y.P., T.J.K., N.S.; resources: N.S.; supervision: T.J.K., N.S.; visualization: S.-Y.P., N.S.; writing—original draft preparation: S.-Y.P.; writing—review and editing: T.J.K., N.S. All authors have read and agreed to the published version of the manuscript.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Institutional Review Board Statement

The study was conducted according to the guidelines of the Declaration of Helsinki and approved by the Institutional Review Board (or Ethics Committee) of the University of Stuttgart (07/2017).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

All relevant data are within the study.

Conflicts of Interest

The authors have declared that no potential conflicts of interest exist.

Footnotes

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

References

  • 1.Bullock T., Giesbrecht B. Acute Exercise and Aerobic Fitness Influence Selective Attention during Visual Search. Front. Psychol. 2014;5:1–11. doi: 10.3389/fpsyg.2014.01290. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Schapschröer M., Lemez S., Baker J., Schorer J. Physical Load Affects Perceptual-Cognitive Performance of Skilled Athletes: A Systematic Review. Sport. Med. Open. 2016;2:37. doi: 10.1186/s40798-016-0061-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Hüttermann S., Memmert D. Does the Inverted-U Function Disappear in Expert Athletes? An Analysis of the Attentional Behavior under Physical Exercise of Athletes and Non-Athletes. Physiol. Behav. 2014;131:87–92. doi: 10.1016/j.physbeh.2014.04.020. [DOI] [PubMed] [Google Scholar]
  • 4.Hadlow S.M., Panchuk D., Mann D.L., Portus M.R., Abernethy B. Modified Perceptual Training in Sport: A New Classification Framework. J. Sci. Med. Sport. 2018;21:950–958. doi: 10.1016/j.jsams.2018.01.011. [DOI] [PubMed] [Google Scholar]
  • 5.Bherer L., Peterson M.S., Kramer A.F., Colcombe S., Erickson K., Becic E. Training Effects on Dual-Task Performance: Are There Age-Related Differences in Plasticity of Attentional Control? Psychol. Aging. 2005;20:695–709. doi: 10.1037/0882-7974.20.4.695. [DOI] [PubMed] [Google Scholar]
  • 6.Voss M.W., Kramer A.F., Basak C., Prakash R.S., Roberts B. Are Expert Athletes “expert” in the Cognitive Laboratory? A Meta-Analytic Review of Cognition and Sport Expertise. Appl. Cogn. Psychol. 2010;24:812–826. doi: 10.1002/acp.1588. [DOI] [Google Scholar]
  • 7.Faubert J., Sidebottom L. Perceptual-Cognitive Training of Athletes. J. Clin. Sport Psychol. 2012;6:85–102. doi: 10.1123/jcsp.6.1.85. [DOI] [Google Scholar]
  • 8.Faubert J. Professional Athletes Have Extraordinary Skills for Rapidly Learning Complex and Neutral Dynamic Visual Scenes. Sci. Rep. 2013;3:1154. doi: 10.1038/srep01154. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Romeas T., Guldner A., Faubert J. 3D-Multiple Object Tracking Training Task Improves Passing Decision-Making Accuracy in Soccer Players. Psychol. Sport Exerc. 2016;22:1–9. doi: 10.1016/j.psychsport.2015.06.002. [DOI] [Google Scholar]
  • 10.Fougnie D., Marois R. Distinct Capacity Limits for Attention and Working Memory: Evidence from Attentive Tracking and Visual Working Memory Paradigms. Psychol. Sci. 2006;17:526–534. doi: 10.1111/j.1467-9280.2006.01739.x. [DOI] [PubMed] [Google Scholar]
  • 11.Legault I., Allard R., Faubert J. Healthy Older Observers Show Equivalent Perceptual-Cognitive Training Benefits to Young Adults for Multiple Object Tracking. Front. Psychol. 2013;4:323. doi: 10.3389/fpsyg.2013.00323. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Parsons B., Magill T., Boucher A., Zhang M., Zogbo K., Berube S., Scheffer O., Beauregard M., Faubert J. Enhancing Cognitive Function Using Perceptual-Cognitive Training. Clin. Eeg Neurosci. 2014;47:1–11. doi: 10.1177/1550059414563746. [DOI] [PubMed] [Google Scholar]
  • 13.Quevedo L., Padrós A., Solé J., Cardona G. Entrenament Perceptivocognitiu Amb El Neurotracker 3D-MOT per Potenciar El Rendimient En Tres Modalitats Esportives. Apunt. Educ. Física Y Deport. 2015;119:97–108. doi: 10.5672/apunts.2014-0983.cat.(2015/1).119.07. [DOI] [Google Scholar]
  • 14.Fernandes M., de Sousa A., Medeiros A.R., Del Rosso S., Stults-Kolehmainen M., Boullosa D.A. The Influence of Exercise and Physical Fitness Status on Attention: A Systematic Review. Int. Rev. Sport Exerc. Psychol. 2019;12:202–234. doi: 10.1080/1750984X.2018.1455889. [DOI] [Google Scholar]
  • 15.Alesi M., Bianco A., Luppina G., Palma A., Pepi A. Improving Children’s Coordinative Skills and Executive Functions. Percept. Mot. Ski. 2016;122:27–46. doi: 10.1177/0031512515627527. [DOI] [PubMed] [Google Scholar]
  • 16.Gutmann B., Mierau A., Hülsdünker T., Hildebrand C., Przyklenk A., Hollmann W., Strüder H.K. Effects of Physical Exercise on Individual Resting State EEG Alpha Peak Frequency. Neural Plast. 2015;2015:1–6. doi: 10.1155/2015/717312. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Hillman C.H., Pontifex M.B., Castelli D.M., Khan N.A., Raine L.B., Scudder M.R., Drollette E.S., Moore R.D., Wu C.T., Kamijo K. Effects of the FITKids Randomized Controlled Trial on Executive Control and Brain Function. Pediatrics. 2014;134:e1063–e1071. doi: 10.1542/peds.2013-3219. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Schott N., Krull K. Stability of Lifestyle Behavior—The Answer to Successful Cognitive Aging? A Comparison of Nuns, Monks, Master Athletes and Non-Active Older Adults. Front. Psychol. 2019;10:1–14. doi: 10.3389/fpsyg.2019.01347. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Antunes H.K.M., De Mello M.T., De Aquino Lemos V., Santos-Galduróz R.F., Galdieri L.C., Bueno O.F.A., Tufik S., D’Almeida V. Aerobic Physical Exercise Improved the Cognitive Function of Elderly Males but Did Not Modify Their Blood Homocysteine Levels. Dement. Geriatr. Cogn. Dis. Extra. 2015;5:13–24. doi: 10.1159/000369160. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Iuliano E., di Cagno A., Aquino G., Fiorilli G., Mignogna P., Calcagno G., Di Costanzo A. Effects of Different Types of Physical Activity on the Cognitive Functions and Attention in Older People: A Randomized Controlled Study. Exp. Gerontol. 2015;70:105–110. doi: 10.1016/j.exger.2015.07.008. [DOI] [PubMed] [Google Scholar]
  • 21.Del Giorno J.M., Hall E.E., O’Leary K.C., Bixby W.R., Miller P.C. Cognitive Function during Acute Exercise: A Test of the Transient Hypofrontality Theory. J. Sport Exerc. Psychol. 2010;32:312–323. doi: 10.1123/jsep.32.3.312. [DOI] [PubMed] [Google Scholar]
  • 22.Chang Y.K., Labban J.D., Gapin J.I., Etnier J.L. The Effects of Acute Exercise on Cognitive Performance: A Meta-Analysis. Brain Res. 2012;1453:87–101. doi: 10.1016/j.brainres.2012.02.068. [DOI] [PubMed] [Google Scholar]
  • 23.Lambourne K., Tomporowski P. The Effect of Exercise-Induced Arousal on Cognitive Task Performance: A Meta-Regression Analysis. Brain Res. 2010;1341:12–24. doi: 10.1016/j.brainres.2010.03.091. [DOI] [PubMed] [Google Scholar]
  • 24.Brisswalter J., Collardeau M., René A. Effects of Acute Physical Exercise Characteristics on Cognitive Performance. Sports Med. 2002;32:555–566. doi: 10.2165/00007256-200232090-00002. [DOI] [PubMed] [Google Scholar]
  • 25.Tomporowski P.D. Effects of Acute Bouts of Exercise on Cognition. Acta Psychol. 2003;112:297–324. doi: 10.1016/S0001-6918(02)00134-8. [DOI] [PubMed] [Google Scholar]
  • 26.McMorris T., Sproule J., Turner A., Hale B.J. Acute, Intermediate Intensity Exercise, and Speed and Accuracy in Working Memory Tasks: A Meta-Analytical Comparison of Effects. Physiol. Behav. 2011;102:421–428. doi: 10.1016/j.physbeh.2010.12.007. [DOI] [PubMed] [Google Scholar]
  • 27.Casanova F., Garganta J., Silva G., Alves A., Oliveira J., Williams A.M. Effects of Prolonged Intermittent Exercise on Perceptual-Cognitive Processes. Med. Sci. Sports Exerc. 2013;45:1610–1617. doi: 10.1249/MSS.0b013e31828b2ce9. [DOI] [PubMed] [Google Scholar]
  • 28.Tsukamoto H., Suga T., Takenaka S., Tanaka D., Takeuchi T., Hamaoka T., Isaka T., Hashimoto T. Greater Impact of Acute High-Intensity Interval Exercise on Post-Exercise Executive Function Compared to Moderate-Intensity Continuous Exercise. Physiol. Behav. 2016;155:224–230. doi: 10.1016/j.physbeh.2015.12.021. [DOI] [PubMed] [Google Scholar]
  • 29.Tsukamoto H., Takenaka S., Suga T., Tanaka D., Takeuchi T., Hamaoka T., Isaka T., Hashimoto T. Effect of Exercise Intensity and Duration on Postexercise Executive Function. Med. Sci. Sports Exerc. 2017;49:774–784. doi: 10.1249/MSS.0000000000001155. [DOI] [PubMed] [Google Scholar]
  • 30.Cooper S.B., Bandelow S., Nute M.L., Dring K.J., Stannard R.L., Morris J.G., Nevill M.E. Sprint-Based Exercise and Cognitive Function in Adolescents. Prev. Med. Rep. 2016;4:155–161. doi: 10.1016/j.pmedr.2016.06.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Lauenroth A., Ioannidis A.E., Teichmann B. Influence of Combined Physical and Cognitive Training on Cognition: A Systematic Review. BMC Geriatr. 2016;16:1–14. doi: 10.1186/s12877-016-0315-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Hötting K., Röder B. Beneficial Effects of Physical Exercise on Neuroplasticity and Cognition. Neurosci. Biobehav. Rev. 2013;37:2243–2257. doi: 10.1016/j.neubiorev.2013.04.005. [DOI] [PubMed] [Google Scholar]
  • 33.Krenn B., Finkenzeller T., Würth S., Amesberger G. Sport Type Determines Differences in Executive Functions in Elite Athletes. Psychol. Sport Exerc. 2018;38:72–79. doi: 10.1016/j.psychsport.2018.06.002. [DOI] [Google Scholar]
  • 34.Holfelder B., Klotzbier T.J., Eisele M., Schott N. Hot and Cool Executive Function in Elite- and Amateur- Adolescent Athletes from Open and Closed Skills Sports. Front. Psychol. 2020;11:694. doi: 10.3389/fpsyg.2020.00694. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Mélard G. On the Accuracy of Statistical Procedures in Microsoft Excel 2010. Comp. Stat. 2014;29:1095–1128. doi: 10.1007/s00180-014-0482-5. [DOI] [Google Scholar]
  • 36.O’Connor P.J. Mental Energy: Assessing the Mood Dimension. Nutr. Rev. Jul. 2006;64:S7–S9. doi: 10.1111/j.1753-4887.2006.tb00256.x. [DOI] [PubMed] [Google Scholar]
  • 37.Fabri T.L., Wilson K.E., Holland N., Hickling A., Murphy J., Fait P., Reed N. Using a Dual-Task Protocol to Investigate Motor and Cognitive Performance in Healthy Children and Youth. Gait Posture. 2017;54:154–159. doi: 10.1016/j.gaitpost.2017.03.002. [DOI] [PubMed] [Google Scholar]
  • 38.Legault I., Faubert J. Perceptual-Cognitive Training Improves Biological Motion Perception: Evidence for Transferability of Training in Healthy Aging. Neuroreport. 2012;23:469–473. doi: 10.1097/WNR.0b013e328353e48a. [DOI] [PubMed] [Google Scholar]
  • 39.Levitt H. Transformed Up-down Methods in Psychoacoustics. J. Acoust. Soc. Am. 1971;49:467–477. doi: 10.1121/1.1912375. [DOI] [PubMed] [Google Scholar]
  • 40.Mangine G.T., Hoffman J.R., Wells A.J., Gonzalez A.M., Rogowski J.P., Townsend J.R., Jajtner A.R., Beyer K.S., Bohner J.D., Pruna G.J., et al. Visual Tracking Speed Is Related To Basketball Basketball-Specific Measures of Performance in Nba Players. J. Strength Cond. Res. 2014;28:2406–2414. doi: 10.1519/JSC.0000000000000550. [DOI] [PubMed] [Google Scholar]
  • 41.Tullo D., Guy J., Faubert J., Bertone A. Training with a Three-Dimensional Multiple Object-Tracking (3D-MOT) Paradigm Improves Attention in Students with a Neurodevelopmental Condition: A Randomized Controlled Trial. Dev. Sci. 2018;21:1–11. doi: 10.1111/desc.12670. [DOI] [PubMed] [Google Scholar]
  • 42.Bruce R.A. Exercise Testing and Training of Apparently Healthy Individuals: A Handbook for Physicians. American Heart Association; Dallas, TX, USA: 1972. Multi-stage treadmill test of submaximal and maximal exercise; pp. 32–34. [Google Scholar]
  • 43.Cohen J. Statistical Power Analysis for the Behavioral Sciences. Routledge; London, UK: 1988. [Google Scholar]
  • 44.Hillman C.H., Erickson K.I., Kramer A.F. Be Smart, Exercise Your Heart: Exercise Effects on Brain and Cognition. Nat Rev Neurosci. 2008;9:58–65. doi: 10.1038/nrn2298. [DOI] [PubMed] [Google Scholar]
  • 45.Yerkes R.M., Dodson J.D. The Relation of Strength of Stimulus to Rabidity of Habit-Formation. J. Comp. Neurol. Psychol. 1908;18:459–489. doi: 10.1002/cne.920180503. [DOI] [Google Scholar]
  • 46.Heywood V. The Physical Fitness Specialist Manual, The Cooper Institute for Aerobics Research, Dallas, TX, revised 2005. In: Heywood V.C., editor. Advanced Fitness Assessment and Exercise Prescription. Human Kinetics; Champaign, IL, USA: 2006. pp. 52–56. [Google Scholar]
  • 47.Lövdén M., Schaefer S., Noack H., Bodammer N.C., Kühn S., Heinze H.J., Düzel E., Bäckman L., Lindenberger U. Spatial Navigation Training Protects the Hippocampus against Age-Related Changes during Early and Late Adulthood. Neurobiol. Aging. 2012;33:620.e9–620.e22. doi: 10.1016/j.neurobiolaging.2011.02.013. [DOI] [PubMed] [Google Scholar]
  • 48.Stine-Morrow E.A.L., Basak C. Handbook of the Psychology of Aging. Academic Press; Cambridge, MA, USA: 2011. Cognitive Interventions; pp. 153–171. [Google Scholar]
  • 49.Heisz J.J., Gould M., McIntosh A.R. Age-Related Shift in Neural Complexity Related to Task Performance and Physical Activity. J. Cogn. Neurosci. 2015;27:605–613. doi: 10.1162/jocn_a_00725. [DOI] [PubMed] [Google Scholar]
  • 50.Heisz J.J., Vandermorris S., Wu J., McIntosh A.R., Ryan J.D. Age Differences in the Association of Physical Activity, Sociocognitive Engagement, and TV Viewing on Face Memory. Heal. Psychol. 2015;34:83–88. doi: 10.1037/hea0000046. [DOI] [PubMed] [Google Scholar]
  • 51.Heisz J.J., Clark I.B., Bonin K., Paolucci E.M., Michalski B., Becker S., Fahnestock M. The Effects of Physical Exercise and Cognitive Training on Memory and Neurotrophic Factors. J. Cogn. Neurosci. 2017;29:1895–1907. doi: 10.1162/jocn_a_01164. [DOI] [PubMed] [Google Scholar]
  • 52.Kempermann G. Why New Neurons? Possible Functions for Adult Hippocampal. J. Neurosci. 2002;22:635–638. doi: 10.1523/JNEUROSCI.22-03-00635.2002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 53.Desjardins-Crepeau L., Berryman N., Fraser S.A., Vu T.T.M., Kergoat M.-J., Li K.Z., Bosquet L., Bherer L. Effects of Combined Physical and Cognitive Training on Fitness and Neuropsychological Outcomes in Healthy Older Adults. Clin. Interv. Aging. 2016;11:1287–1299. doi: 10.2147/CIA.S115711. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 54.Law L.L.F., Barnett F., Yau M.K., Gray M.A. Effects of Combined Cognitive and Exercise Interventions on Cognition in Older Adults with and without Cognitive Impairment: A Systematic Review. Ageing Res. Rev. 2014;15:61–75. doi: 10.1016/j.arr.2014.02.008. [DOI] [PubMed] [Google Scholar]
  • 55.Evans J.J., Greenfield E., Wilson B.A., Bateman A. Walking and Talking Therapy: Improving Cognitive-Motor Dual-Tasking in Neurological Illness. J. Int. Neuropsychol. Soc. 2009;15:112–120. doi: 10.1017/S1355617708090152. [DOI] [PubMed] [Google Scholar]
  • 56.Holzschneider K., Wolbers T., Röder B., Hötting K. Cardiovascular Fitness Modulates Brain Activation Associated with Spatial Learning. Neuroimage. 2012;59:3003–3014. doi: 10.1016/j.neuroimage.2011.10.021. [DOI] [PubMed] [Google Scholar]
  • 57.Griffin É.W., Mullally S., Foley C., Warmington S.A., O’Mara S.M., Kelly Á.M. Aerobic Exercise Improves Hippocampal Function and Increases BDNF in the Serum of Young Adult Males. Physiol. Behav. 2011;104:934–941. doi: 10.1016/j.physbeh.2011.06.005. [DOI] [PubMed] [Google Scholar]
  • 58.Stillman C.M., Cohen J., Lehman M.E., Erickson K.I. Mediators of Physical Activity on Neurocognitive Function: A Review at Multiple Levels of Analysis. Front. Hum. Neurosci. 2016;10:1–17. doi: 10.3389/fnhum.2016.00626. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Data Availability Statement

All relevant data are within the study.


Articles from International Journal of Environmental Research and Public Health are provided here courtesy of Multidisciplinary Digital Publishing Institute (MDPI)

RESOURCES