The relationship between brain injury criteria and brain strain across different types of head impacts can be different

Abstract

Multiple brain injury criteria (BIC) are developed to quickly quantify brain injury risks after head impacts. These BIC originated from different head impact types (e.g. sports and car crashes) are widely used in risk evaluation. However, the accuracy of using the BIC on brain injury risk estimation across head impact types has not been evaluated. Physiologically, brain strain is often considered the key parameter of brain injury. To evaluate the BIC's risk estimation accuracy across five datasets comprising different head impact types, linear regression was used to model 95% maximum principal strain, 95% maximum principal strain at the corpus callosum and cumulative strain damage (15%) on 18 BIC. The results show significantly different relationships between BIC and brain strain across datasets, indicating the same BIC value may suggest different brain strain across head impact types. The accuracy of brain strain regression is generally decreasing if the BIC regression models are fitted on a dataset with a different type of head impact rather than on the dataset with the same type. Given this finding, this study raises concerns for applying BIC to estimate the brain injury risks for head impacts different from the head impacts on which the BIC was developed.

1. Introduction

Traumatic brain injury (TBI), frequently caused by head impacts in sports, traffic accidents and unintentional falls, has become a global health challenge and affects over 50 million children and adults worldwide [1]. Additionally, mild traumatic brain injury (mTBI) is associated with long-term cognitive and emotional sequelae [2], cerebral blood flow alterations [3] and even neurological degenerative diseases [4]. If mTBI goes undetected, the accumulation of brain damage causes higher risks of long-term consequences [5], which calls for better monitoring of brain injury after head impacts.

To quickly estimate the brain injury risk of a head impact, multiple brain injury criteria (BIC) have been developed with reduced-order physical models [6–9] and statistical model fitting [10,11]. These BIC estimate the risk of brain injury based on the measured kinematics of head movement caused by an impact. Additionally, the peak values of head movement kinematics, such as the linear acceleration at the brain centre of gravity, angular velocity and angular acceleration, can also be used as a BIC [8,12,13].

Because the brain can be injured when head movement deforms brain tissue by the inertial force [6–8,14–16], brain strain, particularly the maximum principal strain (MPS), is a key parameter in evaluating the brain injury risk [3,17–19] and has been widely used as an indicator for mTBI. To evaluate the accuracy of estimating brain injury risks of the BIC, typically, the Pearson correlation between the BIC and the brain strain (particularly 95% MPS) and the coefficient of determination (R²) of the linear regression of brain strain on the BIC have been used by researchers [6–8,20].

Although the BIC are typically designed for specific types of head impacts, many BIC are used regardless of their development background. For instance, head injury criterion (HIC) originated in the dummy test of automobile crashworthiness and in the field of motor vehicle regulation [21] but is now used in mTBI research [22]. However, when cross-used to different types of head impacts, even for the same values of MPS, some BIC give different values. As is shown in figure 1, the impacts of college football and mixed martial arts show different characteristics in their kinematics, and the BIC calculated based on the kinematics (§2.2) show clear distinctions. However, these two impacts reach the same peak value of 95% MPS. That different BIC values indicate potentially the same brain strain in different types of head impacts poses a risk to the general use of these BIC across the mTBI field.

Figure 1.

Comparison between two impacts in college football (a) and in mixed martial arts (b). The traces of 95% MPS (left axis) and angular acceleration (Ang. Acc., right axis) magnitudes are plotted in (a,b), and these two impacts have the same peak value of 95% MPS as shown by the dashed line across (a) and (b). The 95% MPS peak value and BIC values are given in (c). The blue and red bars indicate college football and mixed martial arts, respectively. The information of BIC is introduced in §2.2.

To better understand the risk of using a BIC as a general metric, in this work, several datasets from various sports and various types of head impacts are used to investigate the cross-usage of BIC. Linear regression models were built with 18 different BIC as predictors for 95% MPS (MPS95), 95% MPS at corpus callosum (MPSCC95) and cumulative strain damage (CSDM, 15%, indicating the volume fraction of brain with MPS exceeding the threshold of 0.15 [23]), to investigate the regression coefficients of the BIC and the brain strain estimation accuracy across datasets. These models were tested against the different datasets in a variety of statistical strategies to best determine the efficacy of using the BIC across datasets.

2. Methods

2.1. Data description and processing

To evaluate the generalizability of BIC across different types of head impacts, data from six individual sources were used to construct five datasets: (i) dataset 1 contained 2183 laboratory impacts, among which 2130 were simulated by a validated finite-element analysis model of hybrid III anthropomorphic test dummy (ATD) headform impacted by the impactor for football testing [20,24,25]. The remaining 53 were reconstructed head impacts in National Football League [26] using hybrid III ATD headform. The data from these two sources were merged because they both represented the responses of hybrid III ATD headform under football-type impacts; (ii) dataset 2 contained 302 on-site college football head impacts collected by the Stanford instrumented mouthguards [27–29]; (iii) dataset 3 contained 457 on-site mixed martial arts (MMA) head impacts collected by the Stanford instrumented mouthguard [3,30]; (iv) dataset 4 contained 48 head impacts in automobile crashworthiness tests from the National Highway Traffic Safety Administration (NHTSA) [31]; (v) dataset 5 contained 272 numerically reconstructed head impacts in National Association for Stock Car Auto Racing (NASCAR) by hybrid III ATD headform.

Since finite-element (FE) modelling is the state-of-the-art biomechanics modelling tool for brain strain in head impacts, the validated KTH model [32] was used to calculate the MPS95, MPSCC95 and CSDM since these are widely used in mTBI research [6,7,23,33,34]. The KTH model includes the brain, skull, scalp meninges, cerebrospinal fluid and 11 pairs of bridging veins. The modelling was performed using the commercially available FE software LS-DYNA (Livermore, CA, USA). The accuracy in estimating the MPS95/MPSCC95/CSDM was then used to evaluate the accuracy of brain injury risk estimation based on each BIC. The distributions and the summary statistics of the MPS95, MPSCC95 and CSDM in the five datasets are shown in figure 2.

Figure 2.

The normalized distribution and the summary statistics of MPS95, MPSCC95 and CSDM on five datasets. (a) The normalized distribution of MPS95 on five datasets. (b) The normalized distribution of MPSCC95 on five datasets. (c) The normalized distribution of CSDM on five datasets. (d) The summary statistics (mean values, standard deviations (STD) and maximum values) of MPS95/MPSCC95/CSDM on five datasets.

2.2. Brain injury criteria

In this study, 18 BIC were investigated which have been used in TBI/mTBI research. These are listed in table 1 (Lin. Acc., linear acceleration; Ang. Acc., angular acceleration; Ang. Vel., angular velocity). These BIC were based on one or a combination of translational and rotational parameters in the head movement kinematics with experimental data fitting or reduced-order mechanical models [6–8].

Table 1.

The 18 BIC analysed in this study.

BIC name	equation	source impact	kinematics included	FE model	reference
SI	2.1	car crash	Lin. Acc.	—	[35]
HIC	2.2	car crash	Lin. Acc.	—	[21]
GAMBIT	2.3	car crash	Lin. Acc., Ang. Acc.	—	[36]
HIP	2.4	simulation	Lin. Acc., Ang. Acc.	—	[37]
PCS	2.5	football	Lin. Acc., Ang. Acc.	—	[11]
BRIC	2.6	football	Ang. Acc., Ang. Vel.	SIMon [38]	[39]
PRHIC	2.7	football	Ang. Acc.	THUMS [40]	[41]
KLC	2.8	football	Lin. Acc., Ang. Acc.	KTH	[42]
RIC	2.9	football	Ang. Acc.	THUMS	[43]
BrIC	2.10	simulation, football, car crash	Ang. Vel.	GHBMC [44]	[45]
CP	2.11	football	Lin. Acc., Ang. Acc.	—	[10]
RVCI	2.12	car crash	Ang. Vel.a	GHBMC	[46]
CIBIC	—	car crash	Ang. Acc.	GHBMC	[9]
damage	—	idealized impact, football	Ang. Acc.	GHBMC	[7]
BAM	2.13	football, naval sled tests	Ang. Acc.	KTH	[8]
lin_acc_CG_max	—	—	Lin. Acc.	—	—
ang_vel_max	—	—	Ang. Vel.	—	—
ang_acc_max	—	—	Ang. Acc.	—	—

^aThe formula for RVCI is based on the angular acceleration, but the integral of angular acceleration is the change of angular velocity.

The peak values of the magnitudes of the linear acceleration at the brain centre of gravity |a(t)| (lin_acc_CG_max) [12], angular velocity |ω(t)| (ang_vel_max) [13] and angular acceleration |α(t)| (ang_acc_max) [13,20] are the most fundamental BIC definitions. They are calculated by taking the maximum of the magnitude of the respective translational or rotational parameters. In addition to these three BIC, 15 other BIC used in this study are described below.

Severity index (SI), also known as Gadd severity index, was developed by Gadd [35] based on curve fitting the Wayne State Tolerance Curve with skull fracture data in head form simulation. It is calculated based on the following equation, and the integral is calculated from when the signal first exceeds 4 g m s⁻² to when it returns to 4 g m s⁻² after the highest peak:

$$\mathrm{SI}=\hspace{0.17em}\int |a(t){|}^{2.5}\hspace{0.17em}\mathrm{d}t.$$ 2.1

HIC was developed by Versace [21]. It is the most widely used injury criteria developed in the assessment of head injury risks in motor vehicle regulation. HIC is calculated based on the following equation, where t₁, t₂ are chosen to maximize the value of HIC. The duration was initially set as 36 ms and under the current standard t₂ − t₁ ≤ 15 ms, and the resultant term is expressed as HIC₁₅:

$$\mathrm{HIC}=\hspace{0.17em}\underset{(t_1,t_2)}{max}\left\{{\left(\frac{{\int }_{t_1}^{t_2}|a(t)|\mathrm{d}t}{t_2-t_1}\right)}^{2.5}(t_2-t_1)\right\}.$$ 2.2

Generalized acceleration model for brain injury threshold (GAMBIT) was developed by Newman [36]. It was designed to combine rotational and translational components of head accelerations, as an analogue in the kinematics to maximum shear stress/strain theory. It is calculated based on the following equation, where the a_c, α_c are the thresholds for the corresponding acceleration and the constants are n = m = s = 2, a_c = 250 g and α_c = 25 000 rad s⁻²:

$$\mathrm{GAMBIT}\hspace{0.17em}=\hspace{0.17em}\underset{t}{\max }\left\{{\left[{\left(\frac{|a(t)|}{a_c}\right)}^n+\hspace{0.17em}{\left(\frac{|\alpha (t)|}{{\alpha }_c}\right)}^m\right]}^{1/s}\right\}\hspace{0.17em}.$$ 2.3

Head impact power (HIP) was developed by Newman et al. [37]. It was based on the hypothesis that the severity of head injury correlated with the HIP. HIP is calculated based on the following equation, where m denotes the mass, I_ii denotes the principal moments of head inertia and i denotes the components of the acceleration in three spatial directions:

$$\mathrm{HIP}=\hspace{0.17em}\underset{t}{max}\left\{m\sum a_i(t)\int a_i(t)\hspace{0.17em}\mathrm{d}t+\sum I_{ii}{\alpha }_i(t)\int {\alpha }_i(t)\hspace{0.17em}\mathrm{d}t\right\}\hspace{0.17em}.$$ 2.4

Principal component score (PCS) was developed by Greenwald et al. [11] using football data and principal component analysis. It is calculated as a linear combination of HIC, SI, maximum magnitude of linear acceleration and maximum magnitude of angular acceleration, while each term of the kinematics is standardized and the coefficients are fitted empirically [11]:

$$\mathrm{PCS}\hspace{0.17em}=\hspace{0.17em}{\beta }_0+\hspace{0.17em}{\beta }_1\hspace{0.17em}\underset{t}{\max }|a(t)|\hspace{0.17em}+{\beta }_2\hspace{0.17em}\mathrm{SI}+{\beta }_3\hspace{0.17em}\mathrm{HIC}+{\beta }_4\hspace{0.17em}\underset{t}{\max }|\alpha (t)|\hspace{0.17em}.$$ 2.5

Kinematic rotational brain injury criterion (BRIC) was developed by Takhounts et al. [39] by adopting the effects of both angular acceleration and angular velocity. The critical values ω_cr and α_cr were design variables and decided by risk of diffuse axonal injury and the best linear fit between CSDM and the BRIC. Different ω_cr and α_cr were given in [39] and the parameters used were obtained by on-field football data:

$$\mathrm{BRIC}=\frac{\underset{t}{\max }|\omega (t)|\hspace{0.17em}}{{\omega }_{\mathrm{cr}}}\frac{\underset{t}{\max }|\alpha (t)|\hspace{0.17em}}{{\alpha }_{\mathrm{cr}}}.$$ 2.6

Power rotation head injury criterion (PRHIC) was developed by Kimpara et al. [41] with modifications on the HIC. It is calculated based on the following equation under the constraint t₂ − t₁ ≤ 36 ms, where HIP_rot is the second part of HIP contributed by the rotation:

$$\mathrm{PRHIC}\hspace{0.17em}=\hspace{0.17em}\underset{(t_1,t_2)}{max}\left\{{\left(\frac{{\int }_{t_1}^{t_2}|{\mathrm{HIP}}_{\mathrm{rot}}(t)|\hspace{0.17em}\mathrm{d}t}{t_2-t_1}\right)}^{2.5}(t_2-t_1)\right\}\hspace{0.17em}.$$ 2.7

Kleiven's linear combination (KLC) was proposed by Kleiven [42] as a brain injury predictor based on a combination of HIC₃₆ and the maximum magnitude of angular velocity. It is calculated according to the following formula:

$$\mathrm{KLC}=\hspace{0.17em}0.004718\underset{t}{\max }|\omega (t)|+0.000224{\mathrm{HIC}}_{36}.$$ 2.8

Rotational injury criterion (RIC) was developed by Kimpara & Iwamoto [43] in a similar form to HIC but with the linear acceleration replaced by the angular acceleration. RIC is calculated based on the following equation under the constraint t₂ − t₁ ≤ 36 ms:

$$\mathrm{RIC}\hspace{0.17em}=\hspace{0.17em}\underset{(t_1,t_2)}{max}\left\{{\left(\frac{{\int }_{t_1}^{t_2}|\alpha (t)|\hspace{0.17em}\mathrm{d}t}{t_2-t_1}\right)}^{2.5}(t_2-t_1)\right\}\hspace{0.17em}.$$ 2.9

Brain injury criterion (BrIC) was developed by Takhounts et al. [45] based on the assumption that strains calculated by FE modelling can be predicted by angular velocity alone in pendulum and occupant crash tests. It is calculated based on the following formula, where [ω_x, ω_y, ω_z] are the maximum value of the angular velocity in three spatial directions, and [ω_xcr, ω_ycr, ω_zcr] are the corresponding critical values [66.2, 59.1, 44.2] rad s⁻¹ found by experimental data:

$$\mathrm{BrIC}\hspace{0.17em}=\hspace{0.17em}\sqrt{{\left(\frac{{\omega }_x}{{\omega }_{x\mathrm{cr}}}\right)}^2+{\left(\frac{{\omega }_y}{{\omega }_{y\mathrm{cr}}}\right)}^2+{\left(\frac{{\omega }_z}{{\omega }_{z\mathrm{cr}}}\right)}^2}.$$ 2.10

The combined probability of concussion (CP) was developed by Rowson & Duma [10]. It is based on the risk of concussion in football head impacts with the logistic function. The metric is calculated as the basis of the logistic function in a linear combination of the maximum magnitude of translational and rotational accelerations and the interaction between these two terms, where each of the coefficients is determined by logistic regression:

$$\mathrm{CP}\hspace{0.17em}=\hspace{0.17em}{\beta }_0+\hspace{0.17em}{\beta }_1\hspace{0.17em}\underset{t}{max}|a(t)|\hspace{0.17em}+{\beta }_2\hspace{0.17em}\underset{t}{max}|\alpha (t)|\hspace{0.17em}+{\beta }_3\hspace{0.17em}\underset{t}{max}|a(t)|\hspace{0.17em}\underset{t}{max}|\alpha (t)|\hspace{0.17em}.$$ 2.11

Rotational velocity change index (RVCI) was developed by Yanaoka et al. [46] based on pedestrian impact events. It assumed an analogy between the brain strain and the deformation of a spring–mass model. It is calculated based on the following formula, where R_i are the weighing factors related to each axis, which is determined by FE modelling, and the duration constraint was chosen to be t₂ − t₁ ≤ 10 ms:

$$\mathrm{RVCI}=\hspace{0.17em}{\max }_{(t_1,t_2)}\sqrt{R_x{\left({\int }_{t_1}^{t_2}{\alpha }_x\mathrm{d}t\right)}^2+R_y{\left({\int }_{t_1}^{t_2}{\alpha }_y\mathrm{d}t\right)}^2+R_z{\left({\int }_{t_1}^{t_2}{\alpha }_z\mathrm{d}t\right)}^2}.$$ 2.12

The convolution of impulse response for brain injury criterion (CIBIC) was developed by Takahashi & Yanaoka [9]. CIBIC used a similar lumped-mass system as Damage [7], but the coupling effects in different directions were not considered. Furthermore, brain strain caused by the impulse with 1 ms was used to solve the system, and the displacement, which indicated the 95% MPS, was calculated by the convolution integral.

Damage was developed by Gabler et al. [7] based on a three-degree-of-freedom, second-order lumped-mass system. To include the coupling effects between directions, the off-diagonal elements in the stiffness and damping matrices were assumed to be non-zero. The head angular acceleration was used as the input and the displacement of the mass was assumed to indicate the 95% MPS.

Brain angle metric (BAM) was developed by Laksari et al. [8] based on a three-degree-of-freedom lumped parameter brain model and a combined dataset with head impacts from college football, high school football, navy sled tests, etc. BAM is calculated by taking the maximum brain angle ${\mathit{\theta }}_{\hspace{0.17em}\mathrm{brain}}$ in each direction. It is based on the following equation, where k and c are the stiffness and damping coefficient of the system and θ_brain, θ_skull denote the angles of the brain and the skull, with the related model parameters specified [8]:

$$I({\ddot{\theta }}_{\mathrm{brain}}+{\ddot{\theta }}_{\mathrm{skull}})=\hspace{0.17em}-k{\theta }_{\mathrm{brain}}-c{\dot{\theta }}_{\mathrm{brain}\hspace{0.17em}}.$$ 2.13

2.3. Evaluation of brain strain predictability based on brain injury criteria

To evaluate the accuracy of estimating MPS95/MPSCC95/CSDM based on each of the 18 BIC, univariate ordinary least-squares (OLS) regression models were built with each BIC as the predictor and the MPS95/MPSCC95/CSDM as the outcome, respectively. OLS regression was used as it indicates the most direct predictive power of BIC in terms of explained variance (coefficient of determination), and it has been the common practice used by the BIC developers in evaluating the accuracy of BIC [6,8,21,26,33,42,43,45]. Qualitatively, the residuals also generally meet the linear regression assumptions of relative constant residuals and normal distribution.

The univariate regression coefficients were initially analysed to investigate the relationship between each BIC and the brain strain. For OLS, the two key components are the curve's slope and the intercept. The focus was placed on the slope because the slope determines the variance in the outcome that can be explained by the predictor. The intercept, which is in the same unit as the outcome, is used to balance the mean of the outcome in the regression to centre the outcome data. To ensure robust results, the datasets were bootstrapped 1000 times [47]. The mean and 95% confidence intervals of the regression slopes were recorded. The study for each of the 18 BIC was done independently with data standardization (to a mean of zero and an s.d. of 1 for each BIC) before building the regression model.

Next, to further evaluate the cross-usage of BIC in different types of head impacts, the influence of different datasets on the accuracy of using BIC to estimate the MPS95/MPSCC95/CSDM was evaluated. The accuracy of brain strain estimation is recognized as higher if the coefficient of determination (R²) of the OLS regression is higher (equivalent to that the Pearson coefficient of correlation (r) between the predictor and the outcome is higher in this single-predictor regression model).

2.4. Statistical analysis and regression tasks

In the analysis of the regression, the underlying relationship between the BIC and the brain strain across datasets was analysed. Two statistical tests were performed: (i) the one-way ANOVA tests to find whether the regression slopes were statistically significantly different across datasets; (ii) the pairwise Wilcoxon rank-sum tests [48] with the Bonferroni correction to find statistical significance in the regression slope difference between each pair of two datasets. T-tests were not used because the Shapiro–Wilk tests [49] rejected the normal distribution hypothesis on some of the regression slope results, and the Wilcoxon rank-sum test does not rely on the normal assumption of data distribution of the bootstrapped regression slope. The Bonferroni correction was done on the significance level for the multiple tests and we only regarded p-values smaller than 0.005 as statistically significant. What must be mentioned here is that this study compared each BIC across datasets independently. Therefore, there are 10 pairwise tests for the Bonferroni correction in the study of each of the 18 BIC.

Additionally, because the different regression slopes indicate the varying relationship between the BIC and the brain strain, the cross-dataset brain strain estimation accuracy may be different from that in the single-dataset prediction. To analyse the generalizability of the BIC model in estimating brain strain, three different tasks were designed:

• (1) Single-dataset regression: to test the ability of the BIC to estimate brain strain in the same type of head impacts, on each dataset, 80% of the impacts were randomly selected as the training dataset and the remaining 20% impacts as the testing dataset. The remaining 20% impacts were assumed to represent the distribution of the entire dataset from which they were randomly sampled. One hundred iterations of bootstrapping were performed to ensure the robustness of the results. OLS models were fitted on the training dataset and evaluated on the testing dataset, with the mean R² recorded to assess the accuracy. Five models corresponding to five datasets were built and evaluated.
• (2) Cross-dataset regression: in real-world applications, users usually apply BIC with published parameters trained on the development dataset to estimate the brain injury risks of new unseen impacts [33]. To test the ability of the BIC to estimate brain strain on different types of head impacts, a cross-dataset prediction task was conducted by using one dataset as the training dataset and another dataset as the testing dataset. One hundred iterations of bootstrapping resampling were performed on the training dataset and the mean R² was recorded. Twenty models were built and evaluated because each of the five datasets could be either the training dataset or the testing dataset.
• (3) Leave-one-dataset-out regression: this task was completed to seek potential improvement by combining different datasets in training BIC when the brain injury risk of an unseen type of head impact needs to be estimated, which was employed in the development of BAM [8]. To do this, four datasets were combined to be the training dataset with the remaining dataset as the testing dataset. The same pattern of bootstrapping resampling was conducted. Five models were built and evaluated because each of the five datasets was once regarded as the testing dataset.

Based on these three regression tasks, we aim to demonstrate whether the BIC modelled on one type of head impacts can be directly applied to estimate brain strain for another head impact type with similar accuracy.

3. Results

3.1. Analysis of regression slope

To test whether the relationship between the BIC and the brain strain is the same across datasets with different head impact types, the slopes in the OLS regression models were analysed with the outcomes (MPS95/MPSCC95/CSDM) on the predictor (each of the BIC, standardized). Firstly, in all the 270 regression models, the predictor (one of the 18 BIC) was statistically significantly associated with the response (MPS95/MPSCC95/CSDM) (p < 0.01, t-test). Then, one-way ANOVA tests show the statistical significance of the different slopes across datasets on each of the three outcome variables (p < 0.001). Figure 3 and electronic supplementary material, figure S1, show the regression slopes on the different datasets and the pairwise comparison Wilcoxon rank-sum test p-values with the Bonferroni correction. The results show that 524 out of the 540 pairwise comparisons manifest statistical significance at the p-value threshold of 0.005 (Bonferroni correction), and 522 out of the 540 pairwise comparisons manifest statistical significance at the p-value threshold of 0.001. Although the regression slopes are all positive values, which means that larger values in BIC indicate higher brain strain, the statistically significant difference of regression slopes indicates that the exact relationship between the BIC and the brain strain varies with different types of head impacts.

Figure 3.

The p-values of pairwise comparisons of regression slopes on different datasets of head impacts with 1000 iterations of bootstrapping resampling based on the Wilcoxon rank-sum tests with the Bonferroni correction. Datasets 1–5, laboratory impacts, college football, MMA, NHTSA, NASCAR. The p-values of pairwise comparisons in the regression of (a) MPS95, (b) MPSCC95 and (c) CSDM.

3.2. Single-dataset regression (Task 1) and cross-dataset regression (Task 2)

As the analysis of regression slopes indicates the varying relationship between the BIC and the brain strain on different datasets, further testing was performed on the accuracy and generalizability of MPS95/MPSCC95/CSDM estimation on the single dataset or across datasets. The diagonal elements in each plot of figures 4–6 show the coefficients of determination (R²) in single-dataset regression tasks, in which the testing dataset comprised 20% data of the entire dataset, while the training set comprised 80% data of the dataset to fit the OLS regression model. The off-diagonal elements in each plot of figures 4–6 show the coefficients of determination (R²) in cross-dataset regression in which one dataset was used as the training dataset and another dataset as the testing dataset for regression evaluation. The root mean squared error results are shown in electronic supplementary material, figures S2–S4. The diagonal elements in each plot generally show a higher R² in single-dataset regression tasks than the off-diagonal elements which are the cross-dataset regression. There is also a high variance in the performance of different BIC.

Figure 5.

The mean R² in the single-dataset regression and cross-dataset regression of MPSCC95 based on 18 BIC with 100 iterations of bootstrapping resampling. The colour indicates the same training dataset. Dataset 1, laboratory impacts; dataset 2, college football; dataset 3, MMA; dataset 4, NHTSA; dataset 5, NASCAR. (a) SI, (b) HIC, (c) GAMBIT, (d) HIP, (e) PCS, (f) BRIC, (g) PRHIC, (h) KLC, (i) RIC, (j) BrIC, (k) CP, (l) RVCI, (m) CIBIC, (n) damage, (o) BAM, (p) lin_acc_CG_max, (q) ang_vel_max and (r) ang_acc_max.

Figure 4.

The mean R² in the single-dataset regression and cross-dataset regression of MPS95 based on 18 BIC with 100 iterations of bootstrapping resampling. The colour indicates the same training dataset. Dataset 1, laboratory impacts; dataset 2, college football; dataset 3, MMA; dataset 4, NHTSA; dataset 5, NASCAR. (a) SI, (b) HIC, (c) GAMBIT, (d) HIP, (e) PCS, (f) BRIC, (g) PRHIC, (h) KLC, (i) RIC, (j) BrIC, (k) CP, (l) RVCI, (m) CIBIC, (n) damage, (o) BAM, (p) lin_acc_CG_max, (q) ang_vel_max and (r) ang_acc_max.

Figure 6.

The mean R² in the single-dataset regression and cross-dataset regression of CSDM based on 18 BIC with 100 iterations of bootstrapping resampling. The colour indicates the same training dataset. Dataset 1, laboratory impacts; dataset 2, college football; dataset 3, MMA; dataset 4, NHTSA; dataset 5, NASCAR. (a) SI, (b) HIC, (c) GAMBIT, (d) HIP, (e) PCS, (f) BRIC, (g) PRHIC, (h) KLC, (i) RIC, (j) BrIC, (k) CP, (l) RVCI, (m) CIBIC, (n) damage, (o) BAM, (p) lin_acc_CG_max, (q) ang_vel_max and (r) ang_acc_max.

3.3. Leave-one-dataset-out regression (Task 3)

To seek potential improvement by combining different datasets in training BIC when the brain injury risk of an unseen type of head impact needs to be estimated, the leave-one-dataset-out regression was performed by leaving one dataset completely out as the testing dataset and the combination of remaining datasets as the training dataset. As is shown in figure 7, the mean R² was used as the accuracy metric; the accuracy of single-dataset regression tasks generally outperforms the accuracy of the leave-one-dataset-out regression tasks on the same dataset and with the same type of BIC. This result repeats on MPS95, MPSCC95 and CSDM regression, which shows that combining all the data, not from the same type of head impacts in the testing dataset leads to lower accuracy in the brain strain regression.

Figure 7.

Comparison of mean R² in the single-dataset regression task and the leave-one-dataset-out regression task on each dataset and each BIC with 100 iterations of bootstrapping resampling. The mean R² in the regression of (a) MPS95, (b) MPSCC95 and (c) CSDM.

4. Discussion

Although different BIC have been developed based on different types of head impacts, the accuracy of using BIC to estimate brain injury risks across different types of head impacts has previously not been investigated. This study investigated the regression slopes in the OLS regression models of MPS95/MPSCCC95/CSDM on 18 BIC and the generalizability of the regression models to estimate the brain strain if the models are fitted on a different dataset, with five datasets of different types of head impacts (laboratory impacts, college football impacts, MMA impacts, NHTSA impacts, NASCAR impacts). This study found that the regression slopes in most of the OLS regression models showed statistically significant differences across datasets in figure 3 and electronic supplementary material, figure S1. This indicates that the underlying relationship between BIC and brain strain varies with different types of head impacts. For example, if two impacts (one from football and one from car crash) reached the same value of BIC, the brain injury risks for these two impacts can still be very different. A potential explanation is that the characteristics of the kinematics used by BIC to estimate brain strain vary among different types of head impacts [50]. Furthermore, the difference in the relationship between BIC and brain strain suggests that the risks of brain injury vary among different types of head impacts.

The influence of cross-using BIC across head impact types on the accuracy of the brain strain estimation is shown in figures 4–6 and electronic supplementary material, figures S2–S4. The bars were generally higher on the diagonal than off-diagonal, indicating that the single-dataset regression tasks generally showed higher mean R² than those in the cross-dataset regression tasks. In other words, the models were more accurate in the regression of MPS95/MPSCC95/CSDM if they were fitted on the same type of head impacts. It should be noted that exceptions did exist when the off-diagonal regression accuracy was higher, and the potential reasons may be the insignificant slope difference in the regression of the two datasets and the randomness in a small dataset. This finding suggests that the accuracy of the BIC should be evaluated according to the different types of head impact. In previous studies related to BIC, the evaluation of the accuracy in brain strain regression has been mainly done on the same dataset. Although different types of head impacts were combined to develop BIC in previous studies (table 1), it was only until recently that researchers started to investigate the declining performance of the brain strain regression models when applied to a different type of head impact. Zhan et al. [20] developed a deep learning head model for predicting brain strain of the entire brain and showed that the model accuracy deteriorates when the model was trained on a dataset mainly comprising simulated head impacts and applied to MMA head impacts. According to the finding in this study, concerns may arise if the BIC is used to estimate the brain strain if the BIC regression model is fitted on a different type of head impact. Therefore, the BIC, the reduced-order head models, and deep learning head models developed by researchers to predict brain injury risks need to be validated across different types of head impacts if they are ever intended to be used that way.

Additionally, although this study found statistically significant different regression slopes across datasets in figure 3, the findings also showed that there might be similarities between certain pairs of head impact types. In terms of the regression slopes in figure 3, on BrIC and BRIC, statistical significance was not found in the pairwise comparisons between laboratory impacts and NASCAR impacts and between MMA impacts and NASCAR impacts, in the regression of MPS95. On CP, no statistical significance was found in the pairwise comparison between MMA impacts and NHTSA impacts in the regression of MPS95 and CSDM. On RVCI, no statistical significance was found in the regression of MPS95 in the pairwise comparisons between laboratory impacts and college football impacts; no statistical significance was found in the regression of MPSCC95 in the pairwise comparison between laboratory impacts and NHTSA impacts. On CIBIC, no statistical significance was found in the pairwise comparisons between laboratory impacts and college football impacts, between laboratory impacts and NASCAR impacts, and between college football impacts and NASCAR impacts, in the regression of CSDM. On GAMBIT, in the regression of MPSCC95, no statistical significance was found between laboratory impacts and NASCAR impacts and between college football impacts and NHTSA impacts. On PRHIC, no statistical significance was found between college football impacts and NHTSA impacts in the MPSCC95 regression. These similarities among different types of head impacts can also be indicated in the cross-dataset regression results in which there were several high bars off the diagonal. For example, high mean R² was shown in the pairs (training dataset listed first in each pair): NASCAR impacts–laboratory impacts, NASCAR impacts–college football impacts, college football impacts–NASCAR impacts, college football impacts–laboratory impacts and MMA impacts–college football impacts. The possible explanations may be: (i) the laboratory impacts were generally simulated for the research of head impacts in football matches, which may explain the similarity between laboratory impacts and college football impacts; (ii) the MMA head impacts and NHTSA car crash dummy head impacts were measured without protective helmets, while the college football head impacts and the NASCAR head impacts were measured with protective helmets that protect the players/drivers from concussion. The helmet can effectively change the impact response of the head [51], which may explain the similarity between MMA impacts and NHTSA impacts, as well as among NASCAR impacts, college football impacts and laboratory impacts.

Comparing the performances of BIC in a single-dataset task in figures 4–6, most BIC definitions can accurately estimate brain strain, and the BIC that included angular velocity as input (BRIC, BrIC, RVCI, ang_vel_max), as well as the BIC developed by solving the mass-lumped systems (damage, CIBIC, BAM), generally showed higher accuracy (the high diagonal bars), which agrees with the results published in [7,15,52]. Furthermore, these two types of BIC also provided generally accurate regression in the cross-dataset task (the high off-diagonal bars) except for BAM. The different performances of BIC in cross-dataset tasks derive from the fact that each BIC was designed to capture certain characteristics of the head kinematics to estimate the brain strain, and these characteristics varied among different datasets. Although the mechanism underlying how the head kinematics caused the brain strain was not clear, a potential factor that might influence the performance in the cross-dataset task was the impulse duration. As observed in [6] and explained in [16], the MPS95 depends on the angular velocity peak when the impulse duration was short, and on the angular acceleration peak when the impulse duration was longer. The impulse duration was only defined for idealized impact instead of the on-field impact. It is possible that the difference in impulse duration contributed to the difference in BIC performances. Aside from the impulse duration, the multiple peaks (figure 1a) and the different profiles [52–55] were also potential factors that influenced the cross-dataset performance.

The lower accuracy in leave-one-dataset-out regression in figure 7 suggested that combining several different types of head impacts as the training dataset may fail to estimate the brain strain in the unknown type of head impacts. As shown in table 1, most of the BIC were developed based on the football and traffic accident datasets. Therefore, researchers and medical professionals should carefully interpret the results of applying BIC to another type of head impact. For the type of head impact in which the head kinematics characteristic is unknown, sufficient impact cases should be collected and analysed by an FE model before using the BIC. Furthermore, deep learning head models were recently developed based on football dataset [56] and football, MMA and laboratory impact datasets [20], and these models have achieved promising accuracy. Although different datasets were used in [20], the ability of the model to predict the brain strain in an unknown type of head impact was not examined.

Furthermore, multiple negative mean R² occurred in this study in figures 4–7, mainly in the cross-dataset regression tasks and leave-one-dataset-out regression tasks but also in the single-dataset regression tasks. The negative mean R² in the results may be explained by the significantly different distributions of the training dataset and the test dataset. In the cross-dataset and leave-one-dataset-out tasks, the negative values indicate that the distribution of the other datasets may be insufficient to represent the distribution of the selected testing dataset so that the regression models showed high bias. In the single-dataset tasks, the negative values may be caused by the fact that the random sampled 20% testing data may not be well covered by some of the bootstrapping resampled training dataset distributions. For instance, the MMA dataset had two peaks with significantly different kernel densities, which indicates that there are much fewer data points in the minor peak region (as is shown in the red box in figure 2). These may be caused by several exceptionally fierce fights with severe head impacts and the collection of more data may address this issue. The data points located in the minor peak of the MPS95/MPSCC95 distributions of the MMA dataset in the testing dataset might not be well covered by the distribution of the bootstrapping resampled training dataset because the bootstrapping resampled training dataset theoretically covers 63.2% samples in the training data [57]. The NHTSA dataset, which only contains 48 head impacts, may also suffer from the distributional mismatch between the testing dataset and the training dataset.

In addition, the decrease in the regression accuracy from single-dataset tasks to cross-dataset tasks and leave-one-dataset-out tasks also indicates the issue of overfitting of the BIC models, even with simple OLS regression. Because data fitting is widely used to assess BIC accuracy (e.g. fitting a regression model of 95% MPS on HIC) by evaluating the goodness of fit, however, to fit the model and to evaluate the goodness of fit on the same dataset may cause overfitting. The overfitting can lead to an over-optimistic estimate of the accuracy of the BIC. Therefore, instead of fitting the BIC and evaluating the goodness of fit on the same dataset, the accuracy of the BIC across different head impact datasets also should be validated.

There are a few important limitations of this study to note: firstly, most of the BIC definitions were developed with parameters by optimizing the brain injury risk estimation in certain datasets (table 1). In this study, the BIC values were used directly with the parameters reported in the papers. It is possible that recalculating the parameters according to each dataset may probably generate higher accuracy than what was reported in this paper. Therefore, it is suggested that each BIC should be further improved for different types of head impacts in the future.

Additionally, we used the KTH model [32] as the finite-element head model (FEHM), which is relatively simplified compared to the current state-of-the-art FEHM [58–60]. The KTH model has tangential sliding without separation in the normal direction between subdural cerebral spinal fluid and the brain. In addition, there are no gyri or sulci, which have been confirmed to have a significant influence on the FEHM behaviour and its intracranial motion. The KTH model also lacks the interface between white and grey matter. In the future, new FEHM developed in recent years can be applied and tested.

Also, it was assumed that the testing dataset represented the head kinematics in the type of head impacts. This assumption should be examined particularly when the dataset distribution is skewed with multiple peaks or heavy tails. In the future, as more head impact data become available, this assumption will be more robust. Finally, the OLS regression model was the only method used to predict MPS95/MPSCC95/CSDM from the BIC because it directly shows the ability of brain strain estimation based on BIC; however, the underlying relationship between BIC and the brain strain may also be nonlinear [25]. The simple linear regression may have certain biases. Neural-network-based classifiers may perform better across the cohort of comparisons listed here but may even more drastically overfit the data. Lastly, we regarded the ability to predict brain strain as the ability to evaluate brain injury risk. More neurological studies need to be done to validate the brain strain as an accurate supplant of brain injury risk.

5. Conclusion

In this study, we investigated the varying relationship between the BIC and the brain strain and the different accuracy of brain strain estimation over different datasets of head impacts. The slope values of linear regression from each of the 18 BIC to brain strain (MPS95/MPSCC95/CSDM) manifest statistically significant differences across datasets. Furthermore, the accuracy of the estimation of MPS95/MPSCC95/CSDM on each BIC may be lower if the regression models are fitted on different training datasets and then used to estimate the brain strain on another testing dataset. The findings indicate that researchers and medical professionals who use the BIC to monitor the severity of head impact and to estimate the brain strain should take extra caution to interpret the BIC results in different types of head impact. Based on the findings, it may also be advantageous to develop type-specific BIC brain strain estimation models on the same type of head impacts as the head impacts which need to be evaluated.

Data accessibility

The data and codes used in this study (including the script for BIC calculation) are reported at: https://github.com/xzhan96-stf/BIC.

Authors' contributions

X.Z., Yi.L. and Yu.L. conceived this study, processed data, conducted the experiments and analysed the results, and wrote the manuscript. H.V.A., J.R., S.B., S.T. and Yu.L. were involved in the data collection. A.G.D., S.J.R., O.G., M.M.Z., G.A.G. and D.B.C. revised the manuscript. Yu.L., M.Z., G.A.G. and D.B.C. supervised this study. D.B.C. provided the funding for this research. X.Z. and Yi.L. contributed equally to this work.

Competing interests

We declare we have no competing interests.

Funding

This research was supported by the Pac-12 Conference's Student-Athlete Health and Well-Being Initiative, the National Institutes of Health (R24NS098518) and Taube Stanford Children's Concussion Initiative.