Quantifying head acceleration exposure via instrumented mouthguards (iMG): a validity and feasibility study protocol to inform iMG suitability for the TaCKLE project

Abstract

Instrumented mouthguards (iMGs) have the potential to quantify head acceleration exposures in sport. The Rugby Football League is looking to deploy iMGs to quantify head acceleration exposures as part of the Tackle and Contact Kinematics, Loads and Exposure (TaCKLE) project. iMGs and associated software platforms are novel, thus limited validation studies exist. The aim of this paper is to describe the methods that will determine the validity (ie, laboratory validation of kinematic measures and on-field validity) and feasibility (ie, player comfort and wearability and practitioner considerations) of available iMGs for quantifying head acceleration events in rugby league. Phase 1 will determine the reliability and validity of iMG kinematic measures (peak linear acceleration, peak rotational velocity, peak rotational acceleration), based on laboratory criterion standards. Players will have three-dimensional dental scans and be provided with available iMGs for phase 2 and phase 3. Phase 2 will determine the on-field validity of iMGs (ie, identifying true positive head acceleration events during a match). Phase 3 will evaluate player perceptions of fit (too loose, too tight, bulky, small/thin, held mouth open, held teeth apart, pain in jaw muscles, uneven bite), comfort (on lips, gum, tongue, teeth) and function (speech, swallowing, dry mouth). Phase 4 will evaluate the practical feasibility of iMGs, as determined by practitioners using the system usability scale (preparing iMG system and managing iMG data). The outcome will provide a systematic and robust assessment of a range of iMGs, which will help inform the suitability of each iMG system for the TaCKLE project.

Key messages.

What is already known

• The accurate quantification of head acceleration events (HAE) in sport is desirable.

• Instrumented mouthguards (iMGs) are commercially available, and have the potential to measure HAE kinematics.

• The accurate quantification of HAE using instrumented mouthguards (iMGs) can help uncover the biomechanical mechanisms of injury for concussion.

What are the new findings

• This study will determine the laboratory-based and on-field validity of iMG for measuring HAE.

• This study will determine the fit, function and comfort of iMGs from a players perspective.

• This study will determine the practical feasibility of iMG and associated systems from a practitioners perspective.

Introduction

Instrumented mouthguards (iMGs) have considerable potential to allow the accurate quantification of head acceleration events (HAE) in sport. iMGs have the potential to help uncover the biomechanical mechanisms of injury for concussion, which remains a priority for sports. The application of iMGs across levels (professional to amateur) and age groups (senior to youth), in both male and female cohorts, can provide sporting governing bodies with data to make evidence-based performance, welfare and participation decisions across the game. A proactive approach to player welfare would be to develop mitigation strategies to help reduce the magnitudes of HAE from direct head impacts or inertial head loading (ie, head acceleration from impacts to the body) without compromising the dynamics of the sport.

HAE kinematics can be measured by wearable head sensors, instrumented with accelerometers and/or gyroscopes.¹ HAE studies in rugby league^2–4 are currently limited to instrumented patch devices, which suffer from poor skull coupling, and thus have a tendency to overestimate HAE kinematic measures.¹ iMGs demonstrate superior coupling with the skull through the upper dentition and have been recommended for in vivo HAE measurement.¹ iMGs have previously been used to describe HAE in rugby union^{5 6} and American Football.^7–9

To ensure valid data are collected, it is paramount that the most appropriate iMG hardware, firmware and software (eg, associated platforms/portals) are deployed within research studies. Numerous iMGs are available for both commercial and research use. Due to the novelty of iMGs and associated software platforms, limited validation studies exist; therefore, it is difficult to determine the appropriateness of the new technology. Understanding the validity (both laboratory-based and field-based) and feasibility (eg, use by players and practitioners) will influence the success of any research project using iMGs. It is also important to ensure that players wear the iMG during training and match-play to ensure that accurate longitudinal head kinematic data are collected. Practitioner buy-in is also important, given the need to manage iMGs and associated software.

Rugby league is a contact sport, involving frequent tackle events, played at professional and community levels.^10–12 The impact-based nature of rugby league means players are exposed to activities during training and matches that result in HAE. The Rugby Football League (RFL) is the national governing body for rugby league in the UK. The RFL is due to start a large-scale project, which will quantify and describe the head acceleration exposures (TaCKLE project; Tackle and Contact Kinematics, Loads and Exposures) across rugby league in England. The TaCKLE project aims to deploy approximately 1200 iMG across 50 teams, inclusive of men’s Super League, women’s Super League, Academy (male) and the community game (senior to youth age groups). Prior to the selection and deployment of the iMGs, a study into the validity and feasibility of iMGs is required.

This protocol paper outlines the research aims, study design and methodology for a study that will assess the validity and feasibility of iMGs. The findings will support the decision-making process when determining the suitability of iMG devices for the TaCKLE project. The TaCKLE project aims to start in the 2022 rugby league season.

Research aims

The aim of this study is to determine the validity (ie, laboratory validation of kinematic measures and on-field validity for identifying true-positive HAE) and feasibility (ie, player comfort and wearability and practitioner considerations) of available iMGs for quantifying HAE in rugby league. The study will include four phases. The design of the study and concurrent undertaking of each phase is influenced by the duration of time available to complete the project, as determined by the industry funder (ie, RFL in preparation for the TaCKLE project). Patients or public were not involved in designing the study as this was not appropriate. The project has received ethics approval from Leeds Beckett University (reference number: 85551). iMG companies will be identified and invited to provide hardware and software for the study. This manuscript (preprint) will be shared with iMG companies for transparency. iMG companies will be invited to provide a sample of 43 iMG (n=3 phase 1, n=20 phase 2, n=20 phase 3).

• Phase 1 will determine the reliability and validity of the iMG kinematic magnitude measures, based on laboratory criterion standards.^{13 14}

• Phase 2 will provide a sample of academy players with iMGs to wear in matches, to determine the on-field validity of the iMG.^{13 15 16}

• Phase 3 will provide a sample of elite rugby league players with iMGs to evaluate fit, function and comfort from a player’s perspective.¹⁷

• Phase 4 will evaluate the practical feasibility of the iMG, from a practitioner’s perspective.^{18 19}

Phase 1—laboratory validation of kinematic measures

Phase 1 is required to ensure that HAE data recorded on the iMGs are reliable and valid. The methodology builds on the work of Kieffer et al¹³ who conducted a two-phased approach evaluating the accuracy of a range of wearable head sensors (eg, mouthguards, ear patches). The aim of in-lab testing of kinematic measures (phase 1) is to evaluate the reliability and validity of the iMG kinematic magnitude measures at the Virginia Tech Head Impact Lab (USA). The costs of the in-lab testing will be covered by Leeds Beckett University. In-lab testing can quantify measurement error through comparison to reference measurements in dummy headforms that are considered ground truth.¹³ The dummy headform configuration consists of a medium-sized National Operating Committee on Standards for Athletic Equipment (NOCSAE) headform attached to a Hybrid III 50th percentile male neck, mounted on a linear slide table with 5 degrees-of-freedom.^{20 21} Reference kinematics are measured at the headform centre of gravity with an instrumentation package consisting of three linear accelerometers (Endevco 7264b-2000; Meggitt Orange County, Irvine, California) and a triaxial angular rate sensor (DTS ARS3 Pro 18k; Diversified Technical Systems, Seal Beach, California).

In-lab testing will be conducted using a pendulum impactor to simulate bareheaded impacts to the dummy headform. Tests will be performed with a rigid (nylon, 25 mm thickness) and padded (vinyl nitrile foam, 40 mm thickness) impactor to the bareheaded dummy headform to capture the range of impact magnitudes and durations seen in rugby. The impactor is 127 mm in diameter. Impacts will occur at the front, front boss, rear boss and rear locations of the headform at target linear head accelerations of 25 g, 50 g, 75 g and 100 g. Two tests will be conducted at each configuration and three iMGs will be tested for each company. The custom-fit iMG will be mounted inside the headform with a detachable three-dimensional (3D) printed detention composed of acrylonitrile butadiene styrene plastic. An aluminium plate will be inserted in the space between the iMG and lower jaw of the headform. The plate will be screwed upward into place using a torque wrench (set to 2Nm) until there is no gap between it and the mouthguard-clad dentition. The plate will act as a clamp to prevent relative motion of the iMG during testing and could be considered to simulate jaw clenching. Specific variables of interest will be peak linear acceleration, peak rotational velocity and peak rotational acceleration.

Statistical analysis (phase 1)

iMG performance will be assessed by calculating the concordance correlation coefficient (CCC) (Equation. 1).¹³ CCC quantifies the agreement between the sensor measurements and reference measurements based on the deviation of paired measurements relative to the concordance line. CCC addresses misleading characteristics of agreement like location shift, scale shift and precision errors.

$${\displaystyle CCC=\frac{2\rho }{v+\frac{1}{v}+u^2}wherev=\frac{S_x}{S_y}andu=\frac{\hat{x}-\hat{y}}{\sqrt{S_x{S}_y}}}$$

(1)

In Equation (1), ρ represents the Pearson correlation coefficient, x and y represent the reference and sensor measurements, respectively. x̂ and ŷ represent the measurement means and S_x and S_y represent the measurement standard deviation (SD). Sensors output the peak resultant values for linear kinematic measures (eg, peak linear acceleration) and rotational kinematic measures (eg, peak rotational acceleration). After the sensor and reference measurements are recorded, both are normalised relative to the maximum reference measurement. CCC values are then computed for the linear kinematic measure, the rotational kinematic measure(s) and then the combination of linear and rotational measures. The combined CCC value that accounts for peak linear acceleration and peak rotational acceleration will represent the iMG in-lab performance for this study. A previous study illustrated that these two measurements had the greatest variation between iMG devices.¹⁴

Phase 2—on-field validity

Phase 2 is required to ensure that HAE recorded by the iMGs during rugby league match play are valid. Phase 2 will focus towards on-field validity of the iMG. Ten professional male academy rugby players from a single club will be equipped with one iMG from two companies and be required to wear each for two matches to evaluate the on-field validity of each iMG. The total number of players recruited will be based on the number of iMG companies that provide hardware and software for the study. Each player will have a 3D dental scan, undertaken by an experienced dentist, allowing iMG companies to be provided with the same dental details to manufacture the custom-fit iMG. The costs of the 3D dental scan will be covered by Leeds Beckett University. iMG companies will be provided with one opportunity to rectify any problems or errors that occurred in manufacturing the iMG, based on the dental scan. This will need to occur between the agreed delivery date of the iMG and the start of phase 2. The management of the iMG data will be undertaken by a member of the research team. Events recorded by the iMG will be time synchronised with high-quality video footage of match play to verify that each iMG-triggered event (after the companies’ HAE detection algorithm is applied, if applicable) was associated with a HAE experienced by the athlete. A HAE can be either to an athlete’s head or body, as both impart acceleration to the head. Only events with peak resultant linear acceleration greater than 10 g will be verified (or a threshold of the company’s choosing following discussions with the research team), as 10 g has commonly been used as a lower threshold in previous iMG studies to distinguish HAE from voluntary movements like running, changes in direction and jumping.^{5–9 16} However, a recent review illustrated that HAE magnitudes during these types of events are dependent on the wearable head sensor used to measure it, with helmet and headband-based sensors typically showing higher magnitudes than biteplate-based sensors (which could be considered similar to iMG).²² Therefore, iMG companies will be given the option to select a higher or lower threshold. A guided assessment will be undertaken where each iMG-triggered event will be cross-referenced with the video footage and classified by a trained video analyst as either a true-positive or false-positive event. True-positive events are defined as iMG-triggered events where there was a time-matched HAE (ie, impact to the body or head) with the athlete observed on video. False-positive events are defined as iMG-triggered events where there was no time-matched HAE (ie, no impact to the body or head) with the athlete observed on video.

False-negative events are HAE that are not recorded by iMGs. Identifying false negative events is a challenge due to the difficulty in objectively detecting HAE and estimating their severity from video review.¹⁵ Overly aggressive HAE detection filters embedded in iMG systems could perform well at removing false positive events at the cost of poor false negative performance. Therefore, an unguided assessment will be conducted for one-on-one shoulder tackle events for tacklers only. One professional rugby league video analyst who routinely reviews match video footage will track each player wearing an iMG during the matches and label all of their one-on-one shoulder tackle events as the tackler based on qualitative video review.²³ The timestamp of each labelled one-on-one shoulder tackle event will be cross-referenced with the iMG-triggered data set. The event will be recorded as a true-positive or false-negative based on whether the one-on-one shoulder tackle event timestamp does or does not match an iMG-triggered timestamp, respectively. The 6 degree-of-freedom head displacement during the HAE will be reconstructed from the linear and angular kinematic time series data using a customised MATLAB script to aid the analyst’s ability to identify the HAE in the video.

Statistical analysis (phase 2)

Positive predictive values (PPV) (Equation 2) will be calculated from the true-positive and false-positive event counts for each sensor that was tested on-field.¹³ 95% confidence intervals (CI) will be computed for PPV through bootstrapping.¹³ The role that inactive participation during matches could have on the proportion of false-positive events will be addressed. To account for this, activity logs will be generated for each session that differentiated periods of active play from inactivity (eg, substitutions, half-time). PPV will be computed two ways: over entire session lengths and then again for only periods of active play within sessions. For the unguided analysis on one-on-one shoulder tackles only, a sensitivity score (with 95% CI) will be calculated (Equation 3) from the true-positive and false-negative counts for each iMG sensor that was tested on-field.

$${\displaystyle PositivePredictiveValue=\frac{TruePositives}{TruePositives+FalsePositives}}$$ (2)

$${\displaystyle Sensitivity=\frac{TruePositives}{TruePositives+FalseNegatives}}$$ (3)

Phase 3—assessment of player comfort and wearability

Phase 3 is required to ensure that players are able to wear the iMGs, which will support the collection of longitudinal data. Player perception of iMG is important as this largely determines the compliance to wear iMGs during training and matches. The aim of phase 3 is to compare player perceptions of custom-made iMGs provided by mouthguard companies. Twenty senior professional male players from Super League clubs will be recruited via the national governing body (RFL) to participate in phase 3. No more than five players per Super League club will be recruited, to limit potential player-to-player influence on player perceptions. Clubs that already have experience with iMG systems will not be recruited for phase 3 of the study. Players will be required to score the iMG based on comfort, fit and function (ie, speech, swallowing, dry mouth). Each player will have a 3D dental scan allowing iMG companies to be provided with the same dental details to manufacture the custom-fit iMG. The costs of the 3D dental scan will be covered by Leeds Beckett University. iMG companies will be provided with one opportunity to rectify any problems or errors that occurred in manufacturing the iMG, based on the dental scan. This will need to occur between the agreed delivery date of the iMG and the start of phase 3.

Each player will be provided with one iMG from each iMG company and asked to wear the iMG during two training sessions, each of at least 45-minute duration. The order players will be asked to wear the iMGs will be randomised in a cross-over design. One hour following the completion of the second training session, players will be asked (via online questionnaire) to rate the comfort, fit and function of the iMG as per previous methods.¹⁷ A 1–10 Likert Scale (1 being the poorest score and 10 the highest with the poles of the scales labelled accordingly) will be used to rate the comfort of mouthguard. Overall comfort and comfort on lips, gum, tongue, teeth will be included. The fit of mouthguard will be evaluated using a binary (yes [0]/no [1]) for the following questions; too loose, too tight, bulky, small/thin, held mouth open, held teeth apart, pain in jaw muscles, uneven bite. A 3-point Likert question (no[1] /a little [0.5]/a lot [0]) will be used to determine the function of mouthguard, and whether it interferes with speech, interferes with swallowing, causes a dry mouth.

Statistical analysis (phase 3)

Given the repeated measures design (same player evaluating multiple mouthguards) and the presence of multiple individual questionnaire items evaluating comfort, fit and function factors of the iMG, multiple factor analysis will be conducted to compare overall player perception between iMG companies using the factorMineR package in R studio. Multiple factor analysis allows consideration of the variability in scores between individual questionnaire items (eg, lip comfort) that comprise the contributory groups (ie, comfort, fit and function) and supplementary groups (ie, player and iMG company) to produce overall factor scores for each iMG company. The first and second dimension of the multiple factor analysis will be retained for interpretation to compare the differences between iMG companies. iMG companies will be ranked highest to lowest for player perception from their factor scores of the first and second dimensions.

Phase 4—assessment of practitioner considerations

Phase 4 is required to ensure that practitioner perceptions of the iMG preparation and data management are considered, which will improve player compliance and the quality of data (from a study design perspective) collected during the TaCKLE project. The iMG technology requires the practitioner to engage in two main processes, which includes preparing the iMG system (charging and deploying the mouthguard to participants) and managing iMG data (extracting data to interface/access of information and feedback mechanisms within system). A practitioner evaluation and comparison of the usability of these two aspects are necessary to understand the practical feasibility of each iMG company’s system for use across multiple sports clubs. The aim of phase 4 is to determine the extent of satisfaction by practitioners when using the overall iMG system and how this differs between companies. Practitioners working in clubs (eg, club medical staff, sports scientists), where players were recruited for phase 3 will be invited to participate. Each practitioner will evaluate the usability of the two different iMG systems used by players in phase 3. iMG companies will be invited to provide an ‘on-boarding’ session to practitioners to provide a familiarisation with operating procedures for that iMG system. Practitioners will have access to operating procedure documentation throughout this period. Practitioners will then complete two training sessions whereby they will complete the two identified processes (ie, preparing the iMG system and managing iMG data). Following the final session, practitioners will complete an online form comprising two poststudy usability assessment questionnaires: one to evaluate the preparation of iMG data and one to evaluate the management of iMG data. This will be supported by opportunities to provide general, ‘open-ended’ feedback opportunities on the two processes.

Practitioners will provide two assessment scores, one for the preparation of the iMG data and one for the management of the data. The Software Usability Scale (SUS) is a standard, well-established questionnaire in the literature to obtain an overall rating of usability (table 1).¹⁸ There is possibility to change the words like ‘system’ to ‘interface’ and formulate it to suit the current process as studies have shown no effect on outcomes due to these changes.¹⁹

Table 1.

The System Usability Scale (standard version)

		Strongly disagree				Strongly agree
		1	2	3	4	5
1	I think that I would like to use this system	O	O	O	O	O
2	I found the system unnecessarily complex	O	O	O	O	O
3	I thought the system was easy to use	O	O	O	O	O
4	I think that I would need the support of a technical person to use this system	O	O	O	O	O
5	I found the various functions in the system were well integrated	O	O	O	O	O
6	I thought there was too much inconsistency in this system	O	O	O	O	O
7	I would imagine that most people would learn to use this system very quickly	O	O	O	O	O
8	I found the system very cumbersome to use	O	O	O	O	O
9	I felt very confident using the system	O	O	O	O	O
10	I needed to learn a lot of things before I could get going with this system	O	O	O	O	O
Calculating system usability score18: Odd-numbered questions (subtract 1 from the score). Even-numbered questions (subtract their value from 5) Take these new values and add up the total score then multiply by 2.5.

Analysis (phase 4)

The industry standard of average SUS score for internet-based web pages and applications is 68.05.²⁴ Therefore, an overall SUS score of above 68 illustrates the usability of the interface to be above industry average. iMG companies will be ranked for the SUS for both the preparation and management of iMG data using the SUS score.

Conclusion

iMGs have considerable potential to allow the accurate quantification of HAE in sport and help uncover the biomechanical mechanisms of injury for concussion. Given the emergence of this new technology, there is a need for an independent scientific validation and feasibility assessment. The study will determine the validity (ie, laboratory validation of kinematic measures and on-field validity) and feasibility (ie, player comfort and wearability and practitioner considerations) of available iMGs. The outcome will provide a systematic and robust assessment of a range of iMGs, which will help the TaCKLE project team understand the suitability of available iMG systems, which will then be used alongside other considerations (eg, cost and other project deliverables).

Ethics statements

Patient consent for publication

Not required.