Validity and reliability of a standalone low-end 50-Hz GNSS receiver during running

Abstract

The aim of the investigation was to verify the validity and reliability of a low-end 50-Hz Global Navigation Satellite System receiver (GNSSr) for different soccer-specific run distances and average speed assessments. Six soccer players were assessed on two different days while performing eight different running paths with changes of direction for a final total of 44 runs. During the runs, each participant was equipped with the GNSSr, while the time for each single run was recorded using a photocell gate. Reference vs. receiver assessment correspondences for distance and average speed were evaluated by calculating the standard error of the estimate (SEE), coefficient of variation (CV), and mean bias. Residual vs. predicted value comparison was performed by means of Bland-Altman plots. Finally, calculating the intra-class correlations coefficient (ICC) assessed the test-retest reliability of the measurement. Receiver distance assessment showed an SEE of 0.52 m (0.73%), and mean bias of 0.06 m. Receiver average speed assessment showed an SEE of 0.02 m·s^-1 (0.74%) and mean bias of 0.001 m·s^-1. The Bland-Altman plot showed a small difference between the two assessments with the 95% limits of agreement=±1.08 m/0.046 m·s^-1. Receiver distance/speed assessment was found to be reliable, with ICC=0.999. In spite of its low cost, the new low-end GNSSr provides valid and reliable assessments of distance and average speed for young adults performing several standardized running actions of differing lengths within delimited setup spaces.

INTRODUCTION

Global navigation satellite system receivers (GNSSrs) have become a common tool to assess players’ physical activity during competition and training in team sports [1]. Coaches have preferred use of GNSSrs over other tracking techniques (e.g. video analysis) thanks to its time efficiency and real-time feedback [2]. GNSSr presents both good validity and reliability for assessment of distance and speed in some linear displacements [3,4] and during team sport simulated motion activity [4-6].

Current 1-15 Hz sampling-frequency GNSSr technology may present some limitations when measuring distance and average/instantaneous speed in confined spaces and/or during high-speed movements [2,7-11], with potential significant underestimations. Studies showed that a GNSSr’s reliability decreases when measuring distance and average/instantaneous speed during tasks requiring high-speed change of direction (COD [7,8,12]), with the coefficient of variation reaching 33% [9]. Such tasks are common in team sports, with players frequently changing direction and stopping/starting [13]. Ability to change direction is a required skill, as well as a key factor of success [13]. Athletes may perform ~600 turning movements per match and more than half of all sprints (~3 s) involve at least one COD [12].

Increasing GNSSr sampling-frequency above 15 Hz might or might not improve GNSSr technology in measuring skills and actions involving quick and repetitive COD [11]. With a higher sampling frequency, it may be possible to capture data regarding changes in average speeds even in court-based movements. Nevertheless, a GNSSr with a higher sampling frequency should be validated and would be expected to present good reproducibility in that condition. The aim of this investigation was to verify both good validity and reliability of a low-end 50-Hz GNSSr for distance assessment and average speed measures in running, including multiple CODs. It is important to remember that a low-end (non-differential) GNSSr can provide only several-meter accuracy, whereas high-end (differential) models can reach up to several-centimetre accuracy [14]. We hypothesize that some relationships will emerge between: i) direct distance assessment of COD runs and distance measured by GNSSr; and ii) direct assessment of average speed and average speed assessed by GNSSr.

MATERIALS AND METHODS

Subjects

Six male soccer players (age 27.4±0.9 years, height 173±5 cm, mass 68.0±7.6 kg, training experience in team sport 8.4±3.0 years) were recruited from some local sport clubs. All players, in addition to weekly practice, participated in the seasonal championship during the regional phase. Inclusion criteria to participate in the study were: i) participation in at least 85% of training sessions, ii) regularly participating in previous competitive seasons, iii) having valid sport medical certification, and iv) being healthy (no pain/injury in the last year) and clear of any drug consumption. Participants gave written consent after being thoroughly informed about the study’s purpose, benefits, and risks in conformity with the World Medical Association Code of Ethics (Declaration of Helsinki). The university human ethics committee followed ethical standards for human studies and approved all experimental procedures.

Procedures

Participants refrained from drinking alcohol or beverages containing caffeine for 24 hours and did not eat for 3 hours before testing, to reduce their possible interference in the experiment. Each participant completed all trials in the same time period of the testing days and under the same environmental conditions (3:00-5:00 p.m. [i.e., common soccer match time], 21.5±0.3°C temperature, and 46.2±1.4% relative humidity), in order to eliminate any influence of circadian variation and environmental condition. All tests were performed on a regular outdoor soccer pitch (without any closely surrounding natural or artificial obstacles), and the participants wore their official soccer dress.

During players’ warm-up (10-min running at low self-selected speed), two operators switched on GNSSrs and for approximately 10 min the device was fixed to “Zero Points” (the start point for each drill) as the reference axis on outdoor soccer pitch and also to capture the maximum number of satellites. Runs were performed on a soccer pitch along paths measured with a measuring tape with 1-mm sensitivity (Ferritalia Soc. Coop., Padua, Italy).

Global navigation satellite system receiver

Each player was equipped with a 50-Hz 167-channel GNSSr receiving signals only from GNSS GPS (Spin_GNSS_50Hz, Spinitalia S.r.l., Pomezia, Italy), while each run time was recorded using a photocell gate (Brower Timing System, Salt Lake City, UT, USA; accuracy of 0.01 s) connected by means of an external connector to a 100-Hz chronograph (Delta E200, Hanhart, Gütenbach, Germany) set to GPS time for GNSSr continuous signal synchronization.

Each participant was asked to complete as fast as possible previously measured paths in order to evaluate GNSSr assessment accuracy in match play-like conditions. For test-retest reliability assessment, each player was assessed on two different days while performing multiple-COD runs (Figure 1). Each player was always in the operator’s field of view to check for correct run execution. There was a 2-min passive recovery between each run. Administered multiple-COD runs were standardized exercises (e.g., with predetermined and imposed COD number). Therefore there was limited possibility for a participant to perform them differently over two trials.

FIG. 1.

The eight different running paths with changes of direction performed by the subjects.

Data analysis

Before and after each trial the GNSSr was checked by a researcher – always positioned exactly in the same pitch spot – to verify correct positioning repeatability and signal continuity. Data were transferred with the manufacturer’s software (Bridge, Spinitalia S.r.l., Pomezia, Italy) to a computer to calculate distance and average speed detected by the GNSSr. Speed was calculated as distance over time (i.e., by horizontal position differentiation over time). We used only horizontal data.

Statistical analysis

Correspondences of direct and GNSSr assessments for distance and speed were evaluated. GNSSr data standard error of the estimate (SEE), coefficient of variation (CV), and mean bias were compared with direct assessment of both distance and speed. By means of a Bland-Altman plot [15], a comparison of residual versus predicted values was made. Analysis was performed for all runs together and independently for each of eight runs in order to evaluate whether differences existed among them. Measurement reliability was assessed by calculating the intra-class correlations coefficient (ICC). GNSSr distance assessments obtained with six soccer players were compared to each other to evaluate whether participants could influence assessment accuracy. Two-way ICC was used. The significance level was 0.05.

RESULTS

Participants performed eight COD runs each, for a total of 48 runs. Four runs were excluded due to wrong pathway (2), wrong time detection by photocell gate (1), and uncompleted run (1). Only 44 runs were considered for analyses. GNSSr horizontal dilution of precision (GDOP) was 0.97±0.14, and therefore almost ideal [16].

Concerning distance assessment validity, SEE and mean bias resulted in 0.52 m (0.73%) and 0.06 m, respectively (Table 1). The Bland-Altman plot showed a small difference between two assessments with 95% limits of agreement=±1.08 m. There was a trend for the error in distance measurement to decrease and become negative (and therefore underestimate) as running distance increased (Figure 2A).

TABLE 1.

Detailed results for each run type, with the relative SEE, CV, and mean bias for both distance and average speed assessments.

Test	Distance (m)	GNSS distance±SD (m)	CV of distance (%)	Distance Mean bias (m)	Average speed directly assessed (m·s-1)	Average GNSS speed (m·s-1)	SEE of speed (m·s-1)	CV of speed (%)	Average speed Mean bias (m·s-1)
Shuttle 20+20m×2 (3 COD)	80	80.16±0.24	0.24	+0.16	2.94±0.26	2.95±0.27	0.008	0.24	+0.006
Shuttle 15+15m×2 (3 COD)	60	59.79±0.26	0.37	-0.21	2.78±0.25	2.77±0.25	0.012	0.37	-0.009
Shuttle 10+10m×2 (3 COD)	40	40.05±0.24	0.39	+0.05	2.59±0.22	2.60±0.22	0.015	0.38	+0.004
Shuttle 7.5+7.5m×5 (9 COD)	75	75.82±0.63	0.93	+0.82	2.28±0.20	2.30±0.21	0.021	0.93	+0.025
Shuttle 5+5m×5 (9 COD)	50	50.38±0.71	1.06	+0.38	1.97±0.28	1.99±0.29	0.026	1.05	+0.017
Square (5+5+5+5m)×2 (7 COD)	40	39.58±0.45	1.05	-0.42	2.12±0.33	2.10±0.33	0.029	1.00	-0.023
Zigzag (5+5+5m)×2 (11 COD)	60	59.80±0.32	0.41	-0.20	2.08±0.36	2.07±0.36	0.012	0.42	-0.006
Cross-path (10+5+5+10+5+5m)x1 (5 COD)	40	39.91±0.49	0.79	+0.09	2.37±0.34	2.37±0.33	0.034	0.79	-0.007

SEE=Standard error of the estimate; CV=Coefficient of variation.

FIG. 2.

Bland-Altman plots of distance (A) and speed (B) assessments.

For average speed assessment validity, there was an SEE of 0.02 m·s^-1 (0.74%) between the two measurements and a mean bias of 0.001 m·s^-1. The Bland-Altman plot showed a small difference between two assessments with 95% limits of agreement=±0.046 m·s^-1. For average speed assessments, there was no clear trend in residuals from the Bland-Altman plot (Figure 2B). The ICC was 0.999.

DISCUSSION

The aim was to quantify 50-Hz GNSSr validity and reliability for assessing distance and average speed compared with direct measurements. The new low-end 50-Hz GNSSr provides valid and reliable results for above measurements assessed in young soccer players performing several standardized running actions within confined spaces, including one or more CODs. Covered distance, as measured by GNSSr, was similar to real distance for all actions. Maximal distance error detected was within the limit (5%) for GNSSr validity to be rated as good [2].

Validity results were more accurate than those of studies investigating validation of 1-15 Hz GNSSrs under similar conditions [17,18]. Jennings et al. [8] showed that SEE of 5 Hz GNSSr is ~10% for total distance, when compared with using a measuring tape and goniometer in tasks with tight and gradual COD. All similar studies [17,18] showed that GNSSr could underestimate distance and average or instantaneous speed.

By using 50-Hz GNSSr, covered distances were better than those reported in previous studies on 5-15 Hz GNSSrs [11,19,20]. Such a validity improvement might prompt use of a 50-Hz GNSSr for estimating both sprint mechanical properties [21] and metabolic power [22-26] in team sports. As an alternative to 50-Hz GNSSr, validity and usage improvements can be achieved by making use of a further couple of technologies. Some promising local positioning systems have already been shown to provide distance differences within 2% across movements compared with motion analysis measures [27]. In addition, some inertial measurement unit components (IMU) can improve measures’ validity and generic usage [8]. Some IMUs are already used together with GNSSrs or very high-frequency telemetry to track animal movements (e.g., in the dead-reckoning method [28]).

Reliability of 50-Hz GNSSr for distance measurement in actions of varying lengths within confined spaces and involving COD was also good, i.e., the ICC between test and retest was ≥0.9992 (for single and mean). Actions including COD resulted a major problem for reliability of GNSSr with 1-18 Hz [4,9,17,18]. Vickery et al. [9] showed that the CV of a 15-Hz GNSSr for distance can reach 17.0% in an action with COD, which is defined as poor reliability. Portas et al. [4] showed that in more complex scenarios of COD, such as repeated 180°-turn angles, reliability of 1-5 Hz decreases for distance (CV=7.71-6.11), while in the present study reliability of 50-Hz GNSSr results were unchanged with a COD of 180º- or 90º-turn angles.

Our main study finding, that 50-Hz GNSSr is more valid and reliable than previous lower sampling-frequency GNSSrs to measure distance and average speed, is true under the assumptions (which might not be the case) that 1) satellites’ signal sensor sensitivity and 2) GNSSrs’ working conditions (e.g., GDOP) were similar in our and previous studies. Another limitation of our study was that the number of subjects/runs was too small to draw definite conclusions. Additional experiments are highly recommended. A final limitation was that we did not measure the distance the player actually covered with a reference method. Namely, we assumed the player covered a distance corresponding to the nominal running path (i.e., we neglected that the player likely ran with curves and not along polygonal chains). Skilled players might minimise such a difference. Measuring a player’s actual distance could be done by using video-based kinematic methods [14,29,30] or outdoor motion analysis systems [31]. At least some of the acknowledged limitations also affected previous studies [3,4] sharing our approach.

CONCLUSIONS

In comparison with the 1-15 Hz GNSSr, the 50-Hz GNSSr provides valid and reliable results for distance and average speed assessments in young adults performing several standardized actions of differing lengths within confined spaces. Further research is needed to assess the validity, reliability, and convergent validity of this and/or similar devices during real match play. The 50-Hz GNSSr might even show lower start latency compared to lower sampling frequency receivers. That is also a matter for further research. This study provides athletes and coaches with a positive evaluation of a low-end (viz. relatively cheap) 50-Hz GNSSr for sport investigations. Investigations making use of the assessed device could regard, for example, rugby or field hockey.

Conflict of interest

The authors declare that they do not have any conflict of interest.