Reliability and validity of the Chronojump open-source jump mat system

Abstract

Vertical jump performance is a commonly used test to measure lower-limb muscle power that is carried out with several types of equipment. The aim of this study was to validate an open-source jump mat (Chronojump Boscosystems) against a proprietary jump mat (Globus Ergo Tester). Sixty-three active sportsmen (age 23.3 ± 2.4 years) completed 8 maximal-effort countermovement jumps (CMJ). The heights of the 504 CMJ were measured from the two jump mats simultaneously. Reliability was examined with intra-class correlation coefficients (ICC), paired samples t-tests, coefficient of variation (CV) and Cronbach’s α. Bivariate Pearson’s correlation coefficient (r) was used to examine validity. Effects were evaluated using non-clinical magnitude-based inference. There was almost perfect agreement between instruments (ICC = 0.999−1.000, most likely positive 100/0/0). Paired t-test showed a mean difference of 0.03 ± 0.21 cm (90% CI -0.04 − -0.01) between instruments (most likely trivial 0/100/0). Both instruments showed very good stability (α = 1.00, CV = 4.28±1.95%). The smallest worthwhile change and typical error values were 1.3 and 0.29 cm, respectively and therefore, the signal-to-noise ratio of both instruments was large: 4.5. Finally, almost perfect correlation between instruments was observed (r = 0.999, most likely positive 100/0/0). Chronojump can be regarded as a sensitive instrument to detect changes in jump height performance over the possible noise around the measure. The results supported the open-source jump mat to be a useful, valid and reliable, low-cost testing device to monitor variations in vertical jumping performance.

INTRODUCTION

Vertical jump performance is a commonly accepted measure of lower limb muscle power [1] and coordination of lower and upper extremities [2]. Sports professionals can study several neuromuscular and performance qualities of both athletic and non-athletic populations through jump height monitoring.

Although the number and typology of instruments and protocols to measure lower limb muscle power by means of vertical jump tests is considerable [3], including those based in software apps [4], they all can be classified within the following three main methods. Firstly, jump height can be calculated through numerical integration from vertical ground reaction forces with force plates [5]. Alternatively, accelerometric systems are portable devices that compute jump height through estimation of vertical forces exerted by body weight by means of acquisition of vertical accelerations [6, 7]. Secondly, jump height can be obtained as the difference between apex and baseline heights of the excursion of centre of gravity of the body in jump execution through motion capture [8, 9]. Thirdly, the time span between take-off and landing can be transformed into jump height through a basic kinematic equation with simple timekeeping devices that measure the athlete’s flight time [10].

These flight-time instruments are very popular with sports professionals due to their ease of use, portability and low cost compared to force plate and motion capture systems. In order to compute flight time from which to calculate the centre of gravity displacement during flight (jump height), such instruments rely on a precise selection of take-off and landing times. More recently, smartphone apps with high speed video cameras allow users to select take-off and landing frames by observation [11]. Alternatively, photocell mats can measure flight time automatically with parallel bars of emitting and receiving infrared beams that scan any interruption of the optical barrier in the event of athletes executing jumps [12, 13]. Finally, the most popular instrument in this group is the jump mat, also known as the contact mat, which comprises an electric switch in a mat that is pressure-activated due to body weight, and therefore flight time can be assessed automatically [14].

Due to their popularity among sport scientists and physical trainers, a number of companies have launched jump mat models. Recently, a jump mat system (Chronojump Boscosystems, Barcelona, Spain) was launched as an open-source (free software and open hardware) system. With such systems, both software (source code) and hardware (design files) are available to the public on official servers, emphasizing peer review and correction by the user community. As a result, hardware in open-source systems can be produced at low cost due to their non-profit nature and software is usually free for users. However, low-cost instruments can be perceived as a sign of poor quality, so sports professionals may refrain from using them.

The aim of the present study was to compare two jump mats instruments, a proprietary system (Globus Ergo Tester) and an open-source version (Chronojump Boscosystems), to test for any potential differences due to the nature of the equipment.

MATERIALS AND METHODS

Subjects

Sixty-three active sportsmen in various disciplines participated in the present study (mean age: 23.3 ± 2.4 years, body weight 73.9 ± 9.2 kg, body height 176.8 ± 6.6 cm). In order to avoid any interference with the experiment, subjects were told not to drink alcohol or caffeinated beverages during 24 h before testing. In addition, all jumps were performed by each participant at the same time of the day to ensure that no circadian variation was present.

Procedures

This observational study consisted of repeated measurements of maximum jump height on subjects during a single test session. Prior to jump executions, a standardized warm-up period of 5 minutes on a cycle ergometer (Cardgirus Pro Medical, Alava, Spain) set at 80-W power load and 60- to 65-rpm cadence, was performed. Then, subjects were instructed with familiarization jumps to achieve a proper jumping technique and keep balance while landing. Next, subjects performed eight repetitions of countermovement jumps with a rest period of one minute between consecutive jumps. For correct execution, subjects put hands on hips and flex their knees to an angle of 90 degrees and jump to a maximum height in a single movement. Knee right angle was monitored in the sagittal plane by real-time video digitizing software. Subjects were told to repeat any jump performed incorrectly so only successful trials were considered.

Data collection

Two jump mats were used in this experiment: a commercial model (Globus Ergo Tester, Codognè, Italy) and an open-source model (Chronojump Boscosystem, Barcelona, Spain). Each system was composed of an A2-size contact mat comprising two isolated electrical contacts that served as a pressure-dependent electrical switch which was in open-circuit configuration that closed when an athlete stands on the mat. A handheld (Globus) or PC-based (Chronojump) microcontroller was connected to the mat and computed flight time with 1-ms temporal resolution. The excursion of the centre of gravity during flight (jump height h) was computed by means of measured flight time (t) through the standardized equation h = t²·g/8, where g is the gravity acceleration (9.81 m/s²) [15].

All trials were measured simultaneously with both the commercial and the open-source model with only one mat, connected via an electric T-junction to both the handheld (Globus) and PC-based microcontroller (Chronojump). This single-mat arrangement allowed for a unique triggering signal, so any differences in jump height were due to the electronics and software of each jump-mat system.

Statistical analysis

Descriptive statistics (mean ± SD) were used to report characteristics of the 504 jumps collected. The intra-class correlation coefficient (ICC) (2,1), with 2-way random single measurements (absolute agreement), was used to examine the reliability of the open-source jump mat Chronojump in comparison with the proprietary system Globus. In addition, paired samples t-tests and mean differences with 90% CI were computed to compare outcomes from both jump mats. Also, Bland-Altman plots were depicted as representative of the agreement between the two instruments [16]. The identification of any potential systematic bias between jump mats can be performed with Bland-Altman plots through analysis of mean bias, standard deviations and regression lines of differences of instruments’ outcome pairs against mean values. The coefficient of variation (CV) and Cronbach’s α were computed to evaluate stability of the instruments when measuring the eight jump executions of each subject. Finally, bivariate Pearson’s product moment correlation coefficient (r) with 90% confidence intervals (CI) via bootstrapping was used to examine validity. All statistical analyses were performed using SPSS v.22 (IBM Corp, Armonk, NY).

Magnitude-based inference (MBI) [17] was used to evaluate the magnitude of the effects, since p-values in null-hypothesis significance testing are sample-size dependent. MBI is based on 90% confidence intervals (CI) representing the uncertainty in the true value and the smallest worthwhile change (SWC), depicting the minimum improvement likely to have a practical impact. SWC defines three ranges: substantially positive, trivial and substantially negative [18]. An inference is made from the chances that the true values were within these scales, calculated by comparison with the CI. If such a chance overlaps positive and negative values substantially, i.e. the true value could be in a positive and a negative sense larger than 5% simultaneously, the true value is deemed as unclear. Otherwise, it is clear and is estimated to have the magnitude of the observed value with the following probabilistic terms: possibly, 25–75%; likely, 75–95%; very likely, 95–99.5% and most likely, > 99.5%. In order to look for meaningful correlations, the SWC for all correlation outcomes was set to 0.5 [19] and 0.2 of the between-subjects standard deviation [18]. The usefulness of the instrument was assessed by comparing its associated noise or typical error of measurement and the SWC [20, 21]. Such mechanistic inferences and confidence limits were calculated through an available spreadsheet [22].

Ethics

The study protocol conformed to the guidelines of ethical principles of the Declaration of Helsinki. The aims and risks of the study were understood before the beginning of the study by all participants, who signed informed consent.

RESULTS

Globus and Chronojump mean jump heights were 31.95±6.53 and 31.97±6.52 cm, respectively for the whole 504 trials. There was almost perfect agreement between Globus and Chronojump (ICC = 0.999−1.000, most likely positive 100/0/0). Paired t-test showed a mean difference of 0.03 ± 0.21 cm (90% CI -0.04 − -0.01) between instruments (most likely trivial 0/100/0). Bland-Altman plots displaying limits of agreement for jump height between instruments showed that almost all outcomes lie within ± 1.96 standard deviations, and no correlation between paired difference and means was found (R² = 0) (Figure 1).

FIG. 1.

Both instruments showed very good reliability for the eight countermovement jumps of each subject (α = 1.00, CV = 4.28±1.95%). SWC and the typical error of measurement (TE) values were 1.3 and 0.29 cm, respectively and therefore, the signal-to-noise ratio of both instruments was large: 4.5. Finally, the bivariate Pearson’s product moment correlation coefficient showed an almost perfect correlation between Globus and Chronojump systems (r = 0.999, most likely positive 100/0/0), as shown in Figure 2.

FIG. 2.

DISCUSSION

The aim of this study was to analyse the reliability and validity of an open-source jump mat (Chronojump) to measure vertical jump height derived from flight time, by comparing their outcomes with a proprietary jump mat (Globus). The main finding of this study was that the open-source jump mat was found to be valid and reliable in measuring vertical jump heights.

The almost perfect agreement between these instruments, showing ICC values near unity, suggests that the reliability of Chronojump is most likely perfect. Also, narrow CIs demonstrated a high level of agreement between instruments. Bland-Altman plots verified that systematic bias between outcomes from both instruments was most likely trivial with a negligible overestimation of Chronojump of 0.03 cm. Moreover, the lack of linear correlation shown in Bland- Altman plots suggests that no differences between SD of measurements from Globus and Chronojump are expected, and that bias between methods is constant over the range of values [23]. However, the use of Bland-Altman plots is controversial for some authors due to artefactual bias using methods with substantial random error. Finally, the stability of Chronojump for measuring the eight jumps for each participant was excellent, as shown by low CV and high Cronbach’s α magnitudes.

The minimum improvement likely to have a practical impact, as a function of the between-subject SD, is around 1 cm. The typical error of measurement is very low for both instruments (0.29 cm), and therefore the ratio between the smallest practical change and the uncertainty of the measure (the so-called signal to noise ratio) is relatively large. In practice, the latter means that Chronojump is a sensitive tool to monitor changes in jump height magnitudes over the possible noise around the measure. Our results also showed that Chronojump would provide valid measurements of jump height as demonstrated by the most likely perfect bivariate Pearson’s product moment correlation coefficient (r = 0.999).

Previous studies comparing flight-time instruments with alternative, laboratory-based methods, gave larger magnitudes of error. Jump mats were reported to overestimate flight time or jump height with reference to the force platform in a range from approximately 1 cm in jump height [24], 1.7 cm in squat jump (SJ) and 2.8 cm in CMJ [25], 11.7 cm [26] to 16 cm [5]. Such overestimation is probably due to a minimum force necessary to activate the mechanical circuit of jump mats. When comparing force plates with accelerometry, disagreements ranged from 1.4−1.7 cm [27] to 3.6 cm [6]. With regards to 2D and 3D photogrammetry systems, height values can differ by up to 2.2 cm due to different knee and ankle angles during take-off and landing stages [8, 10]. Other reliability studies comparing jump mats and motion capture reported similar differences: 2.4 and 1.8 cm, respectively [27, 28]. However, such dis-agreements would also be present in other methods comparing exerted forces and centre-of-gravity displacements since body posture may not be completely extended during take-off and landing as a consequence of incorrect jump technique. For example, photocell mats have shown large systematic bias when compared to force plates in the range of -14.49 to -11.08 cm for SJ [29].

The results of this study are in accordance with other validation studies that used force platforms as the criterion. For example, the Just Jump System (Probotics, Inc., Huntsville, AL, USA) showed similar high correlation coefficients, such as r = 0.998 [5] or r = 0.994 [30], which were slightly reduced to r = 0.967 when the criterion was 3D motion capture [31]. Other proprietary jump mats reported similar strong correlations, such as r = 0.996 for FLS Jump Mat (Tyrone, Ireland) [24] or r = 0.995 for Elite Jump (S2 Sports, São Paulo, Brazil).

In spite of such moderate to large systematic bias between instruments, jump mats can be regarded as reliable and valid tools to monitor changes in jump height. However, absolute jump mat height measurements are not directly interchangeable with instruments measuring jump height based on principles other than pressure-activated timekeeping.

CONCLUSION

This study has shown that the open-source jump mat Chronojump is a valid and reliable instrument for measuring vertical jump height. Practitioners and researchers can be confident in using an open-source jump mat to monitor athletes’ vertical jump performance. The typical error of measurement is very low in comparison with the smallest worthwhile change, and therefore Chronojump can be regarded as a sensitive instrument to detect variations in jump height performance over the possible noise around the measure. We can conclude that Chronojump is a trustworthy tool that provides practitioners and researchers with accurate information with regards to changes in physical performance of athletes at a fraction of the cost of alternative proprietary systems.

Conflict of interest declaration

The authors have no conflicts of interest: there is no financial, commercial, legal, or professional relationship with other organizations, or with the people working with them, that could influence the research.