Technique to estimate human reaction time based on visual perception

Abstract

The design and implementation of a wearable system to estimate the human reaction time (HRT) to visual stimulus based on two identical wireless motion sensors are described. Each sensor incorporates a motion sensor (gyroscope), a processor and a transceiver operating at the industrial, scientific and medical frequency of 2.45 GHz. Relevant tests to estimate the HRT are performed in two different scenarios including simple and recognition tests for 90 pairs of measurements. The obtained results are compared with a computer-based system to determine the accuracy of the proposed system. The root mean square error, standard deviation error and mean error of the results are 2.88, 6.17 and 0.3 ms for simple test while for recognition test as low as 3.34, 7.83 and 0.35 ms, respectively. The outcomes of the HRT estimation tests confirm HRT can increase by 40–87% due to increased fatigue levels.

1. Introduction

Human reaction time (HRT) is the amount of time elapsed between the application of a stimulus and a person's response to it. Reaction time is the first step in the decision-making process. HRT is an important indicator of our information processing speed and the appropriate motor response [1]. Quick physical reaction to an external stimulus, or short HRT, can also be considered as a human physical performance indicator of well-being. In some professions such as airline pilots, athletes and drivers [2, 3], short HRT is critical to performance. HRT is usually obtained from three different tests [4] – simple [5], recognition [6] and choice [7].

In the simple HRT test, only one stimulus and one reaction are observed, for example, spot the dot or our reaction to sound. In the recognition test, the response time to some stimuli are measured while others are ignored. In contrast to both simple and recognition tests, in the choice tests, HRT is estimated from multiple stimuli and it is expected the reacting individual makes a choice from among multiple responses. Several systems were developed and reported to estimate the reaction time of drivers using virtual reality time [8], computer-based measurement of cognitive functioning [9] and wearable systems [10–12]. The latter ones usually incorporate motion sensors mounted on the reacting individual. Among the three HRT systems, virtual reality and computer-based systems are commonly used to estimate HRT. However, these two systems are bulky, expensive and are also inaccurate due to the influence of surrounding environment. Recently, wearable systems have emerged in a variety of applications to estimate the HRT values because they are readily available, inexpensive, lightweight and compact [13, 14].

In this Letter, we describe the implementation of a wearable system suitable for HRT estimation. The system depends on visual perception and uses the simple and recognition HRT tests. In comparison with the previously reported HRT measurement systems [8, 9], the system we developed is compact, low-cost, easy to wear and non-intrusive. Also, compared with the technique used in [10] which is a haptic-based approach, our proposed system is simple to use when estimating HRTs of professional drivers and pilots without physical or motion restrictions.

2. Proposed technique for HRT estimation

The proposed HRT technique is based on time-stamping of the stimulus and reaction in order to calculate the time difference. As shown in Fig. 1, this time difference is proportional to the HRT. The stimulus duration ($t_{\mathrm{sd}}$) is an important factor in estimating HRT which is required to last for a reasonable period, so that the stimulus can be sensed. Usually, the duration of the stimulus is shorter than the reaction period ($t_{\mathrm{rd}}$).

Fig. 1.

Interpretation of the HRT estimation by two departures of stimulus and reaction. HRT $\gg t_{sd}and{t}_{rd}$

To realise the proposed technique for HRT estimation, two identical motion sensors are used. One is worn by the stimulating individual on the wrist which is intended to time-stamp the stimulus action. The second one is set to record the reaction time and is placed on the head of the reacting individual (Fig. 2) to achieve a maximised sensitivity and high dynamic range in measurements. However, the sensor can arbitrarily be placed on different parts of the reacting individual's body. Owing to the rotational physical behaviour of the human body, particularly head with respect to the body, to stimulus actions, the choice of a gyroscope to detect short duration motions can enhance the accuracy in measuring the HRT when compared with using other types of motion sensors such as accelerometers or magnetometers.

Fig. 2.

Proposed HRT estimation systems including two wearable sensors and a central node coordinating the communication between the two sensors

3. Developed measurement system

The developed system for HRT measurement uses two wireless motion sensors (WMSs), in addition to a central node which is directly connected to a personal computer (PC) that is only used for collecting the measured data (see Fig. 2). As shown in Fig. 3, the essential components of each WMS, similar to a typical wireless sensor [15–17], are a microcontroller, an MicroElectroMechanicalSystems (MEMS) based motion sensor, a transceiver operating at the industrial, scientific and medical frequency of 2.45 GHz, in addition to a built-in meander planar inverted-F antenna (PIFA). This type of antenna is used for personal wireless communication devices due to its low profile, small size and moderate performance. The central node consists of a transceiver and an Recommended Standard (RS) 232 to universal serial bus port converter which is directly plugged into the PC (Fig. 4) to establish a data stream at an average sampling rate of 50 Hz. Also, the central node coordinates the access of the WMSs to the wireless medium to prevent data collision and subsequent loss of information [18, 19]. The hardware settings including the transceiver output power and data rate were set 0 dBm and 1.8 Mbps, respectively.

Fig. 3.

Developed WMS

a Schematic

b Top (the transceiver and PIFA antenna)

c Bottom (the microprocessor and motion sensor) photograph

Fig. 4.

Proposed central node

a Schematic

b Top (the transceiver and PIFA antenna) photograph

4. Results and discussion

To estimate the HRT in the simple and recognition tests, six volunteers participated in the tests, in addition to an individual who performed the stimulus action. First, the two WMSs were set to receive a stimulus simultaneously (i.e. a quick physical movement) while the gyroscopes outputs are recorded in order to achieve a time synchronisation between them. Then, the person performing the stimulus action wore one WMS (WMS #1) on the wrist, and the six other volunteers wore the second motion sensor (WMS #2) on their heads during each round of measurement, as shown in Fig. 2. For the simple test, three stimulus actions were performed over a 25 s period for a participant. In the simple test, which is accomplished based on a computer-centric approach as [20], the responses of the volunteer to a specific stimulus on the monitor screen (green colour) were recorded. Fig. 5a demonstrates one set of measured HRT values for the simple test obtained by using the proposed system, where the average of the obtained HRT is 267 ms, whereas the average of HRT was determined 265 ms for computer-centric approach.

Fig. 5.

HRT results for one of the volunteers

a Simple

b Recognition test

As the recognition test implies, the person performing the stimulus action holds two objects of different colours (e.g. blue and red coloured objects) and the volunteers are instructed to react to only one of them. Fig. 5b shows the results of the presented system for both stimulus action and reaction related to the volunteer in the recognition test that the average of the obtained HRT was ∼430 ms. Also, in the recognition test, which is established based on a computer as [21], the volunteer randomly sees two words Yes and No on the monitor screen and reacts only Yes word by using the Enter key. The computer-based system test is immediately performed after recording HRT of the developed system and the average of the HRT was obtained about 420 ms. As the results show, the reaction time increases in the recognition test compared with the simple one. Also, it was found that the reaction time is quicker for the last stimulus action (i.e. third one in Fig. 5) since the reacting individual becomes more alert or has had some practise from responding the previous two times when the stimulus was applied. These tests are repeated for any participant 15 times and also the measured data are recorded to investigate Bland–Altman analysis [22]. Fig. 6a illustrates a Bland–Altman plot of the HRT measurements obtained from the simple test for the 90 pairs of measurements in the presented system and computer-based system (gold standard). This plot is obtained by calculating difference and average of the HRT measurements from the outcome results. As the results reveal, the mean error is 0.35 ms with 95% limits of agreement −11.78 to 12.39 ms. Also, Bland–Altman plot of the HRT measurements from the recognition test for the 90 pairs of measurements has been shown in Fig. 6b. In this case, the mean error is 0.35 ms with 95% limits of agreement −14.99 to 15.69 ms. As it can be seen in the Bland–Altman plots, the averages of HRT measurements are 246 and 392 ms for simple and recognition tests in turn. Additionally, root mean square error (RMSE) and standard deviation error (STD) of the results were obtained 2.88, 6.17 ms for simple test and also 3.34, 7.83 ms for recognition test. As these values illustrated, very good agreement were obtained between the proposed system and the computer-based system.

Fig. 6.

Bland–Altman plots for HRT measurement

a Simple

b Recognition tests. The graphs show the agreement of 90 pairs of measurements from different participants. Mean error is depicted with slashed light grey and 95% limits are depicted with slashed dark grey lines

To investigate the most appropriate inertial sensor to use, we compared the gyroscope output with two other motion sensors — accelerometer and magnetometer. The results indicated that the gyroscope sensor can sense the movements in the action and reaction time with a better resolution than either the accelerometer or the magnetometer. For example, a simple test was recorded using the three motion sensors. The measured results in Fig. 7 illustrate the stimulus and reaction times of the gyroscope are before either that of the accelerometer and or the magnetometer. Moreover, the response time of the gyroscope ($t_{\mathrm{r},\mathrm{g}}$), accelerometer ($t_{\mathrm{r},\mathrm{a}}$) and magnetometer ($t_{\mathrm{r},\mathrm{m}}$) are 4.8, 4.9 and 7.3 ms, respectively [23]. These values prove that human body movements are mostly rotational movements; therefore, the gyroscope sensor can measure HRT with an improved accuracy compared with the other motion sensors.

Fig. 7.

Time-stamping for three motion sensors accelerometer, gyroscope and magnetometer in the action and reaction

To inspect the effects of fatigue on HRT, two sets of tests were performed one at the beginning of the day and the other one at the end of the day. As the results show in Fig. 8, the reaction time increases nearly 40–68 and 60–87%, respectively, for the simple and recognition tests, which is believed to be due to the fatigue level. For example, for volunteer #3, the effect of fatigue on their HRT values is ∼60 and 80% longer, respectively, for the simple and recognition tests. Fig. 8 shows, the effect of fatigue is rather considerable on the recognition test as the rate of correct answers declines substantially when compared with the simple test.

Fig. 8.

Result showing the influence of fatigue level on HRT obtained in the simple and recognition tests

To check whether fatigue is the most influential factor in increasing the HRT besides factors such as age, gender and type of stimulus, the volunteers were asked to do a light and intense physical activity. According to the results obtained, it was found that intense sport results in increased reaction times (simple test by 70–80% and recognition 83–92%). However, light physical activity has a positive effect on HRT, resulting in a shorter reaction time in both simple and recognition tests of ∼17–20 and ∼5–8%, respectively. The obtained results confirmed that light exercise can improve HRT, which are in good agreement with the results reported in [24]. Table 1 presents a comprehensive comparison between the proposed system and the approaches previously reported in the literatures [8–10].

Table 1.

Developed system in comparison with the other work

Reference	Device, s	Method		Advantage, s	Limitation, s
		Action	Reacting
[8]	advanced computer	computer	human	highly accurate	unavailability, bulky equipment, cost, complexity
[9]	accelerometer and PC	computer	human	accurate	cost, bulky equipment, unavailability
[10]	two gyroscopes	computer	human	accurate, low profile, availability and portability	high power, subjective application
Presented System	two gyroscopes	human	human	highly accurate, available and portable, measuring HRT based on visual perception, small-size (16.5 × 29.1 × 8.5 mm3), low-power (60 mw)	—

Bold represents the results of the “Presented System” described the paper. It is to highlight our work in comparison to the results in [8–10].

5. Conclusions

In this Letter, a comprehensive description of the design and realisation of a wearable system for estimating the HRT in both the simple and recognition tests was presented. The obtained results showed that the HRT values are 40–68 and 60–87% faster in the beginning compared with the end of the day, respectively, for simple and recognition tests. Additional tests revealed that fatigue level is a degrading factor that can significantly increase the HRT values. Also, an excellent agreement between the results from the developed system and a computer-based technique was obtained (RMSE < 4 and STD < 8 ms). However, in contrast to the computer-based technique whose applications are limited, the developed system can be used in a variety of applications such as determining the HRT of professional drivers, pilots and also athletes where highly rapid reactions are expected. Also, it can be mentioned that the proposed system can be used to quantify the human fatigue as the outcome results revealed.

6. Funding and declaration of interests

Conflict of interest: none declared.