Addressing the Need for Validation of a Touchscreen Psychomotor Vigilance Task:Important Considerations for Sleep Health Research

The psychomotor vigilance task (PVT) ¹ is a validated measure of sustained attention and vigilance and the gold standard for detecting objective neurobehavioral impairment due to sleep loss². The PVT has been used to assess impairment of vigilance and sustained attention from total and partial sleep deprivation^{3, 4} and circadian dysregulation⁵, and has also been used as a method for estimating daytime sleepiness ⁶. The standard PVT duration is 10 minutes in duration, although shorter versions have been validated for specific applications/platforms ^{7, 8}.

The standard laboratory-based PVT device (the PVT-192, marketed by Ambulatory Monitoring, Inc., Ardsley, NY) has several practical limitations. The test equipment can be expensive and includes outdated legacy electronic hardware. The PVT-192 equipment is not easily portable, which limits its utility in field research. To date, there have been several attempts to address portability. The Walter Reed Palm PVT, a software program for the legacy Palm Pilot device, successfully demonstrated the proof-of-concept of a more portable PVT test ⁹. However, Palm Pilots are no longer manufactured and the Palm PVT software is no longer supported. More recently, the PVT has been translated into a touchscreen interface by multiple groups^10-14, though validation studies are limited. Also, these applications are generally not designed to be easily disseminated in large research projects.

In addition to portability, implementation of a PVT in community health research needs to address a few operational issues. First, transitioning from analog buttons to digital touch screens affects PVT implementation. In the traditional PVT approach, an individual rests their finger on the button, pressing as soon as they see the stimulus producing an immediate reaction time. However, touchscreen functionality is such that the individual cannot rest their hand on the screen without triggering a response. Prior work by our group examined several approaches that could capture reaction time on a touchscreen, including goal-crossing, finger lift, finger tilt, touch down, and home button press¹⁴. We found users strongly preferred screen touch but this created a problem – the added time it takes a finger to traverse space and press the screen (and the lack of standardization of that distance) creates a systematic slowing in reaction time values¹⁴. Any touchscreen solution and analysis would need to account for this.

Second, various devices present hardware and software permissions issues that disallow unfettered access to a reliable timekeeping clock or the ability to ensure minimal delay between the software’s stimulus presentation and its display on screen. Further, these devices lack a reliable rate of polling touchscreen sensors (i.e., when a finger touches the screen, how quickly will the system register that contact), and some of these devices do not allow programs to access the screen refresh rate (how quickly an instruction to display something on the screen is actually displayed). All of these concerns lead to challenges in implementing a valid touchscreen PVT application. It is unclear (and likely variable across devices) the degree to which problems with touchscreen implementation of the PVT are related to systematically slower response times due to the finger traversing space vs touchscreen timing issues at the device level.

To address these concerns, we developed the PVT-Touch (http://pvttouch.com) for the Android operating system, the most common operating system among smartphones in the US ¹⁵. To validate the PVT-Touch relative to the gold-standard PVT-192, participants (N=60 adults age 25-60) were recruited through flyers and online advertisements for a study of health outcomes associated with habitual sleep duration. Subjects completed extensive screening questionnaires and were free from major comorbidities that impact sleep including depression or other psychiatric illness, insomnia disorder, or sleep apnea. This study was approved by the IRB of both the University of Pennsylvania and University of Arizona.

Participants were administered both the PVT-192 and PVT-Touch during a visit to the laboratory during the daytime. Each test was 10 minutes long. Tests were presented in a block-randomized order, with N=5 receiving the PVT-192 first and N=5 receiving the PVT-Touch first, for each block of 10 participants. Thus, the presentation order was randomized but balanced across the sample. After each test presentation, subjects were asked to rate their distraction level, and the test administrator recorded any distracting noises or other issues.

We conducted Pearson correlations for mean RT, median RT, and number of lapses across subjects. To determine precision of the estimate, the Pearson r values were converted to Fisher's z' values, which are normally distributed. Then 95% confidence intervals were computed around the transformed z' values, then converted back to Pearson r values. To determine whether mean values from tests were systematically higher or lower from each other, paired t-tests evaluated differences between the 2 variables.

Due to device errors (devices losing power during the test, devices failing to initialize properly, and corrupted data), N=44 subjects completed both the PVT-Touch and the PVT-192 in a randomized order (N=6 hardware or software problems with the PVT-Touch, N=10 PVT-192 device failure, and N=3 noted distractions; some with multiple problems). PVT-Touch and PVT-192 correlated highly despite differences in input style and device hardware (Table 1). For mean RT, median RT, and lapses, correlations were all r>0.8. The PVT-Touch systematically produced slower reaction time values, and a greater number responses that would be scored as lapses (Table 1). To adjust for this difference, a scaling factor of 0.75 was applied to PVT-Touch mean and median RT. This resulted in no differences between groups, with t-test p values of 0.9994 for mean and 0.9526 for median. For lapses, a scaling factor of 0.5 was applied, resulting in a t-test p-value of 0.8617.

Table 1.

Mean and Median Reaction Time and Attentional Lapses using the PVT-Touch and the PVT-192

	PVT-192		PVT-Touch		PVT-Touch (Corrected)		Correlation
	Mean	SD	Mean	SD	Mean	SD	r	95% CI	p
Mean Reaction Time (ms)	284.10	54.83	378.81	98.13	284.09	73.57	0.83	(0.71, 0.91)	<0.0001
Median Reaction Time (ms)	258.75	39.00	344.50	68.44	258.17	51.34	0.82	(0.70, 0.90)	<0.0001
Lapses (>500ms)	4.32	5.85	8.20	11.48	4.10	5.74	0.83	(0.71, 0.91)	<0.0001

In-laboratory PVT scores have been shown to depend on time of day, duration of wakefulness, and can be influenced by a large number of environmental factors. Importantly, apps such as the PVT-Touch are typically not intended to replace the in-laboratory versions of the PVT, but rather to expand the research applicability of the PVT to more generalizable contexts than can be reasonably assessed with standard PVT approaches. With this in mind, a minimal validation of these apps may not necessitate exhaustively comparing results to the standard PVT across a wide range of circadian, sleep homeostatic, and environmental conditions – rather, a simple comparison of the two devices, demonstrating some degree of relative equivalence, should be sufficient as a first step. As more data becomes available, it can further clarify the degree to which a novel implementation of a PVT could generate useful data. Many commercially-available PVT products, though, fail to even meet this minimal standard. Either they are validated on different platforms (e.g., validation on desktop despite the commercial product being on a touchscreen device) or frequently not validated at all. It may be difficult to discern which PVT products are validated on the platform on which they are offered (and to what degree), whether they are validated on an alternate platform, or whether they are validated at all. This information should be transparently available.

In conclusion, consumer sleep technologies are changing the manner in which the public and the sleep research community conceptualize sleep assessment in the “real world¹⁶.” From the consumer perspective, these technologies allow longitudinal objective assessment of sleep in a typical sleep environment and individual assessment of the impact of behavior change on aspects of sleep. PVT-Touch contributes to this paradigm shift in sleep assessment by providing the opportunity to demonstrate changes in psychomotor vigilance linked to habitual sleep behaviors and can help guide individuals on their journey to optimal sleep health. The development and rigorous validation of touchscreen PVTs will enhance the sleep research community. First, it will reduce costs by increasing portability and accessibility. Second, sleep health researchers could use this type of software in large samples at low cost. Third, the general public, especially those with consumer wearables, may employ the software to assess their own performance. The sleep research community should work together to develop and welcome a new validated touchscreen based PVT. Preliminary results from PVT-Touch that our team has developed shows promise in achieving that goal.