Incremental Validity of Useful Field of View Subtests for the Prediction of Instrumental Activities of Daily Living

Abstract

Introduction

The Useful Field of View Test (UFOV^®) is a cognitive measure that predicts older adults’ ability to perform a range of everyday activities. However, little is known about the individual contribution of each subtest to these predictions and the underlying constructs of UFOV performance remain a topic of debate.

Method

We investigated the incremental validity of UFOV subtests for the prediction of Instrumental Activities of Daily Living (IADL) performance in two independent datasets, the SKILL (n = 828) and ACTIVE (n = 2426) studies. We, then, explored the cognitive and visual abilities assessed by UFOV using a range of neuropsychological and vision tests administered in the SKILL study.

Results

In the four subtest variant of UFOV, only subtests 2 and 3 consistently made independent contributions to the prediction of IADL performance across three different behavioral measures. In all cases, the incremental validity of UFOV subtests 1 and 4 was negligible. Furthermore, we found that UFOV was related to processing speed, general non-speeded cognition, and visual function; the omission of subtests 1 and 4 from the test score did not affect these associations.

Conclusions

UFOV subtests 1 and 4 appear to be of limited use to predict IADL and possibly other everyday activities. Future experimental research should investigate if shortening the UFOV by omitting these subtests is a reliable and valid assessment approach.

The Useful Field of View Test (UFOV^®; Visual Awareness, Inc., Punta Gorda, FL) is a measure of older adults’ cognitive function. In the past 20 years, UFOV has received considerable attention because the test scores predict older adults’ ability to perform a range of everyday activities including driving (Clay et al., 2005; Owsley, Sloane, McGwin, & Ball, 2002). The test score is the sum of four subtest scores, but little is known about the incremental validity of each subtest. The present study investigated the incremental validity of UFOV subtests for the prediction of Timed Instrumental Activities of Daily Living (IADL), Observed Tasks of Daily Living (OTDL) and the Everyday Problems Test (EPT)—standardized tests of older adults’ ability to perform everyday tasks. We also explored the underlying constructs of UFOV performance.

Since it was first conceived (Ball, Beard, Roenker, Miller, & Griggs, 1988; Sekuler & Ball, 1986) several variants of the UFOV paradigm have been proposed (for an overview see Edwards et al., 2005a). In its current form, the UFOV test consists of four increasingly difficult subtests designed to assess visual processing speed under demands of focused attention (subtest 1), divided attention (subtest 2), and selective attention (subtests 3 and 4; Figure 1). In subtest 1 (processing speed), a pictogram of a car or truck is briefly displayed in a central fixation box. In this two-alternative, forced-choice task, participants are asked to identify the object. In subtest 2 (divided attention), along with the car or truck presented in the central fixation box, a second pictogram of a car is simultaneously presented at one of eight peripheral locations. Following the identification of the central object, participants indicate at which location the peripheral car was displayed. For subtest 3 (selective attention), 47 distractor triangles of the same size, contrast, and luminance as the peripheral target are added to the stimulus display in three concentric circles. Finally, subtest 4 (selective attention two) is a variant of subtest 3 with two pictograms presented in the central fixation box. Participants indicate whether the two objects inside the fixation box are identical or different, and again locate the peripheral car. Across subtests, performance is scored as the stimuli display time at which test takers achieve an accuracy threshold of 75%. Response times are not recorded.

Figure 1.

Illustration of the UFOV procedure. The stimulus displays for subtests 1-4 are shown in the box at the top. The subtests are designed to assess visual processing speed under conditions of focused attention (subtest 1), divided attention (subtest 2), and selective attention tasks (subtests 3 and 4). Presentation times vary between 16.67 ms and 500 ms in steps of 16.67 ms (1 frame on a 60 Hz computer screen). In subtests 1-3 participants give a non-speeded response identifying the briefly presented object in the center as either car or truck; in subtest 4 participants determine whether the two objects in the center are identical or different. For subtests 2-4, following the identification task, participants indicate at which location the peripheral car was displayed. A trial is marked as correct if both responses are correct; response times are not recorded. Dashed lines indicate trial elements with variable duration.

UFOV is best known for its ability to predict older adults’ driving performance (Anstey & Wood, 2011; Clay et al., 2005; Wood, Chaparro, Lacherez, & Hickson, 2012), including the probability of incurring motor vehicle collisions (Ball, Owsley, Sloane, Roenker, & Bruni, 1993; Goode et al., 1998; e.g., Owsley et al., 1998; Rubin et al., 2007). Some consider UFOV to be a gold standard of cognitive driving tests (e.g., Classen, Wang, Crizzle, Winter, & Lanford, 2013; Weaver, Bédard, McAuliffe, & Parkkari, 2009). Thus, UFOV commonly serves as a proxy for driving ability among older adults and in clinical populations such as Parkinson's disease, multiple sclerosis, Alzheimer's disease, and HIV (Classen et al., 2009, 2013; Crizzle, Classen, & Uc, 2012; Duchek, Hunt, Ball, Buckles, & Morris, 1998; Fisk, Novack, Mennemeier, & Roenker, 2002; Goode et al., 1998; Weaver et al., 2009; Wood et al., 2012).

Besides its well-known ability to predict driving abilities, UFOV is more generally related to older adults’ everyday abilities. IADL competence is most commonly assessed using self-report or proxy-reports by questionnaire (e.g., Lawton & Brody, 1969), but recent work has emphasized objective, performance-based measures, such as Timed IADL (Owsley, McGwin, Sloane, Stalvey, & Wells, 2001; Owsley et al., 2002), OTDL (Diehl et al., 2005), or EPT (Willis & Marsiske, 1993). While their administration is more convenient, self-report measures can be affected by memory biases and socially desirable responding. Thus, performance-based assessment of everyday abilities is merited. For a review of measures of IADL, please see Moore, Palmer, Patterson, & Jeste (2007).

Our study examined three different performance-based IADL measures. The Timed IADL test consists of five tasks intended to assess efficiency and accuracy of functional performance in the domains of telephone communication, finances, nutrition, shopping, and medication usage (Owsley et al., 2001, 2002). Test takers are, for example, asked to make change or read directions on medication containers under time constraints. The OTDL and EPT tests were designed to assess everyday problem solving across IADL domains without strict time constraints. OTDL consists of nine tasks from the domains of medication usage, telephone communication, and financial management, such as balancing a checkbook or completing a patient record form. Similarly, in EPT test takers solve problems in seven domains, such as food preparation, medication usage, and telephone communication. All three measures have been previously validated, established as reliable, and are widely used to assess IADL competence (Burton, Strauss, Bunce, Hunter, & Hultsch, 2009; Edwards et al., 2005b; Farley, Higginson, Sherman, & MacDougall, 2011; Goverover, Genova, Hillary, & DeLuca, 2007; Jobe et al., 2001; Pressler et al., 2011; Yam & Marsiske, 2013). Performance across these three IADL tests is intercorrelated (e.g., Sartori et al., 2012), and each is associated with UFOV performance (Owsley et al., 2002; Sartori et al., 2012).

UFOV is also of interest in that performance can be improved by training, and such gains transfer to improved IADL (e.g., Ball et al., 2002; Sekuler & Ball, 1986; Wolinsky, Vander Weg, Howren, Jones, & Dotson, 2013). UFOV training performance gains endure longitudinally, with training effects still evident at 10 years (Rebok et al., 2014). What is more, large-scale clinical trials have demonstrated that UFOV training, also known as speed of processing training, immediately transfers to improved Timed IADL performance (Edwards et al., 2002, 2005b) and longitudinally results in less difficulties with IADL, as indicated by self-report (Rebok et al., 2014; Wolinsky, Vander Weg, Howren, Jones, & Dotson, 2015). UFOV training also has lasting positive effects on several indices of driving safety and mobility (Ball, Edwards, Ross, & McGwin, 2010; Edwards, Delahunt, & Mahncke, 2009; Edwards et al., 2009; Roenker, Cissell, Ball, Wadley, & Edwards, 2003). These findings are remarkable because very few cognitive training protocols exhibit far transfer effects to improved everyday functional performance (Kelly et al., 2014; Rabipour & Raz, 2012).

Validity of UFOV subtests

While the validity of UFOV seems well established, few studies have examined the validity of the individual subtests. It appears that not all UFOV subtests contribute equally to the prediction of driving performance. Hoffman, McDowd, Atchley, & Dubinsky (2005) found that only subtest 2 contributed uniquely to simulated driving performance, but Wood et al. (2012) found that subtests 2 and 3 were both related to on-road driving performance. Owsley et al. (1998) indicated that only subtest 2 is diagnostic of the risk of traffic accidents. Based on these findings, some researchers have used subtest 2 as a short version of UFOV (e.g., Anstey, Horswill, Wood, & Hatherly, 2012; Ball et al., 2006; Friedman, McGwin, Ball, & Owsley, 2013; Owsley, McGwin, & Searcey, 2013; Vance et al., 2006). On the other hand, in a prospective study, Rubin et al. (2007) found that both subtest 1 (HR = 1.27) and 2 (HR = 1.47) were individually related to motor vehicle collision involvement among older adults. Thus, the results regarding the association of UFOV subtests and motor vehicle collisions are mixed. All of the described studies administered the three-subtest version of UFOV, that is, almost no research on the incremental validity of subtest 4 is available. Because we are unaware of any study that has examined the association of UFOV subtests and the ability to perform IADL, we tested the incremental validity of each subtest for the prediction of Timed IADL, OTDL, and EPT performance.

In a recent study, Edwards, Ruva, O'Brien, Haley, & Lister (2012) found that the transfer of UFOV training to enhanced Timed IADL performance was fully mediated by improvements on subtests 2—subtest 1, 3, and 4 did not contribute to training transfer. One possible explanation for this finding is that only subtest 2 is related to IADL and, therefore, training gains on the other subtests cannot benefit IADL performance. It is also possible that subtests 1, 3, and 4 are related to IADL, but only due to variance they share with subtest 2. Finally, subtests 1 and 4 may have little predictive validity because of limited variance, in that most participants perform at ceiling or floor, respectively (Edwards et al., 2006). In each case, we would expect that these subtests possess no incremental validity for the prediction of IADL. If, however, any of the other UFOV subtests are independently related to IADL, it would appear that the additional abilities needed for these tasks are not substantially improved by UFOV training or that these improvements do not transfer to IADL. Thus, our investigation of the incremental validity of UFOV subtests for the prediction of IADL will help to better understand the effects of UFOV training. We hypothesized that subtest 1 and 4 would not make independent contributions to IADL performance.

The current study

The present study investigated the individual contributions of UFOV subtests to the prediction of IADL and examined the underlying constructs to UFOV performance. As outlined above, a detailed investigation of the association between UFOV and measures of IADL performance will (1) possibly suggest a shortened version of the testing protocol, (2) help to identify cognitive abilities that are essential to everyday tasks, and (3) help to understand the effects of cognitive training based on the UFOV paradigm. We analyzed data from the Advanced Cognitive Training for Independent and Vital Elderly (ACTIVE, Jobe et al., 2001; Tennstedt et al., 2010) and the Staying Keen In Later Life (SKILL, Edwards et al., 2005b) studies. To test the incremental validity of each subtest, we performed regression analyses predicting Timed IADL, OTDL, or EPT performance. The SKILL and ACTIVE studies administered UFOV and Timed IADL to large samples of older adults, which allowed us to test our hypothesis and perform a direct replication of our results in an independent dataset. OTDL and EPT were administered only in the ACTIVE study. For all analyses, we expected that subtests 1 and 4 possess limited incremental validity. Finally, the neuropsychological tests administered in the SKILL study, enabled us to explore the underlying constructs of UFOV performance, and to test if shortening the UFOV protocol affected its association with cognitive and visual abilities.

Method

Participants

Prior to all analyses, we excluded participants who exhibited either cognitive decline, as indicated by MMSE scores of 24 or less (M. F. Folstein, Folstein, & McHugh, 1975), and a maximum score of 500 ms on all of the UFOV subtests, or poor vision (acuity worse than 20/60) during the baseline assessment. Refer to Table 3 for demographic characteristics of the analyzed datasets. We analyzed the data from the baseline measurement phase of the SKILL and ACTIVE studies to test our hypothesis. We further examined test-retest reliabilities using data from baseline and immediate post-test assessments of the ACTIVE study no-contact control group.

Table 3.

Demographic characteristics of the SKILL and ACTIVE samples.

		SKILL		ACTIVE
Variable	Level	n	%	n	%
Gender	Female	482	58.2	1822	75.1
	Male	346	41.8	604	24.9
Ethnicity	African American	73	8.8	593	24.4
	Caucasian	749	90.5	1791	73.8
	Other	4	0.5	42	1.7
Education	High school	771	93.1	2199	90.6
	College education	346	41.8	830	34.2

Variable	M	SD	Min	Max	M	SD	Min	Max
Age (years)	73.2	5.8	62	96	73.4	5.8	65	94
Education (years)	14.1	2.7	6	20	13.7	2.7	5	20
Far acuity (score)	72.0	11.2	11.5	90	73.5	11.4	31.8	90
MMSE (score)	28.5	1.3	25	30	27.8	1.5	25	30

Note. Other ethinicities include Hispanic, Asian, Native American, and unspecified categories. Far visual acuity was assessed using a Early Treatment Diabetic Retinopathy Study (ETDRS) letter chart and we report Mini Mental State Exam (MMSE; M. F. Folstein, Folstein, & McHugh, 1975) as an inidicator of cognitive impairment.

SKILL study

The SKILL study examined the relationship of cognitive, sensory and functional abilities among older adults. 1052 community-dwelling adults 60 years and older from Bowling Green, Kentucky, Birmingham, Alabama, and surrounding areas were screened for inclusion (Clay et al., 2009; Wood et al., 2005). Edwards et al. (2005b) provide a detailed description of the SKILL study protocol. The dataset consisted of 894 participants who completed the baseline measurement phase. After excluding participants missing data on UFOV subtests or Timed IADL (n = 17), with a MMSE score of 24 or less (n = 37), a near visual acuity worse than 20/60 (n = 13), a maximum score of 500 ms on all UFOV subtests (n = 2) and one influential case (see Data analysis section), the analyzed sample consisted of 828 older adults, 92.6% of the original dataset. Participants with corrected near visual acuity worse than 20/60 were excluded because Timed IADL performance depends on intact vision (Owsley et al., 2002).

ACTIVE study

The ACTIVE study tested the effects of three types of cognitive training on everyday activities that rely on cognition; Jobe et al. (2001) provide a detailed description of the ACTIVE study protocol. Community-dwelling adults 65 years and older were recruited at the University of Alabama at Birmingham, Hebrew Rehabilitation Center for the Aged, Indiana University, Johns Hopkins University, Pennsylvania State University, and Wayne State University. We retrieved the data from the Inter-university Consortium for Political and Social Research (Tennstedt et al., 2010). The dataset consisted of 2802 participants. To obtain a sample comparable to the SKILL dataset, we again excluded participants missing data on UFOV subtests, Timed IADL, OTDL, or EPT (n = 55), with MMSE scores of 24 or less (n = 333) or a maximum score of 500 ms on all UFOV subtests (n = 4). All participants had a far visual acuity of 20/60 or better; no information on near visual acuity data was available. The remaining sample consisted of 2426 older adults (86.6% of the original dataset).

Measures

Our primary variables of interest were UFOV and performance-based IADL assessments. UFOV and Timed IADL were administered in both the SKILL and ACTIVE studies, allowing for a direct replication of our results. OTDL and EPT were only administered in the ACTIVE study.

UFOV

Both the SKILL and ACTIVE studies employed the four subtests version of UFOV, Figure 1. Performance on all subtests is scored as the display presentation time at which the test taker achieves an accuracy of 75% as determined by a double-staircase procedure. Response times are not recorded and do not affect scoring. In all subtests, participants view briefly presented white stimuli on a black background. In subtest 1, a pictogram of a car or truck (1.91 × 1.43°) is briefly displayed in the fixation box (2.86 × 2.86°); presentation times vary between 16.67 ms and 500 ms in steps of 16.67 ms (1 frame on a 60 Hz computer screen). The screen is then masked by a random dot pattern. At the end of a trial, participants identify the object as either car or truck in a non-speeded, two-alternative, forced choice task. In subtest 2, a second pictogram of a car is simultaneously presented at one of eight radial positions at approximately 10.47° from the central object. Following the identification of the central object, participants indicate at which location the peripheral car was displayed. A trial is marked as accurate only if both responses are correct. In subtest 3, 47 downward pointing triangular distractors of the same size, contrast, and luminance as the targets are added to the briefly flashed stimulus display in three concentric circles, with the furthest being 10.47° from the central target. Finally, subtest 4 is a variant of subtest 3 with two pictograms presented in the central fixation box. Participants determine whether the two objects in the central fixation box are identical or different and locate the peripheral car. The final test score is a sum of the four subtest scores. Depending on the method of response collection (mouse or touch screen) and the retest interval, the reliability of the subtests has been reported to range from .51 to .78 for subtest 1, from .58 to .81 for subtest 2, and from .71 to .85 for subtest 3 (Edwards et al., 2005a). Estimates of the retest-reliability of the sum score of all three tests range from .72 to .88; no estimates for subtest 4 are available.

Instrumental Activities of Daily Living

Timed IADL

Timed IADL assesses speeded functional abilities and consists of five tasks from the domains of telephone communication, finances, nutrition, shopping, and medication usage (Owsley et al., 2001, 2002). Test takers are asked to look up a phone number, make change, read directions on food cans or medication containers, and find items on a food shelf in a standardized setting. If a task is completed within the preset time limit, the response time is recorded, otherwise the task is terminated. As per standard procedure, a time penalty is added for minor errors (see Table 1 in Owsley et al., 2002). The penalty is defined as 1 SD of response times from all participants who completed the task without an error. In case of a major error, or if the task is not completed within the time limit, the maximum time allotted to the task is recorded. Thus, per standard procedure, Timed IADL scores are the mean of z-standardized response times on all five tasks, penalized for errors. The test-retest-reliability of Timed IADL is r = .85

Table 1.

Intercorrelations of UFOV subtests 1-4, UFOV sum score, Timed IADL, OTDL, and EPT in the SKILL and ACTIVE studies.

	UFOV
Measure	Subtest 1	Subtest 2	Subtest 3	Subtest 4	Sum score	Timed IADL	OTDL	EPT
UFOV subtest 1	—	.44	.30	.15	.49	.23	−.17	−.21
UFOV subtest 2	.41	—	.55	.29	.81	.35	−.33	−.39
UFOV subtest 3	.27	.57	—	.52	.89	.34	−.33	−.33
UFOV subtest 4	.13	.31	.50	—	.65	.22	−.22	−.20
UFOV sum score	.42	.81	.89	.68	—	.39	−.37	−.40
Timed IADL	.21	.38	.37	.22	.42	—	−.43	−.48
OTDL							—	.55
EPT								—

Note. Intercorrelations for the ACTIVE study (n = 2426) are presented above the diagonal and intercorrelations for the SKILL study (n = 828) are presented below the diagonal. All correlations are significant after Bonferroni correction for multiple tests within each study. UFOV = Useful Field of View; Timed IADL = Timed Instrumental Activities of Daily Living; OTDL = Observed Tasks of Daily Living; EPT = Everyday Problems Test.

OTDL

The revised OTDL was designed to assess everyday problem solving (Diehl et al., 2005) and is available at http://www.webcitation.org/6ar3eSSlg. The measure consists of three tasks from each of the domains of medication usage, telephone communication, and financial management. Among other things, the tasks require test takers to follow directions on medication containers, complete a patient record form, look up phone numbers, make change, and balance a check book. Each task consists of several subtasks yielding 28 items in total. Subtasks are scored as either correct or incorrect and the overall number of correct subtasks comprises the test score ranging from 0 to 28. The internal consistency of OTDL is high, α = .82.

EPT

EPT was designed to assess everyday problem solving (Willis & Marsiske, 1993). A shortened version was administered in the ACTIVE study. Test takers received 14 items from the seven domains of food preparation, medication usage and health behaviors, and telephone communication, shopping and consumerism, financial management, housekeeping, and transportation. Each item is a piece of everyday information printed in high-contrast large print, such as medication labels or transportation schedules, accompanied by two questions that need to be answered. The test is paper-pencil based and the time to respond was unlimited–response times were not recorded. We analyzed the number of correctly answered questions ranging from 0 to 28. According to the manual, EPT has a high internal consistency (α = .88) and test-retest-reliability, r = .93.

To explore the underlying constructs of UFOV performance, we analyzed neuropsychological measures of processing speed, executive functioning, short-term memory, and intelligence, as well as vision tests to explore the underlying constructs of UFOV performance. We also examined associations of UFOV subtests with these measures to explore if shortening the UFOV protocol may affect its association with cognitive and visual abilities. We included all available measures of visual and cognitive functioning from the SKILL study, with the exception of the Hopkins Verbal Learning Task, as it did not load substantially on any of the extracted factors (see Data analysis section).

Processing speed

Letter and Pattern Comparison

Letter and Pattern Comparison (Salthouse & Babcock, 1991) are paper-and-pencil tasks that measure processing speed with a selective attention component (Lustig, Hasher, & Tonev, 2006). Participants compare two columns of letter sets (or line patterns) row by row to determine if the stimuli are identical or different. The complexity of the letter sets and line patterns in each column varies in three levels. In the Letter Comparison Test, participants receive two pages with 34 letter pairs consisting of either three, six, or nine letters each; similarly, in the Pattern Comparison Test, they receive four pages with 16 pattern pairs consisting of either three, six, or nine line segments each. Participants are given 20 seconds for every section and the total number of correct responses comprises the test score.

Shape Color Size

Shape Color Size (SCS) is a computer-based attention switching speed of processing task adapted from L. T. Miller & Vernon (1997). Participants determine if the shape, size, or color of two objects is the same or different. On each trial, the to-be-compared attribute (e.g., shape) is indicated by the corresponding word in the center of the screen. Participants are asked to respond as quickly as possible using a computer mouse. The test was administered in four blocks. In two blocks the comparison task varied from trial to trial; in the other two blocks the comparisons of shape, size, and color were grouped. We used the mean reaction time of all correct responses as score for our analyses.

Digit Symbol Substitution

Like Letter and Pattern Comparison, the Digit Symbol Substitution Test (DSS, Wechsler, 1981) from the the Wechsler Adult Intelligence Scale-Revised is a paper-and-pencil measure of processing speed with a selective attention component (Lustig et al., 2006). In a grid layout, participants substitute symbols for the numbers 1-9 according to a key at the top of the page. The symbols are filled into 93 empty squares beneath the numbers and participants complete as many substitutions as possible in 90 seconds. To account for individual difference in motor speed, participants perform an analogous task in which they simply copy symbols (Digit Symbol Copy Test, DSC). The average times to correctly substitute and copy one item are the test scores. For our analyses we used the DSS score adjusted for the time required to write the symbol by subtracting the DSC score. Scores reflect symbols correctly substituted per second.

Trail Making Test Part A

The Trail Making Test Part A (TMT-A) is a paper-and-pencil measure of processing speed (Reitan & Wolfson, 1993). Participants connect a series of 25 numbers in sequential order and time to complete (in seconds) is recorded.

General non-speeded cognition

Trail Making Test Part B

Trail Making Test Part B (TMT-B) is commonly used to assess executive functioning (Reitan & Wolfson, 1993). Similar to TMT-A, participants connect 25 letters and numbers in alternating sequential order (e.g., 1-A-2-B) and the time required to complete the task is recorded. In the SKILL study the time for TMT-B was limited to a maximum of 480 seconds.

Stroop task

The adaptation of the original Stroop task (Trenerry, Crosson, DeBoe, & Leber, 1989) used in this study consisted of (1) reading a series of color words (e.g., red, green, or blue), (2) naming the color of patches displayed on the screen, and (3) naming the incongruent font color of color words (e.g., the word red displayed in a green font). In all three tasks, the response times and the number of uncorrected mistakes were measured. As done previously (Edwards et al., 2005b; Wood et al., 2005), we analyzed the font color naming response times adjusted by the response times in the color naming task and an added a penalty for the number of uncorrected mistakes in the font color naming task.

Digit and Spatial Span

In the Digit Span and Spatial Span tasks from the Wechsler Memory Scale-III (Wechsler, 1987), participants are asked to recall progressively longer lists of items to gauge their memory (i.e., numbers for Digit Span; position sequences for Spatial Span). In Digit Span the strings of numbers were presented aurally and participants attempted to verbally repeat them. In Spatial Span, participants viewed a white board that contained 10 blue pegs. The tester touched the pegs in a particular order and participants attempted to repeat the sequences by touching the pegs in the same order. We used the number of correctly repeated sequences as test scores for our analyses.

Matrix Reasoning

The Matrix Reasoning subtest from the Wechsler Abbreviated Scale of Intelligence (WASI; Wechsler, 1999) measures non-verbal fluid intelligence. Participants view matrices of related pictures with one empty cell. They choose one out of five pictures that would best complete the matrix. We converted the raw scores to z scores for our analyses.

Vocabulary

The Vocabulary subtest of the WASI is a measure of verbal crystallized intelligence (Wechsler, 1999) in which participants define words. For our analyses, we converted raw scores to z scores.

Visual function

Participants wore corrective lenses for all vision tests, if applicable.

Far visual acuity

A GoodLite Model 600A illuminated cabinet with a standard Early Treatment Diabetic Retinopathy Study (ETDRS) letter chart was used to measure far visual acuity binocularly at a distance of three meters (Good-Lite Co., Elgin, IL). Ten points were given for each of nine lines read correctly. Total ETDRS scores could range from 0 (a Snellen score of 20/125) to 90 (a Snellen score of 20/16).

Near visual acuity

The Lighthouse Near Visual Acuity Modified ETDRS letter chart was used to assess near visual acuity binocularly at a distance of 40 cm per standard procedure (Lighthouse International, New York, NY). For consistency with our far visual acuity scores, recorded log Minimum Angle Resolvable (logMAR) was transformed to ETDRS scores for analysis (Gregori, Feuer, & Rosenfeld, 2010).

Contrast sensitivity

The Pelli-Robson chart was used to measure contrast sensitivity binocularly, per standard procedure (Pelli & Robson, 1988). This chart includes two sets of three letters in each of eight rows on a white background. Letters gradually decrease in contrast from left to right and top to bottom. Scores range from 0.00 to 2.25 log₁₀, with lower scores indicating worse contrast sensitivity.

Data analysis

To test our hypothesis about the incremental validity of the UFOV subtests, we performed regression analyses predicting Timed IADL, OTDL, or EPT performance. Because Timed IADL scores deviated substantially from a normal distribution (SKILL: skewness = 2.23, kurtosis = 7.73; ACTIVE: skewness = 2.35, kurtosis = 9.17, Figure 2) and because of heteroscedasticity in ordinary least squares regression (SKILL: χ²[4, n = 828] = 33.61, p < .001; ACTIVE: χ²[4, n = 2426] = 60.72, p < .001, studentized Breusch-Pagan test), we performed nonparametric bootstrap linear regression analyses based on 10,000 bootstrap samples and computed bias-corrected and accelerated (BCa) 95%-confidence intervals. We inspected jackknife-after-bootstrap plots (Efron, 1992) and identified one influential case in the SKILL dataset, 0.12% of the analyzed sample. We report the results excluding the influential case. However, including it did not affect our conclusions.

Figure 2.

Violinplots of measures in the SKILL (top row) and ACTIVE studies (bottom row). UFOV scores are the display presentation time at which test takers achieves an accuracy of 75%. Timed IADL scores are the mean of z-standardized response times on all five tasks penalized for errors. The black lines represent Tukey boxplots giving median, interquartile range (IQR), and most extreme values within 1.5 IQR of the upper and lower quartile. Grey planes represent kernel densities of each distribution. UFOV = Useful Field of View; Timed IADL = Timed Instrumental Activities of Daily Living; OTDL = Observed Tasks of Daily Living; EPT = Everyday Problems Test.

The distribution of OTDL and EPT scores did not deviate substantially from normality (OTDL: skewness = −0.38, kurtosis = 0.01; EPT: skewness = −0.63, kurtosis = −0.06). Formal testing indicated a violation of homoscedasticity (OTDL: χ²[4, n = 2426] = 23.33, p < .001; EPT: χ²[4, n = 2426] = 24.56, p < .001, studentized Breusch-Pagan test) but inspection of residual plots showed that these were minor violations. We, thus, report ordinary least squares linear regression analyses of OTDL and EPT scores¹.

To explore cognitive and visual abilities related to UFOV we performed a factor analysis with oblimin-rotation on the neuropsychological and vision tests administered in the SKILL study. Prior to all analyses, we reversed scores of measures where lower scores represented better performance and replaced missing data in eight variables with median values. To select the number of factors to retain, we consulted the scree plot, Kaiser-Guttman-criterion and the results from a modified parallel analysis (Glorfeld, 1995; Horn, 1965), using the 99th percentile to estimate bias. We derived factor scores for each participant, using the regression method and tested the association between UFOV and the derived factor scores in a linear regression analysis. Finally, to establish if omitting subtests from the UFOV sum score changed the associations, we compared nested regression models. We predicted performance on the shortened protocol using the regression equation for the complete sum scores and compared the results to three regression models, each of which estimated one of the regression coefficients for the three factors from the data.

The α-level for all analyses was .05. We performed all analyses in R (3.2.2, R Core Team, 2015)², analysis scripts are available at http://osf.io/grke7.

Results

We first tested our hypothesis that UFOV subtests 1 and 4 possess no incremental validity for the prediction of Timed IADL in the SKILL dataset and then replicated our analyses in the ACTIVE dataset to confirm our findings. We then tested the same hypothesis for OTDL and EPT in the ACTIVE dataset. Intercorrelations of UFOV subtests, sum score, Timed IADL, OTDL, and EPT are reported in 1.

Prediction of IADL

Timed IADL

As Figure 2 shows, the distribution of sum scores of all four UFOV subtests approximated a normal distribution, skewness = 0.74, kurtosis = 0.32. The distributions of individual subtest scores, however, deviated substantially from normality, skewness = −1.18 - 6.71, kurtosis = −0.88 - 54.26. Furthermore, inspection of the distributions revealed substantial range restriction in the scores of subtest 1 and 2, with 91.43% and 31.88% of scores falling below 32 ms. Subtest 4 scores amassed at the upper boundary between 484 and 500 ms including 53.86% of scores. Subtest 3 exhibited the least range restriction with 0% and 20.17% of scores at the lower and upper boundary, respectively. The results of the bootstrap regression analysis are given in Table 4. In line with our hypotheses, UFOV subtests 1 and 4 did not contribute independently to the prediction of Timed IADL. Controlling for age and years of education did not affect these results. Addition of subtests 1 and 4 only minimally increased the explained variance in Timed IADL by $\Delta R_{bc}^2=.01,[.00,.01]$ Subtest 2 $(b_{bc}^{\ast }=.23,95\%\mathrm{CI}[.11,.34])$ and subtest 3 were predictive of Timed IADL performance, $b_{bc}^{\ast }=.21,95\%\mathrm{CI}[.13,.30]$. Variance inflation factors indicated acceptable levels of multicollinearity (all VIF ≤ 1.80). Controlling for age and years of education did not affect these results.

Table 4.

Results from nonparametric bootstrap linear regression analyses predicting Timed IADL for the SKILL and ACTIVE dataset.

Study	Predictor	$b_{bc}^{\ast }$	95% CI	p	VIFbc	95% CI
SKILL	Subtest 1	.05	[−.03, .15]	.210	1.21	[1.13, 1.32]
	Subtest 2	.23	[.11, .34]	< .001	1.65	[1.50, 1.83]
	Subtest 3	.21	[.13, .30]	< .001	1.80	[1.66, 1.94]
	Subtest 4	.04	[−.03, .09]	.238	1.34	[1.27, 1.41]
ACTIVE	Subtest 1	.09	[.03, .14]	.001	1.25	[1.20, 1.30]
	Subtest 2	.20	[.14, .25]	< .001	1.62	[1.54, 1.71]
	Subtest 3	.18	[.13, .23]	< .001	1.80	[1.72, 1.87]
	Subtest 4	.05	[.01, .09]	.013	1.37	[1.33, 1.41]

Note. In the SKILL study (n = 828) UFOV subtests accounted for $R_{bc}^2=.19$, 95% CI [.13, .24], p < .001, of Timed IADL variance and in the ACTIVE study (n = 2426) for $R_{bc}^2=.16$, 95% CI [.13, .19], p< .001, of Timed IADL variance. All estimates are based on 10,000 bootstrap samples, bias-corrected (bc) and we computed 95%-confidence intervals using the bias-corrected and accelerated (BCa) method. Regression coefficients $b_{bc}^{\ast }$ are standardized and we report Variance Inflation Factors (VIF) as measures of collinearity. UFOV = Useful Field of View; Timed IADL = Timed Instrumental Activities of Daily Living.

The distributions of UFOV and Timed IADL scores in the ACTIVE study closely correspond to those in the SKILL study. UFOV sum scores approximated a normal distribution, skewness = 0.53, kurtosis = −0.05, but again the distributions of individual subtest scores deviated from normality, skewness = −1.48 - 4.99, kurtosis = −1.22 - 31.07, Figure 2. Scores of subtests 1 and 2 were restricted in range, with 84.95% and 21.93% of scores falling below 32 ms, while scores of subtest 3 and subtest 4 amassed at the upper boundary between 484 and 500 ms, 26.46% and 58.37% of scores, respectively. We performed a direct replication of our analysis of the SKILL dataset in the ACTIVE dataset and were able to confirm our findings. Although the regression coefficients of UFOV subtests 1 and 4 were larger than zero when tested in this three-fold larger sample, both subtests’ contributions to the prediction of Timed IADL were negligible. When we controlled for age and years of education, subtest 4 no longer contributed significantly to the prediction, $b_{bc}^{\ast }=.03,95\%\mathrm{CI}[-.01,.07]$, p = .082; all other results were unchanged. Addition of subtests 1 and 4 only minimally increased the explained variance in Timed IADL by $\Delta R_{bc}^2=.01,[.00,.02]$. Only subtest 2, $b_{bc}^{\ast }=.20,95\%\mathrm{CI}[.14,.25]$, and subtest 3, $b_{bc}^{\ast }=.18$, were independently associated with Timed IADL performance to an extent that is practically relevant, Table 4. Variance inflation factors indicated acceptable levels of multicollinearity (all VIF ≤ 1.80).

OTDL & EPT

The results from the analysis of OTDL and EPT confirm our previous results, Table 5. Even in the large ACTIVE dataset, UFOV subtest 1 did not contribute significantly to the prediction of OTDL and the contribution of subtest 4 was again small. Using only subtests 2 and 3 to predict OTDL decreased the explained variance by a minimal amount, ΔR² < .01, 95% CI [.00, .01], F(2, 2421) = 5.59, p = .004. Similarly, the contribution of UFOV subtest 1 for the prediction of EPT was negligible and in this case subtest 4 did not contribute significantly. When we controlled for age and years of education, subtest 1 no longer contributed significantly to the prediction, b* = −.04, 95% CI [−.07, .00], t(2417) = −1.81, p = .071; all other results were unchanged. Again, the decrease in explained variance due to using only subtests 2 and 3 to predict EPT was practically irrelevant, ΔR² < .01, 95% CI [.00, .01], F(2, 2421) = 3.36, p = .035. Controlling for age and years of education did not affect these results.

Table 5.

Results from linear regression analysis predicting OTDL and EPT in the ACTIVE dataset.

IADL measure	Predictor	b*	95% CI	t(2421)	p	VIF
OTDL	Subtest 1	−.02	[–0.06, 0.02]	−0.86	.389	1.24
	Subtest 2	−.20	[–0.25, –0.15]	−8.34	< .001	1.62
	Subtest 3	−.18	[–0.23, –0.13]	−7.01	< .001	1.79
	Subtest 4	−.07	[–0.11, –0.03]	−3.24	.001	1.37
EPT	Subtest 1	−.04	[–0.08, 0.00]	−2.09	.037	1.24
	Subtest 2	−.28	[–0.33, –0.24]	−12.08	< .001	1.62
	Subtest 3	−.14	[–0.19, –0.09]	−5.74	< .001	1.79
	Subtest 4	−.03	[–0.08, 0.01]	−1.55	.121	1.37

Note. UFOV subtests accounted for R² = .14, 90% CI [0.12, 0.16] of the variance in OTDL (F[4, 2421] = 100.33, p < .001) and for R² = .17, 90% CI [0.15, 0.19] of the variance in EPT (F[4, 2421] = 127.07, p < .001) scores. UFOV = Useful Field of View; OTDL = Observed Tasks of Daily Living; EPT = Everyday Problems Test.

In summary, we found that only UFOV subtests 2 and 3 consistently predicted performance on three different behavioral measures of IADL competence. If subtest 1 or 4 were independently related to an outcome measure, their explanatory value was limited. Taken together these results suggest that it may be feasible to omit subtests 1 and 4 from the UFOV protocol if the goal is to predict IADL performance.

Effects of shortening UFOV

Test-retest reliability

We also tested if the omission of UFOV subtests 1 and 4 affected the test's reliability. We determined test-retest reliabilities by correlating test performance from baseline visits and immediate post-training assessments of the ACTIVE control group (n = 548). The test-retest interval was 5-6 weeks. UFOV sum scores in the analyzed sample exhibited a reliability of r = .80, 95% CI [.77, .83] (t[546] = 31.12, p < .001), which corresponds to previous estimates. The estimated reliability of subtest 2 and 3 sum scores was r = .79, 95% CI [.75, .82] (t[547] = 29.78, p < .001) and did not differ from the complete test score reliability, Δr = .01, 95% CI [.00, .03], z = 1.59, p = .112. Thus, the test-retest reliabilities of the sum score of all four UFOV subtest and the sum score of subtests 2 and 3 may be comparable.

Association with cognitive and visual abilities

To explore cognitive and visual abilities related to UFOV performance, we tested the association of UFOV scores to three common factors from the neuropsychological and vision tests administered in the SKILL study. Refer to Table 6 for descriptive statistics for all neuropsychologcial and vision tests and to Table 2 for intercorrelations of all measures. The data were suitable for factor analysis, Kaiser-Meyer-Olkin MSA = .86 (all individual MSA ≥ .68), det = .01, Bartlett's χ²(91, n = 828) = 3532.51, p < .001. The scree plot (available at http://osf.io/grke7), and the Kaiser-Guttman-criterion suggested three factors, and parallel analysis yielded five meaningful factors. Factor 4, e.g., was characterized by high loadings of TMT-A and TMT-B and was highly correlated with Factor 1, which was characterized by high loadings of Letter Comparison, Pattern Comparison, and SCS. We, thus, retained three factors for further analysis, Table 7.

Table 6.

Descriptive statistics for the neuropsychological and vision tests administered in the SKILL study.

Measure	M	SD	Min	Max	Missing (%)
Letter comparison (ncorrect)	40.10	9.02	13.00	69.00	0.36
Pattern comparison (ncorrect)	27.46	6.13	4.00	45.00	0.12
Shape Color Size RT (ms)	1392.41	243.03	692.25	2175.61	2.17
Digit Symbol Substitution (ncorrect/s)	1.29	0.61	−0.55	8.05	0.00
Trail Making Test-A (s)	42.56	17.62	14.61	205.53	0.12
Trail Making Test-B (s)	129.15	91.29	36.60	480.00	0.12
Stroop task (score)	31.28	16.25	3.12	147.56	3.99
WMS-III Digit Span (ncorrect)	9.66	2.12	4.00	16.00	0.00
WMS-III Spatial Span (ncorrect)	7.51	1.73	0.00	13.00	0.00
WASI Matrix reasoning (z score)	0.34	1.15	−2.20	2.80	0.60
WASI Vocabulary (z score)	0.66	0.82	−3.00	2.60	0.36
Far visual acuity (score)	72.01	11.24	11.50	90.00	0.00
Near visual acuity (score)	81.93	6.26	61.00	90.00	0.00
Contrast sensitivity (log10)	1.69	0.15	0.90	1.95	0.00

Note. We analyzed response times (RT) in the Shape Color Size task. Digit and Spatial Span were taken from the Wechsler Memory Scale-III (WMS-III, Wechsler, 1987); Matrix reasoning and Vocabulary tasks were taken from the Wechsler Abbreviated Scale of Intelligence (WASI; Wechsler, 1999).

Table 2.

Intercorrelations of UFOV subtests, UFOV sum score, and neuropsychological and vision tests administered in the SKILL study.

Variable	1.	2.	3.	4.	5.	6.	7.	8.	9.	10.	11.	12.	13.	14.
1. Letter comparison	—
2. Pattern comparison	.70	—
3. Shape Color Size RT	−.47	−.56	—
4. Digit Symbol Substitution	−.52	−.45	.39	—
5. Trail Making Test-A	−.48	−.53	.39	.49	—
6. Trail Making Test-B	−.40	−.44	.40	.49	.53	—
7. Stroop task	−.36	−.38	.39	.43	.27	.39	—
8. WMS-III Digit Span	.31	.27	−.27	−.26	−.16	−.27	−.26	—
9. WMS-III Spatial Span	.19	.22	−.25	−.25	−.28	−.29	−.20	.29	—
10. WASI Matrix reasoning	.26	.32	−.23	−.33	−.26	−.36	−.29	.27	.27	—
11. WASI Vocabulary	.32	.36	−.26	−.39	−.27	−.39	−.33	.31	.21	.55	—
12. Far visual acuity	.19	.23	−.15	−.16	−.21	−.21	−.12	.05	.07	.09	.07	—
13. Near visual acuity	.14	.15	−.13	−.06	−.16	−.15	−.10	.05	.11	−.02	−.01	.50	—
14. Contrast sensitivity	.24	.24	−.18	−.21	−.27	−.19	−.14	.08	.11	.06	.06	.41	.28	—
15. UFOV sum score	−.46	−.48	.41	.46	.45	.43	.38	−.27	−.28	−.28	−.24	−.29	−.22	−.38
16. Shortend UFOV sum score	−.43	−.46	.41	.45	.44	.42	.39	−.26	−.28	−.29	−.24	−.30	−.23	−.38

Note. Correlations are based on pairwise complete observations. UFOV = Useful Field of View; RT = response time; WMS-III = Wechsler Memory Scale-III (WMS-III, Wechsler, 1987); WASI = Wechsler Abbreviated Scale of Intelligence (WASI; Wechsler, 1999)

Table 7.

Factor loadings, communalities, and explained variance for the factor analysis of the neuropsychological and vision tests administered in the SKILL study.

Measure	Factor 1	Factor 2	Factor 3	Communality
Letter comparison (ncorrect)	.85	−.05	−.03	.66
Pattern comparison (ncorrect)	.84	−.01	.01	.70
Shape Color Size RT (ms)	.58	.05	.01	.38
Digit Symbol Substitution (ncorrect/s)	.44	.30	.03	.44
Trail Making Test-A (s)	.52	.13	.11	.42
Trail Making Test-B (s)	.30	.39	.13	.43
Stroop task (score)	.29	.26	.03	.24
WMS-III Digit Span (ncorrect)	.18	.32	−.03	.19
WMS-III Spatial Span (ncorrect)	.10	.31	.06	.16
WASI Matrix reasoning (z score)	−.06	.75	.00	.51
WASI Vocabulary (z score)	.04	.71	−.04	.52
Far visual acuity (score)	−.04	.03	.80	.63
Near visual acuity (score)	.01	−.08	.63	.39
Contrast sensitivity (log10)	.17	−.05	.45	.27
Explained variance	.20	.13	.10
Cummulative explained variance	.20	.33	.42

Note. We used oblique oblimin-rotation yielding moderately correlated factors, r₁₂ = .55, r₁₃ = .34, r₂₃ = .14. We analyzed response times (RT) in the Shape Color Size task. Digit and Spatial Span were taken from the Wechsler Memory Scale-III (WMS-III, Wechsler, 1987); Matrix reasoning and Vocabulary tasks were taken from the Wechsler Abbreviated Scale of Intelligence (WASI; Wechsler, 1999).

For the interpretation of our factors, we considered factor loadings of .20 or greater (Stevens, 2002). Measures of processing speed, such as Pattern Comparison, SCS, or DSS, loaded on the first factor and largely exhibited negligible loadings on the other factors; only DSS and TMT-A additionally loaded on the second factor. We, thus, interpreted the first factor, which explained 19.85% of the variance, as representing processing speed. The second factor accounted for 13.07% of the variance and was characterized by loadings of WASI Matrix reasoning and Vocabulary, memory tasks, and measures of executive function, such as the Stroop task and TMT-B. Because most of these tasks require little speeded cognition, we interpreted the second factor as general non-speed cognitive ability. The Stroop task, WMS-III Spatial Span, and TMT-B also loaded on the first factor, which is plausible given the speed components of these tasks. The third factor explained 9.56% of the variance in the analyzed measures and represented basic visual functions: It was only correlated with near and far visual acuity and contrast sensitivity. Overall, the three extracted factors explained 42.48% of the variance in the neuropsychological and vision tests.

We used factor scores of all three factors to predict the sum score of UFOV subtests 1-4. UFOV performance was related to all three extracted factors. Processing speed was the strongest predictor of the UFOV sum score, b* = −.64, 95% CI [−.70, −.58], t(824) = −20.76, p < .001. General non-speeded cognition (b* = −.40, 95% CI [−.46, −.35], t[824] = −13.54, p < .001) and basic visual functions (b* = −.37, 95% CI [−.43, −.32], t[824] = −13.24, p < .001) also predicted UFOV performance, albeit the association was weaker, all VIF ≤ 1.28. The three factors explained R² = .39, 90% CI [0.35, 0.43] of variance in UFOV performance (F[3, 824] = 175.92, p < .001). When added individually to the regression model, age (ΔR² = .04, 95% CI [.02, .07], F[1, 824] = 62.22, p < .001) and Timed IADL (ΔR² = .01, 95% CI [.00, .02], F[1, 824] = 10.00, p = .002) accounted for little additional variance. Hence, after accounting for processing speed, general non-speeded cognition, and visual functions, age³ and Timed IADL shared little unique variance with UFOV.

The sum score of all four UFOV subtests and the sum score of subtests 2 and 3 were highly correlated, r = .96, 95% CI [.96, .97], t(826) = 103.16, p < .001 and r = .97, 95% CI [.97, .97], t(2424) = 192.04, p < .001 in SKILL and ACTIVE, respectively. To corroborate that omitting UFOV subtests 1 and 4 from the testing protocol does not change the assessment of the psychological constructs, we predicted scores from the shortened UFOV protocol consisting of subtests 2 and 3 using the same three factors. Processing speed still was most strongly associated with the sum score of UFOV subtests 2 and 3, b* = −.61, 95% CI [−.68, −.55], t[824] = −19.79, p < .001. General non-speeded cognition (b* = −.41, 95% CI [−.47, −.35], t[824] = −13.57, p < .001) and basic visual functions (b* = −.38, 95% CI [−.44, −.33], t[824] = −13.50, p < .001) exhibited weaker associations, all VIF ≤ 1.28. The three factors explained R² = .38, 90% CI [0.34, 0.42] of the variance of subtests 2 and 3 (F[3, 824] = 167.55, p < .001) and, again, age (ΔR² = .04, 95% CI [.02, .06], F[1, 824] = 51.84, p < .001) and Timed IADL (ΔR² = .01, 95% CI [.00, .03], F[1, 824] = 12.99, p < .001) accounted for little additional variance. Note the close correspondence of all estimates between the complete and the shortened UFOV protocol.

To formally test the differences between the coefficients, we compared nested regression models. First, we predicted the sum score of the shortened UFOV protocol using the coefficients estimated for the complete protocol. This constrained model fit the data well, χ²(3, n = 828) = 1.28, p = .735. We then compared the constrained model to three models in which either the association with processing speed, general non-speeded cognition, or visual function, was estimated freely. The differences between fixed and freely estimated coefficients were not significant; free estimation of the regression coefficients did not improve model fits, χ²(1, n = 828) = 1.11, p = .291; χ²(1 n = 828) = 0.29, p = .592; χ²(1, n = 828) = 0.38, p = .538, respectively. These comparisons confirmed that the associations between UFOV and processing speed, general non-speeded cognition, and visual function were not affected by omitting subtests 1 and 4.

Discussion

The results of our analyses support the a priori proposed hypotheses: Across three behavioral measures of IADL competence, we found either no support for a unique association of UFOV subtests 1 and 4 with IADL performance or the strength of the association was negligible. The correlation of UFOV and IADL performance essentially appears to be a function of test takers performance on UFOV subtests 2 and 3. In an exploratory analysis, we found that UFOV performance is associated with processing speed, but also with general cognitive abilities and visual function. These associations were not affected by omitting subtests 1 and 4.

An efficient administration of cognitive tests is important in applied settings, such as in departments of motor vehicles (Ball et al., 2006). Time saved by omitting subtests 1 and 4 could be used to reduce costs or administer a greater variety of measures. Subtest 2 appears to be the best predictor of incurring motor vehicle collisions (Owsley et al., 1998; Rubin et al., 2007) and only subtest 2 and 3 appear to be related to on-road driving performance (Wood et al., 2012). Our findings extend previous research by confirming that these results generalize to the prediction of IADL performance. Moreover, our study is the first to test the incremental validity of subtest 4 and to examine the effect of shortening the UFOV protocol on test-retest-reliability. These results indicate it may be feasible to shorten UFOV by omitting subtests 1 and 4.

Our proposed explanation for our findings is statistical in nature, namely that the limited variance in the subtest 1 and 4 scores precludes strong associations with the outcome variables of interest. The obvious question that follows is whether the incremental validity of UFOV subtests 1 and 4 can be improved by adapting the procedure in a way that reduces ceiling and floor effects but preserves the subtests’ cognitive demands (e.g., by not truncating scores at 500 ms). Resolving this question is an interesting direction for future research and could guide further development of the UFOV testing and training procedures.

Because UFOV is often referred to as a measure of processing speed (e.g., Lunsman et al., 2008), the second aim of this investigation was to identify cognitive abilities assessed by UFOV that may be essential to everyday tasks. We found that UFOV—the four-subtest version and the shortened protocol—is related to speeded cognition (i.e., Letter and Pattern Comparison, DSS, SCS response times), general non-speeded cognition (i.e., Digit and Spatial Span, Matrix reasoning, Vocabulary, and TMT-B), and basic visual function (near visual acuity, far visual acuity, and contrast sensitivity). Employing a similar approach, Anstey et al. (2012) related UFOV subtest 2 to cognitive and visual factors extracted from a range of measures. They found that an executive speed factor, characterized by high loadings of assessments such as visual search, number comparison, and a digit symbol matching task, explained a large portion of the variance in subtest 2 performance. Spatial abilities, characterized by high loadings of mental rotation tasks, Trail-Making-Test B, and memory (e.g., Digit Span) were also related to subtest 2. While the executive speed factor of Anstey et al. appears to correspond to our speeded cognition, our general cognition factor shares properties of both the spatial abilities and the working memory factor of Anstey and colleagues. Matas, Nettelbeck, & Burns (2014) similarly found that UFOV subtest 2 is related to change detection and processing speed, while subtest 3 is related to crowding, contrast sensitivity and processing speed. Thus, our findings are in line with recent reports that UFOV not only taps processing speed but is also related to more general aspects of cognition and vision. However, commonly used measures of processing speed, such as DSS or Letter and Pattern Comparison, also require selective attention (Lustig et al., 2006). Because no distinct measures of selective attention were administered in the SKILL study, we are open to the possibility that our processing speed factor may in part reflect participants’ ability to suppress distracting information. UFOV taps selective attention in that subtests 3 and 4 employ visual distractors. Thus, future research should examine the relative importance of processing speed and selective attention for UFOV performance.

Some studies have found that UFOV subtest 1 does not predict driving outcomes (Hoffman et al., 2005; Owsley et al., 1998; Wood et al., 2012, but see Rubin et al., 2007). However to date, no published study has evaluated the incremental validity of subtest 4 for driving outcomes. We speculate that our findings regarding the expendability of subtest 4 are also relevant to the prediction of driving safety. First, the high correlations between UFOV sum scores and the sum score of subtests 2 and 3 indicates that subtests 1 and 4 add little unique variance. Their limited predictive value, thus, is likely to be a property of the subtests and not specific to prediction of IADL performance. Second, we found that UFOV is related to processing speed, general non-speeded cognition, and basic visual function and that omission of subtests 1 and 4 did not affect these associations. These three factors explained most of the common variance between UFOV and Timed IADL. If the association between UFOV and Timed IADL originates from a similar set of functional abilities that are also relevant for safe driving our findings are relevant to the prediction of driving performance. While visual acuity and contrast sensitivity are typically only weakly related to driving performance (Owsley & McGwin, 2010), speeded and general cognitive abilities are more strongly related to driving safety (e.g., Anstey, Wood, Lord, & Walker, 2005). Taken together these findings indicate that subtest 4 may also be expendable when predicting driving safety.

Our results also inform the study of UFOV training benefits. As previously mentioned, Edwards et al. (2012) demonstrated that improvements in Timed IADL after UFOV training were fully mediated by performance gains on subtest 2. A possible explanation for this finding is that only subtest 2 is related to Timed IADL or that subtests 1, 3, and 4 are related to Timed IADL but only due to variance they share with subtest 2. However, the results of our analyses show that this explanation is insufficient. Subtests 1 and 4 are not independently associated with Timed IADL performance and it is, therefore, not surprising that training gains on these subtests are unrelated to improvements in IADL. However, performance on UFOV subtest 3 contributed as much unique variance to the association with Timed IADL performance as did subtest 2. Subtest 2 requires processing of two concurrently presented targets and was designed to assess divided attention. Subtest 3 is a variant of subtest 2 with visual distractors. It is, thus, assumed that the key difference between the tasks is that subtest 3 requires selective attention while subtest 2 does not. If we accept this assumption, it appears that UFOV training may not substantially improve selective attention or the improvements do not transfer to Timed IADL.

A final point that warrants discussion is the strength of the association between UFOV and Timed IADL. In a considerably smaller sample, Owsley et al. (2002) reported that UFOV explained roughly a third of the variance in and Timed IADL performance, but we found a considerably weaker association. Owsley et al. (2002) converted UFOV scores to decile ranks, but we found that our estimates of the correlations were unaffected by this transformation. Reassured by the converging estimate from the SKILL study, we conclude that the correlation between UFOV and Timed IADL was previously overestimated.

Limitations

Note that like all previous studies examining the UFOV subtests (Edwards et al., 2006; Hoffman et al., 2005; Lunsman et al., 2008; Owsley et al., 1998; Rubin et al., 2007; Wood et al., 2012), we investigated the incremental validity of each subtest as part of the full UFOV procedure, that is, participants completed every subtest. Hence, all available evidence rests on the assumption that the correlation of performance on UFOV subtests with other variables is not affected by previous subtests. This assumption should be tested experimentally in future research.

Our inclusion criteria restricted our sample to community-dwelling older adults. Thus, further research is needed to test if our findings generalize to the clinical populations to which UFOV has been administered, such as older adults suffering from mild cognitive impairment, Alzheimer's disease, or HIV.

Some standard measures of processing speed require selective attention in that stimuli need to be ignored while completing the task at hand (Lustig et al., 2006). We were unable to quantify the extent to which UFOV performance requires selective attention in addition to processing speed and divided attention. As our findings also raise the question if UFOV training can substantially improve selective attention, future studies need to scrutinize the contribution of different attentional processes to UFOV performance in general and to performance on individual subtests specifically. Relating UFOV to established and theory-based measures of attention, such as the partial report paradigm of the Theory of Visual Attention (TVA; Bundesen, 1990) or the Attentional Network Test (ANT; Fan, McCandliss, Sommer, Raz, & Posner, 2002) is a promising direction for future research (see Weaver et al., 2009). Furthermore, we would like to see future studies on UFOV training include established measures of attentional processes to quantify transfer effects.

Conclusions

We investigated the individual contributions of UFOV subtests to the prediction of IADL performance. Our findings (1) show that only subtests 2 and 3 are related to Timed IADL performance, (2) confirm that UFOV not only taps processing speed, but is also related to more general non-speeded cognitive abilities and vision, and (3) suggest that older adults’ improvements in Timed IADL performance after UFOV training may not be attributable to improved selective attention.