Automated iPad-Based Gaze Detection in the NIH Baby Toolbox Norming Study

Abstract

The NIH Baby Toolbox^® (“Baby Toolbox”) is a comprehensive battery of assessments administered on an iPad and designed for children aged 1–42 months. It is also the first normed developmental assessment to include gaze paradigms, which are common in developmental research and provide a non-invasive technique for measuring attention and cognitive processes without requiring a coordinated motor or verbal response. Baby Toolbox gaze paradigms collect real-time gaze data using augmented reality features native to the iPad itself. Here, we describe our approach to developing and validating this gaze technology for the Baby Toolbox norming study. In a sample of 1079 infants between 6 and 23 months, we found strong calibration success across ages, sex, race/ethnicity, parental education and eye-color. Some group differences emerged for race/ethnicity and parental education. Reliability results against hand-coding also demonstrated moderate to strong reliability, although results varied substantially by individual. Overall, these results highlight the Baby Toolbox gaze technology as an exciting and novel approach to developmental assessment.

1.1. Introduction

For over half a century, developmental researchers have used infant eye-gaze behavior to uncover how young children perceive and process the world (See Aslin, 2007; Gredebäck et al., 2009; Oakes, 2012). Patterns of visual attention and eye movement have served as powerful indicators of a wide range of cognitive and perceptual processes including language comprehension, (Cooper & Aslin, 1990; Fernald et al., 2008), understanding of causal events (Oakes, 2010; Sobel & Kirkham, 2006), numerical reasoning (Libertus & Brannon, 2010; Starr et al., 2013) and social preferences (Fawcett & Liszkowski, 2012). Eye-tracking has also been used for classifying cognitive processes, such as attention, processing speed and memory (Richmond & Nelson, 2009; Rose et al., 2012) as well as learning mechanisms like habituation (Colombo & Mitchell, 2009). It has proven useful in identifying differences between typical and atypical development (Chawarska et al., 2013). A key advantage of eye-gaze for infant assessment is that it is non-invasive and does not require a coordinated verbal or motor response, making it particularly well-suited for studying early cognition.

Despite the widespread use of eye-tracking in laboratory research studies, it is seldom integrated into standardized assessments, although see (although see Fagan III & Detterman, 1992for an early, low-tech attempt at this approach). In general, gaze paradigms require either highly trained individuals to undertake time-consuming hand-coding or requires expensive and often bulky hardware such as infrared eye-trackers (e.g., Tobii, EyeLink). To bridge this gap, the NIH Baby Toolbox (Baby Toolbox) incorporated eye-tracking measures using only a standard iPad Pro. The Baby Toolbox is a developmental assessment contracted by the NIH Neuroscience Blueprint in 2019 and designed to measure neuropsychological, motor, and social-emotional functioning in children aged 1–42 months (Gershon et al., 2024). One of the primary goals of the Baby Toolbox was to include many direct measures of infant and toddler capacities, rather than relying exclusively on parent-report or behavioral observation. For children older than 2 years of age, direct measures can include touch-based or verbal responses. However, these are not appropriate for infants younger than 2 years given the language demands required to understand instructions and the motor precision required to carry out a touch or even verbal response. For the youngest infants, gaze-based measures offer one of the best methods of direct-assessment.

Additional goals in developing the Baby Toolbox were to create a battery that minimized additional materials (beyond the iPad), that would be simple to administer, and that would automatically analyze data to provide immediate scores. In order for the Baby Toolbox gaze-measures to meet those criteria, we developed a method of automated gaze-capture that utilized the built-in iOS ARKit software and front facing camera of a standard 11” iPad Pro. Indeed, our approach never aimed to reach the spatial resolution of commercial eye-trackers, but rather focused on preferential looking—that is, classifying looks as on-screen or off-screen, and right side or left side. Acknowledging this trade-off in precision, the resulting technology allowed us to incorporate simple gaze paradigms (e.g., preferential looking, habituation) that can be administered with relatively little training, and return scores automatically, with no coding or postprocessing necessary.

This paper describes the development of the Baby Toolbox gaze-tracking methodology for the norming study that took place from May 2023-March 2024. Our aim is to document a practical and scalable approach to gaze-tracking technology tailored for large-scale infant assessment. By providing a clear account of this method, we aim to support transparency, reproducibility, and future use of the Baby Toolbox norming data as well as validate the use of these gaze measures for future Baby Toolbox use. We provide a brief description of the iPad technology that tracks and analyzes infant gaze, present evidence on the success rates of calibration in a large demographically diverse sample, and provide evidence of reliability of our gaze-classification algorithms compared to hand-coding.

2. Method

2.1. Gaze Tracking using iOS ARKit

Baby Toolbox gaze detection utilizes the front-facing camera built into a standard iPad Pro. To capture gaze, we exploit iOS ARKit (Apple Inc), a software framework for augmented reality applications. The iPad’s front camera is equipped with an infrared illuminator that projects 30,000 dots on the user’s face, and the resulting 2D infrared image is recorded using a low spatial infrared camera (Apple, 2017). Through this, a depth map for face localization and identification along with eye position and gaze direction can be estimated. Specifically, a matrix encoding the position, orientation, and scale of the center of the participant’s face is obtained, known as the anchor (Apple, 2024). Through ARKit’s algorithm, the position of the center of the eyeball can be obtained relative to the anchor, and the matrix indicates the orientation of the eyeball. This allows for estimation of what point, relative to the anchor, the eyes are focused on, with head and eye movement sampled at a rate of 60 Hz. The resulting x- and y-coordinates of the eyes are mapped to iPad screen coordinates, known as lookAtPoints in the X and Y directions (Apple, 2024). Baby Toolbox takes this one step further by re-drawling the screen size based on the coordinates collected during calibration, with the expectation that the child is tracking the calibration stimuli across the screen, and thereby confirming screen size dimensions specific to that testing occasion. We note that the gaze-tracking software developed for the Baby Toolbox norming study was designed for the 11-inch iPad Pro available at the time of development and testing (2019–2024), which had a portrait-oriented camera (i.e., the camera was positioned at the side when in landscape mode, which we standardized to always being on the left-hand side).

All Baby Toolbox gaze measures include three components: (1) head placement, (2) calibration, (3) test items (See Figure 1). During head placement, engaging animations appear on the screen together with a video of the participant within a colored box. ARKit calculates the boundaries of the face and the distance of the face from the iPad camera. When the face is between 0.25 and 0.75 meters from the front facing camera and the outer boundaries of the face are within the head placement box, the box changes color to indicate to the examiner that the iPad and participant are positioned appropriately for data collection (blue for correct placement, red for incorrect placement).

Figure 1.

Gaze Measure screenshots from Head Placement, Calibration and Test. Image of child obtained with parental permission for use in this manuscript. Test images are not real items but reflect the nature of real items. ©Toolbox Assessments, 2025.

Once criteria are met during head placement, the measure proceeds to calibration, during which gaze points are paired with known x- and y-locations on the screen. This is used to define the range of points that are “on-screen” and classification of subsequent x- and y-values during the test portion of the measure into Areas of Interest (AOIs), which are the regions of the screen that infants are expected to look at. Gaze-contingent calibration targets appear sequentially in the four corners of the iPad screen. Animated targets remain in each corner up until the iPad registers a “look¹” or for 5 seconds if no look is registered. Calibration targets move between the four corners of the screen, supporting gaze following for the youngest infants.

For calibration to be “successful,” two criteria must be achieved. First, a “look” must be registered in response to each of the four calibration targets. Second, the looks must meet separation and alignment checks. That is, the average of the x-coordinates of the two left targets and the two right targets must be within 20 “units” of each other, and the average y-coordinates of the two top targets and the two bottom targets must be within 30 “units” of each other², See Figure 2. These criteria help ensure that gaze responses to calibration targets are in the appropriate space and help adjust the screen estimates uniquely for each child. The look-at-points collected during calibration are translated into percentages representing both on- and off-screen locations. These parameters are then applied to look-at-point data collected during the test items to classify looks as either on-screen right, on-screen left, or off screen. Calibration rounds can be repeated up to three times until criteria are met, at which point the measure advances automatically to the test portion. If the child looks away during calibration, the iPad will return to head placement to reorient the child and give the examiner a chance to adjust if necessary. If the child does not pass calibration criteria within three rounds, the measure still advances to test but does not classify gaze data into AOIs. Regardless of whether a child passes calibration, photographs of the child looking at the screen are captured throughout the calibration and test portions at a rate of approximately 12–20 frames per second. These photos can be hand-coded or submitted to machine-learning approaches (e.g., iCatcher, Erel et al., 2022) for additional or alternative analyses, or to score children who failed calibration.

Figure 2.

Successful gaze calibration involves a ~1s “look” to be registered at 4 animated calibration targets at each of the 4 screen corners (3 rounds allowed). Once four successful “looks” are detected, subsequent alignment and separation checks are conducted to determine whether all four “looks” detected are within tolerance. If failed, “looks” are discarded.

During the test, gaze data are classified into AOI categories, the boundaries of which are created uniquely for each participant based on calibration. In paradigms with dual-image displays (e.g., Looking while Listening), right and left AOIs have a 6% central buffer zone as a “no man’s land” to avoid misclassification of ambiguous looks. For on- vs. off-screen paradigms (e.g., Familiarization), AOIs include a buffer zone extending up to 60% of the screen width to the right and left and up to 300% of the screen height above and below, based on pilot testing showing that gaze within these regions was rarely indicative of true off-screen looking. Data available for exports includes raw x- and y-coordinates collected at a rate of 60 Hz, Area of Interest (AOI) classifications, and processed scores.

3. Reliability Study

3.1. Participants

Data from infants between the ages of 5 months 16 days and 24 months 15 days (N=1130, 49% females, M_age = 14.8 months, SD_age = 5.4 months) who participated in the Baby Toolbox norming study were used to assess calibration success and reliability of eye-gaze measures during the norming study. Participants were recruited to be racially and ethnically diverse, from a range of familial educational backgrounds and locations in the US, representative of the 2020 US census (See Table 1). Eye color was indicated by experimenter in a subset (46.6%, n=527) of individuals (“Light”: 121 (23% of subsample), All other colors (i.e., “Dark”: 406 (77% of subsample). Participants with known genetic conditions were excluded from recruitment; however children with non-generic clinical conditions as well as children born prematurely, were included in the sample. See (Han et al., 2025) for more on details study recruitment and participant characteristics.

Table 1.

Sample Summary by Demographic Category

	Full Sample Calibration (N=1130)	Looking While Listening Hand-coding Subsample (N=10)	Familiarization Hand-Coding Subsample (N=10)
Age (months)  5.50–8.49  8.50–11.49  11.50–14.49  14.50–17.49  17.50–20.49  20.5–24.49	162 (14.34%) 221 (19.56%) 173 (15.31%) 172 (15.22%) 171 (15.14%) 231 (20.44%)	0 (0%) 3 (30%) 3 (30%) 0 (0%) 3 (30%) 1 (10%)	2 (20%) 2 (20%) 1 (10%) 2 (20%) 3 (30%) 0 (0%)
Gender  Female  Male	552 (49%) 578 (51%)	5 (50%) 5 (50%)	5 (50%) 5 (50%)
Race/Ethnicity  Any race, Hispanic  Asian, Non-Hispanic  Black, Non-Hispanic  Other, Non-Hispanic  White, Non-Hispanic	333 (29%) 26 (2%) 239 (21%) 111 (10%) 421 (37%)	0 (0%) 0 (0%) 3 (30%) 1 (10%) 6 (60%)	2 (20%) 0 (0%) 1 (10%) 1 (10%) 6 (60%)
Parent Education  Less than high school  High school GED  Some college  Bachelor’s degree  Greater than Bachelors	44 (4%) 199 (18%) 355 (31%) 320 (28%) 212 (19%)	0 (0%) 1 (10%) 3 (30%) 3 (30%) 3 (30%)	1 (10%) 1 (10%) 4 (40%) 3 (30%) 1 (10%)

A subset of 20 participants (M_age = 14.04 months, SD = 4.53; 10 from Looking While Listening and 10 from Executive Functioning) was randomly selected from the larger sample of successfully calibrated participants to assess gaze reliability compared to hand-coding (See Table 1 for demographic characteristics).

3.2. Materials

Gaze measures were administered using the Baby Toolbox app on 4^th generation 11-inch iPad Pro tablets.

3.3. Procedure

3.3.1. Testing:

Participants were administered the full Baby Toolbox norming battery, consisting of 11 to 15 measures depending on the age of the child (See Han et al., 2025). Infants aged 5.5–21.49 months were administered four unique gaze measures - Looking While Listening (Novack et al., 2025), the Executive Function-Cognition Gaze Battery (which includes three portions - Familiarization, Novelty Preference, and Visual-Delayed Response, Flynn et al., 2025), Numerical Change Detection, and Spatial Change Detection (Pila et al., 2025). Infants 21.5 to 24.49 months were administered three gaze measures (Looking While Listening, Numerical Change Detection, and Spatial Change Detection). Gaze measures were administered with other types of measures interspersed between them (i.e., not back-to-back), but the order remained identical unless intentionally changed by the examiner at the examiner’s discretion. Gaze measures were administered with the child sitting either on a high-chair or on their caregiver’s lap facing an iPad, which was held on a tall, adjustable iPad stand (See Figure 3). If the latter, caregivers were asked to wear an opaque face mask so that their own face would not be captured unintentionally during the test administration. The examiner asked the parent not to talk to or interact with the child during the measure. Children were not permitted to use a pacifier during gaze measures, as it can interfere with ARKit’s face-detection. Examiners stood to the right of the child during head placement while adjusting the iPad. As soon as the iPad was positioned correctly and head placement was successful (i.e., solid blue frame around the face), the examiner tapped a button to proceed to calibration and then stepped off to the side, behind the child and out of sight for the remainder of the measure. Examiners were trained to look away and avoid eye-contact if the child looked at them during the measures.

Figure 3.

Administration of NIH Baby Toolbox gaze measures, in which the child sits on a highchair and faces an iPad.

3.3.2. Hand-Coding.

To assess the reliability of our gaze-tracking algorithms, three independent coders manually annotated images captured at a rate of approximately 15 frames per second from 20 participants who participated in the norming study. We selected two measures for reliability analysis: Looking While Listening, which uses a right-left paradigm, and Familiarization from the Executive Functioning battery, which uses an on-off paradigm. These measures were chosen because they represent the two primary gaze-tracking approaches used in the Baby Toolbox and were the two gaze measures ultimately included in the final release of the battery. Coders manually classified all captured images from the test portion of the measure from the 20 participants (total N=56,020 frames), indicating gaze location: off-screen or on-screen for Familiarization, and off screen, on screen right, or on screen left for Looking While Listening. Coders were blind to the trial content and had previously achieved a minimum of 90% agreement with one another on a training subsample.

4. Analyses

In evaluating the success of our gaze technology, we consider two metrics: (1) rates of successful calibration in a large, diverse sample of infants, and (2) reliability of gaze classification algorithms compared to hand-coding for a randomly selected subset.

4.1. Calibration success.

Calibration success was defined as a child meeting all calibration criteria within three rounds. First, we analyzed calibration success across all measures administered to a child, which provided a sense of calibration success in the context of the norming study specifically. However, this approach underestimates the general capabilities of the Baby Toolbox gaze technology, as participants were administered up to four gaze measures and we anticipated that children’s attention (and therefore willingness to sit through calibration) would decrease with multiple presentations. Therefore, we also considered calibration on the first administration of a gaze measure as a closer comparison to how gaze measures are administered in laboratory settings (where it is typical for only a single eye-tracking study to be run on an infant participant). Finally, as the most flexible analysis, we asked whether participants ever calibrated across all administrations to determine whether participants in each demographic category are capable of calibrating with our technology. Studies with commercial eye-trackers have found differences across race and eye-color, with lower calibration rates and accuracy among Asians individuals and individuals with light colored eyes (Blignaut & Wium, 2014; Hessels et al., 2015). Calibration success across demographic factors (age, sex, race/ethnicity, parent education) as well as eye-color would support the use of our gaze-tracking approach with diverse participants.

We conducted univariate logistic regressions of calibration success across demographic categories to determine whether calibration success was differentially affected by the child’s age group, sex assigned at birth, race/ethnicity, and parental education. Post-hoc pairwise tests were Tukey comparisons of the estimated marginal means and were conducted where the covariate estimate was associated with calibration success with greater than 95% certainty to identify differences among pairs within the demographic categories.

4.2. Reliability.

To consider reliability of gaze-tracking we calculated chi-squared tests to determine differences in ARKit AOI classifications to hand coding classifications. Since ARKit was sampling at a higher frequency than still images were captured, we chose to use the AOI classification from the x- and y-coordinates immediately preceding the capture of a still image for these comparisons, allowing for the chi-square test to be a test of independence between the methods. The Cramer’s V statistic was additionally calculated to provide a measure of effect size for the chi-square test. A Cramer’s V of greater than .4 is generally considered a medium to large effect size for most degrees of freedom, with V of .5 or more considered a large effect size for all degrees of freedom (Cohen, 1988; Rea & Parker, 2014). Cohen’s Kappa was calculated to quantify the interrater reliability of the handcoded data and ARKit values, which is more robust than percent agreement, as this statistic corrects for when raters may agree by chance (Cohen, 1960; McHugh, 2012). A Cohen’s Kappa of 0-.2 is considered to have no agreement, .21-.39 to have minimal agreement, .40-.59 to have weak agreement, .60-.79 to have moderate agreement, .80-.90 to have strong agreement, and above .90 almost perfect agreement (McHugh, 2012).

5. Results

5.1. Calibration success

The average success rate for calibration was greatest on the first (69.29%) administration, with lower rates on the second (61.15%), third (54.51%) and fourth (37.35%) administrations. Given this decline with administration order, we explored calibration success on the first administration specifically, as this approach is most comparable to eye-tracking in laboratory settings, where children are typically only administered a single eye-tracking study in each visit.

For the first administration of a gaze measure, there was no difference in the log odds of calibration success by eye-color (β=−0.10, SE = 0.22, p=.67) as well as a non-significant trend of age (β=−0.02, SE = 0.01, p=.08), and a non-significant effect of sex assigned at birth (β=.25, SE = 0.13, p =.05). There were however significant effects of race/ethnicity. Asian children had higher log odds of calibrating on the first measure compared with White Non-Hispanic children (β=2.24, SE = 1.02, p=.03) and Black non-Hispanic children had lower log odds of calibrating on the first measure as compared with White non-Hispanic children (β= −0.51, SE = 0.17, p=.003). Hispanic children (any race) (β= −0.08, SE = 0.16, p=.62) and Other non-Hispanic children (β= −0.20, SE = 0.23, p=.38) did not have significantly different log odds of calibrating on the first measure compared with White non-Hispanic children. Significant differences in calibration success on the first administration also emerged by parental education. Children of parents with some college or higher had higher log odds of calibrating on the first measure compared with children of parents with less than high school education, (all comparisons 0.72≤ β’s ≤ 1.06, .001≤ p’s ≤ .02). See Figure 4 for a visualization of all calibration results by demographic category.

Figure 4.

Calibration success by demographic category.

Finally, as our most flexible analysis, we considered whether participants ever calibrated. Eighty-seven percent (n = 981/1130) of children calibrated successfully on at least one of the three to four gaze measures they were administered. This success was very high across all age groups (79.65% - 96.91%) although there was a significant negative relationship in log odds of calibrating as the child matured in age (β=−0.08, SE = 0.02, p<.001). The log odds of calibration success was not impacted by eye color (β=−0.04, SE = 0.32, p=.92), or sex assigned at birth (β=0.13, SE = 0.18, p=.46). There were also no significant changes in log odds of calibration success for any race/ethnicity groups compared to reference (White, non-Hispanic, .067≤ p’s ≤.80). Additional pairwise estimated marginal means indicated that other comparisons between races also did not produce significant differences in log odds of calibration success (.35≤ p’s ≤.99).

Finally, consistent with the findings for the first gaze measure, there were significant differences in the odds of calibration success on at least one measure based on parental education. Children of parents from all education groups with a high school degree and above had significantly higher log odds of calibrating on at least one measure compared with children of parents who had less than high school education (all comparisons, .99≤ β’s≥1.66, all p’s<.0001). Numerically, child calibration success increased with increasing parental education (see Figure 4).

5.2. Reliability of algorithmic gaze classification

To compare the reliability of gaze classification versus hand coding, we created contingency tables and calculated chi-square tests to look at how the ARKit algorithms classifications compared to hand-coding. First, we considered ARKit’s classification of on vs. off screen looks using the Familiarization paradigm. Across all participants and items, there was a significant relationship between hand-coding and ARKit classification for on screen vs. off screen values, χ² (1, N=24,619) = 9168.3, p<.001, Cramer’s V =.61, Cohen’s κ =.58. Table 2 shows very high agreement for looks classified by ARKit as off screen (92.65%) as well as high agreement for looks classified by ARKit as onscreen (68.07%).

Table 2.

Contingency Table of Hand Coded vs. ARKit Classification Values for On-screen vs. Off-screen (Familiarization)

	Familiarization
	Off screen (hand-coding)	On screen (hand-coding)
Off screen (ARKit) On screen (ARKit)	92.65% (9880) 7.33% (782)	31.93% (4456) 68.07% (9501)
Total	100% (10662)	100% (13,957)

Note. Table includes values across all participants and items in the Familiarization task, including filler items that are used to grab the infant’s attention between trials. Total samples = 24,619; number of samples per cell indicated in parentheses.

Second, we considered the three-way classification for the right-left paradigm by examining test items on the Looking While Listening task. Here, we compared Arkit classification to hand-coding for on-screen right, on-screen left and off-screen. Again, there was a significant relationship between hand-coded and ARKit classification across all samples, χ² (4, N=31,401) = 27,519, p<.001, Cramer’s V = .66, Cohen’s κ = .65. ARKit and hand-coding classification showed 87.45% agreement for looking left on screen, 70.21% agreement for looking right on screen, and 75.33% agreement for looking off screen, see Table 3 for full descriptives.

Table 3.

Contingency Table of Hand Coded vs. ARKit Classification Values for Left On-screen, Right On-screen, and Offscreen (Looking While Listening).

	Off screen (hand coded)	Left on screen (hand coded)	Right on screen (hand coded)
Off screen (ARKit coded) Left on screen (ARKit coded) Right on screen (ARKit coded)	75.33% (7,812) 15.23% (1,579) 9.45% (980)	10.49% (942) 87.45% (7,850) 2.06% (185)	13.91% (1,677) 15.87% (1,913) 70.21% (8,463)
Total	100% (10,371)	100% (8,977)	100% (12,053)

Note. Table includes values across all participants and items in the Looking While Listening task, excluding filler items that are used to grab the infants’ attention between trials, as they are in the center of the screen. Total samples = 31,401; number of samples per cell indicated in parentheses.

Next we examined the gaze classification reliability at the individual level for the 20 randomly selected hand-coded participants. Chi-square values indicated that there was a significant relationship between hand coding and ARKit classification for all participants with corresponding moderate to strong effect sizes of Cramer’s V>.4 for most, see Table 4. Thirteen out of twenty participants showed moderate to very strong agreement between hand coding and ARKit with Cohen’s κ above .6, and two individuals showed a fair agreement (>.4). Three individuals showed minimal agreement with Cohen’s κ <.3, and two participants, Fam3 and LwL4, showed no agreement see Table 4.

Table 4.

Participant-level comparisons of hand coding vs. ARKit classification

Participant	Total eye-tracking samples	% agreement	χ 2 (df=4)	p-value	Cramer’s V	Cohen’s κ
Fam1	3079	92.1	2032.0	<.001	.812	.807
Fam2	3294	78.7	1271.8	<.001	.621	.585
Fam3	3827	27.1	11.1	<.001	.058	.018
Fam4	3109	96.9	2694.0	<.001	.931	.931
Fam5	3077	95.1	2462.8	<.001	.895	.893
Fam6	3322	88.7	1362.5	<.001	.640	.624
Fam7	3347	91.2	2181.6	<.001	.807	.807
Fam8	3289	89.4	2090.8	<.001	.797	.790
Fam9	3069	52.7	294.6	<.001	.310	.199
Fam10	3356	60.3	406.5	<.001	.348	.224
LwL1	3,330	75.6	2521.2	<.001	.615	.601
LwL2	3,139	93.2	4243.8	<.001	.822	.877
LwL3	3,426	93.5	5004.4	<.001	.855	.890
LwL4	3,164	40.2	500.5	<.001	.218	.111
LwL5	2,936	92.0	Fisher’s exact test p<.001			.849
LwL6	3,130	77.6	3069.9	<.001	.700	.664
LwL7	1,950	75.7	822.2	<.001	.459	.468
LwL8	3,365	57.5	1571.0	<.001	.483	.374
LwL9	3,350	78.2	1943.5	<.001	.539	.505
LwL10	3,357	87.3	4476.6	<.001	.817	.806

Note. A significant chi-square rejects the null hypothesis that the two methods are independent from each other; that is, there is an association between the methods.

6. Discussion

The NIH Baby Toolbox is the first standardized, nationally normed developmental assessment to incorporate gaze tracking measures for evaluating cognitive abilities in infants. This paper outlined the methodology used to develop the Baby Toolbox’s automated gaze-tracking system and presented evidence of calibration success and reliability from the norming study (See Han et al., 2025). Results highlight the Baby Toolbox gaze technology as a novel approach to developmental assessment.

Our primary goal in developing the Baby Toolbox was to create a battery that could be administered without extensive training or expensive equipment, and that would not require burdensome post-processing to obtain scores. To incorporate gaze-measures – a cornerstone of infant cognitive development research – we developed an automated gaze-capture method capable of classifying right-left and on-off screen looking using only a standard iPad Pro. The Baby Toolbox gaze measures offer data exports that include both computed scores and the complete x- and y-coordinates, allowing researchers to perform their own post-processing analyses if desired.

Results from a study of over 1000 infants who were diverse in age (6–23 months), racial/ethnic identity, parental education, and eye-color, suggest that the Baby Toolbox gaze capture has excellent success of calibration and classification reliability for infant looking. Overall, we found very high rates of calibration success. Not only were the youngest infants very successful in calibrating, but we saw no differences by eye-color or sex assigned at birth. Differences in calibration success emerged for race/ethnicity, with black participants having lower rates of calibration success relative to other races on the first administration. However, when considering if participants ever calibrated across all administrations of a gaze measure, this difference disappeared, with all races showing strong (>83%) success. It is also notable that we did not see any differences in calibration success for the groups that commercial eye-trackers typically struggle to track (Blignaut & Wium, 2014; Hessels et al., 2015). There were no differences in calibration success by eye color (light vs. dark) or for Asian children, who have historically been discriminated against with traditional infra-red eye-tracking approaches. Together, these results suggest that our gaze-tracking approach can work across diverse racial and ethnic populations.

A more consistent difference in calibration success emerged with respect to parental education. When looking either at the first administration or at success across any administration, there was a significant and directional decrease in calibration success following parent education with children of parents with higher education having the highest calibration success and children of parents with lower education having the lowest calibration success. Follow-up analyses of video-recordings, where available, may be able to offer additional insight into whether these differences are emerging from issues with our technology, administration error, or behavioral differences on the part of the children and/or caregivers.

That said, we also saw that calibration success dipped with multiple administrations, which is not unexpected. Although screens are highly appealing for older children, live interaction with examiners is more motivating for young infants. It is possible that children became impatient sitting through multiple administrations of a gaze procedure. This may have contributed to the lower success rates on administrations two through four compared to the first. Subsequent to this evaluation, the suggested Baby Toolbox battery currently includes only two gaze measures (Looking While Listening and Familiarization) which are meant to be administered nonconsecutively. By reducing the number of gaze administrations overall, examiners should experience higher rates of success.

Results from the reliability analysis of 20 participants and over 55,000 datapoints, indicate overall high rates of agreement between our automated AOI and hand-coding classifications. For the Familiarization paradigm we observed greater reliability for off-screen compared to on-screen, suggesting that the Arkit classifications are biased to over-estimate how long individuals are looking at the screen. For the Looking while Listening paradigm we observed greater reliability for left-side compared to right-side, likely because the model of iPad used for norming situated the camera on the child’s left. As the Looking while Listening measure counterbalanced target image across the two sides we do not believe this would have impacted results.

Although reliability broadly was high, we observed variability in reliability across participants. In particular, two participants – one from Looking while Listening and one from Familiarization – showed very poor reliability with hand-coding. Further examination of one of these participants (LwL4) indicated that their positioning (leaning back against the caregiver’s chest) did not follow administration recommendations. This emphasizes the importance of rigorous training and certification protocols to ensure that scores are as reliable as possible. In contrast, examination of the other participant with very low reliability (Fam3) showed no obvious deviations in protocol, raising the possibility that some third factor led to poor tracking with this individual. Future work should examine reliability across a larger sample of participants, which can provide further insight into the reliability success across different groups.

It is worth noting that our testing sample was normative and while we did not exclude participants with a clinical diagnosis, we were also not able to examine diagnostic status in the current analyses. Future work is ongoing to examine the success of the Baby Toolbox gaze-tracking with various clinical populations. Indeed, our gaze-tracking approach may be optimal for testing populations with limited language, even beyond the ages tested in the current study. Future work should evaluate the capabilities of this methodology among children, and even adults with disabilities.

Our decision to use an Apple iPad to administer the Baby Toolbox battery has many strengths because iPads are relatively inexpensive (compared to commercial eye tracking hardware) and easy to acquire. However, the downside is that any changes to the hardware or software may have implications for the measures. For example, at the time of testing (Fall 2023) the iPad was released with a portrait front-facing camera, which was positioned to the left during this testing. However, in 2024, new iPads were released with a landscape camera, centering the camera when positioned horizontally. This change in position may very well allow for more accurate tracking as it will be less biased. However, full follow up studies with the new versions will be needed to determine how the change in camera affects reliability. Researchers are encouraged not to change iPad models in the middle of the study.

Taken together, the results reported here support the use of the Baby Toolbox gaze measures in diverse samples. By capitalizing on the technological capabilities of a standard iPad Pro, the Baby Toolbox has the potential to catapult developmental assessment into a new age. This technology has great potential for expansion, making the benefits of gaze tracking available to large scale testing like never before.

Highlights.

• The iPad-based NIH Baby Toolbox measures infant gaze with built-in eye tracking.

• Calibration success was high across age, sex, race/ethnicity, and eye color.

• Comparison against hand-coding showed moderate to strong reliability.

• Results support iPad-based gaze technology for use in infant cognitive assessment.

Data statement

All data from the Baby Toolbox norming study, including the data reported in this paper, will be made publicly available in accordance with contract guidelines.