Cross-Cultural Applicability of Eye-Tracking in Assessing Attention to Emotional Faces in Preschool-Aged Children

Abstract

Humans show an attention bias toward emotional versus neutral information, which is considered an adaptive pattern of information processing. Deviations from this pattern have been observed in children with socially withdrawn behaviors, with most research being conducted in controlled settings among children from urban areas. The goal of the current study was to examine the cross-cultural applicability of two eye-tracking–based measures in assessing attention biases and their relations to children’s symptoms of socially withdrawn behaviors in two independent and diverse samples of preschool children. The cross-cultural comparison was conducted between the Navajo Birth Cohort Study (NBCS), an indigenous cohort with relatively low socioeconomic status (SES), and the Illinois Kids Development Study (IKIDS), a primarily Non-Hispanic White and high SES cohort. Children in both cohorts completed eye-tracking tasks with pictures of emotional faces, and mothers reported on children’s symptoms of socially withdrawn behaviors. Results showed that general patterns of attention biases were mostly the same across samples, reflecting heightened attention towards emotional versus neutral faces. The differences across two samples mostly involved the magnitude of attention biases. NBCS children were slower to disengage from happy faces when these emotional faces were paired with neutral faces. Additionally, socially withdrawn children in the NBCS sample showed a pattern of attentional avoidance for emotional faces. The comparability of overall patterns of attention biases provides initial support for the cross-cultural applicability of the eye-tracking measures and demonstrates the robustness of these methods across clinical and community settings.

Recognition of emotional cues in our environment is critical to understanding our world and social interactions. Individuals are more attentive to emotionally charged rather than neutral cues and to negative versus positive cues because negatively charged cues carry more informational value and likely have evolutionary, developmental and survival functions (Vaish, Grossmann, & Woodward, 2008). For example, detection of an angry or fearful face in a crowd can help us modify our behaviors and decisions to protect ourselves from a potential threat. These adaptive patterns of attention biases, characterized by paying more attention to negative over neutral and positive information, appear to exist early in life starting in infancy (Morales et al., 2017; Peltola, Yrttiaho, & Leppanen, 2018; Perez-Edgar et al., 2011). Infants begin to display increased attention to negative emotions around 7 months, which has been linked to changes in infants’ ability to move around and/or maturation of neural networks associated with recognition and detection of emotions (Heck, Hock, White, Jubran, & Bhatt, 2016; Leppanen & Nelson, 2009; Vaish et al., 2008).

Although attention bias toward negative cues in humans has adaptive functions and is considered to be normative, sensitivity to emotional cues varies significantly among individuals (Fu & Perez-Edgar, 2019; Nozadi, Spinrad, Johnson, & Eisenberg, 2018). Increased sensitivity towards negative emotions may be manifested in different types of responses to negative stimuli (Brown et al., 2013; Fu & Perez-Edgar, 2019; Perez-Edgar et al., 2011). These responses include (a) increased vigilance toward negative stimuli characterized by quicker detection of these stimuli, (b) difficulty disengaging from negative stimuli (i.e., longer looking times at negative versus neutral stimuli), or, conversely, (c) attentional avoidance of negative stimuli (i.e., shorter looking times at negative versus neutral stimuli), which can be a strategy for regulating one’s emotions (Fu & Perez-Edgar, 2019). Heightened sensitivity towards negative emotional stimuli has been found in young children with socially withdrawn behaviors and has been shown to contribute to the development of anxiety in late childhood or adolescence (Burris, Barry-Anwar, & Rivera, 2017; Field & Lester, 2010; Fu & Perez-Edgar, 2019; Nozadi, White, Degnan, & Fox, 2018). These findings highlight the importance of studying attention biases early on in children who have predisposition to anxiety, such as children with social withdrawn behaviors, so that appropriate interventions can be implemented to prevent or minimize mental health problems later in life.

Past work investigating attention biases in children under 5 years of age has been limited due to shortcomings associated with traditional measures. Typically, attention biases have been assessed in older children by asking them to produce a specific response, either verbally or by carrying out a motor action, such as pushing a computer button, to indicate their detection of a stimulus. These types of studies require that participants have more advanced motor, cognitive, and verbal abilities, which are often beyond the abilities of children below 5 years of age (Burris et al., 2017; Nozadi, Spinrad, et al., 2018; Perez-Edgar et al., 2010). To circumvent these limitations, recent studies have utilized eye-tracking technology to assess attentional processes, including attention biases in infants and children under 6 years of age (Armstrong & Olatunji, 2012; Armstrong, Olatunji, Sarawgi, & Simmons, 2010). The use of eye-tracking technology also has been recommended in older children and adult populations because it can provide a highly sensitive, continuous and direct measure of various components of visual attention underlying conscious and unconscious cognitive processes (Armstrong & Olatunji, 2012; Armstrong et al., 2010). Eye-tracking allows researchers to objectively and precisely determine where and how long a person looks at a stimulus without requiring any verbal or behavioral responses from the individual (Dodd et al., 2015; Stuijfzand, Stuijfzand, Reynolds, & Dodd, 2020). The two most common tasks that have been used in eye-tracking studies with both adults and children to assess visual attention to emotional stimuli are free-viewing and visual search tasks.

In the free-viewing task, one emotional stimulus (e.g., a fearful face) is paired with one neutral stimulus (e.g., a neutral face), whereas in the visual search task, the emotional stimulus is presented among multiple neutral stimuli. These tasks can assess different components of attention biases (Armstrong & Olatunji, 2012). The free-viewing task is the most common task used in eye-tracking studies to measure first fixation duration, i.e., defined as the length of time it took the individual to look at the emotional stimulus after it was detected, and dwell time, defined as cumulative looking time at each stimulus during the entire trial. Some studies also have measured detection latency, i.e., the time it takes a participant to look at the stimulus for the first time as compared to the neutral stimuli. Quicker detection of, and longer first fixations on negative stimuli reflect vigilance towards and initial attentional engagement with negative stimuli that are mostly driven by reactive and unconscious cognitive processes (Sagliano, Trojano, Amoriello, Migliozzi, & D’Olimpio, 2014). On the other hand, the dwell time assesses maintenance of attention on emotional stimuli over time, which can be indicative of effortful and conscious cognitive processes (Armstrong & Olatunji, 2012; Nozadi, Spinrad, et al., 2018). Significantly longer dwell times on emotional versus neutral stimuli are inferred as difficulty disengaging from the emotional stimuli, whereas significantly shorter dwell times on emotional negative stimuli are inferred as attentional avoidance of stimuli. Some studies have shown that anxious individuals may initially show vigilance towards negative stimuli by showing difficulty disengaging from them but display attentional avoidance of emotional stimuli over the entire trial (Eysenck, Derakshan, Santos, & Calvo, 2007; Sagliano et al., 2014).

The visual search task is often used to measure one component of attention bias, detection latency, which reflects vigilance toward emotional stimuli. The only difference between visual search and free-viewing is that the emotional stimulus is paired with more neutral facial expressions to reduce the floor effect (Armstrong & Olatunji, 2012). Increased vigilance toward the emotional stimuli in the visual search task is characterized by the significantly shorter detection latencies of emotional stimuli across trials. Evidence from eye-tracking studies with infants and preschool-aged children have shown shorter detection latencies in response to emotionally charged stimuli and longer dwell times on emotional versus neutral stimuli among children (LoBue, 2010; Peltola et al., 2018). Further, evidence suggests that young children who demonstrate anxious and socially withdrawn behaviors are more vigilant toward negative emotions or detect these stimuli faster than non-anxious children (Morales, Perez-Edgar, & Buss, 2015; Schmidtendorf, 2018).

To date, eye-tracking studies that examined attention biases to emotional information in preschool-aged children have been limited to controlled, university-based laboratory settings and, hence, to homogenous samples of young children composed of predominantly White children living in urban areas. The main goal of the current study was to examine if patterns of children’s attention biases to emotional faces (i.e., preferences for emotional, particularly threat-related, information versus neutral) were the same in two samples of preschool-aged children that were very different demographically, geographically, racially and culturally. Specifically, if increased attention towards emotional versus neutral information is evolutionary adaptive and universal (independent of individuals’ characteristics such as culture and language), we hypothesized that we would observe the same patterns across two samples using eye-tracking methodology. We also examined whether children who had mother-reported symptoms of socially withdrawn and anxious behavior showed different patterns of attention biases toward emotionally charged stimuli. Children’s symptoms of internalizing behavior problems were assessed via Child Behavior Checklist (CBCL), which is a commonly- and widely-used measure. Because the CBCL has not been validated in Navajo population, we tested the invariance of factor structure(s) across two samples.

The two samples included in analysis are part of the Environmental influences on Child Health Outcomes (ECHO) program, which is a national multidisciplinary research initiative launched by the National Institutes of Health (NIH) (Bush et al., 2020). Existing pediatric cohorts across the U.S. were competitively selected to become part of the ECHO consortium with the goal of enrolling more than 50,000 children from diverse ethnic/racial, socioeconomic, and geographical backgrounds to examine the impacts of diverse environmental, biological, social, and behavioral exposures on children’s health and development. Participants from two high-contrast ECHO cohorts were selected to test our hypothesis: the Navajo Birth Cohort Study (NBCS), an indigenous cohort with relatively low socioeconomic status (SES) participants enrolled from Tribal lands across New Mexico, Utah, and Arizona; and the Illinois Kids Development Study (IKIDS) cohort, a predominantly Non-Hispanic White cohort from relatively high SES families living in a college town in central Illinois. Children in our rural Navajo sample historically have not been included in developmental studies, including those investigating attention biases, whereas the IKIDS sample closely resembles the children assessed in most of the previously published eye-tracking studies of attention bias toward emotional stimuli (LoBue, 2010; Peltola et al., 2018; Perez-Edgar et al., 2011). Navajo children were tested in different community sites across the Navajo Nation (e.g., clinic and hospital spaces, Tribal facilities) using a portable eye-tracking system, whereas IKIDS children were all tested in the same university campus research laboratory using the more traditional stationary eye-tracking setup. If attention biases toward emotionally charged stimuli are universal due to their developmental and evolutionary functions (Mobbs, Hagan, Dalgleish, Silston, & Prevost, 2015; Vaish et al., 2008), then we expected to find similar attentional patterns to emotional information across our two cohorts irrespective of the differences in their cultures and demographics. Consistency in the response patterns across these two samples may provide evidence for the cross-cultural applicability of eye-tracking technology to assess emotional processing and attention biases. Establishing eye-tracking as a useful methodology can also be considered for testing other aspects of early cognition that appear to be independent of culture and language, such as recognition of basic negative emotions and infants’ ability to distinguish contrasts in languages (Sauter, Eisner, Ekman, & Scott, 2010; Werker, 1984).

Methods

Participants

Transparency and Openness:

The subsequent sections include detailed information about two samples, recruitment, exclusion criteria, measures, data exclusion and manipulations. Data were analyzed using R, and all analyses were performed by two independent biostatisticians. The study design and analysis were not pre-registered. All data, analysis code, and research materials will be available upon requests directed to the ECHO Data Analysis Center (DAC).

Navajo Birth Cohort Study Sample.

The NBCS was initiated in 2010 to examine how birth outcomes and children’s neurodevelopment were impacted by uranium exposure from abandoned mine wastes following decades of uranium mining across the Navajo Nation. Navajo children included in the current study were a subset of children from 723 pregnant women recruited by NBCS at Indian Health Service (IHS) or Public Law 638 hospitals in Gallup and Shiprock, New Mexico, and Chinle, Fort Defiance, Kayenta, and Tuba City, Arizona, between February 2013 and June 2016. To be included in the initial phase of the NBCS, which spanned from pregnancy to the child’s first birthday, women needed to be between 18-45 years of age, have a medically confirmed pregnancy, have resided on the Navajo Nation for five years and plan to deliver at one of the participating hospitals. Among other data collected, eligible mothers reported on their sociodemographic and medical history and lifestyle information during pregnancy.

More than 300 children from the initial phase were reenrolled in the initial ECHO phase of the study, and of those children, 136 reached the 4- to 5-year-old neurodevelopmental assessment window (42-66 months of age), which included the eye-tracking tasks between September 2018 and March 2020. The rural nature of the Navajo Nation and the lack of transportation and communication infrastructure presented challenges for scheduling participants, and thus, a broad contact window and age range for scheduling were implemented to minimize the loss to follow-up of participants. During the assessment, mothers reported on their children’s behavioral problems and socioemotional functioning using the preschool version of the Child Behavior Checklist for Ages 1½-5 (CBCL/1^1/2-5) (Achenbach & Rescorla, 2000). A detailed description of this instrument is provided in the Methods section. Children also participated in two eye-tracking tasks, described in further detail in this section, to assess their attention biases. From the battery of assessments administered, the CBCL/1½-5 and eye-tracking assessments are the focus of the current analysis. The assessments were completed in a variety of spaces reserved in six participating communities (e.g., hospitals, colleges, Navajo Nation program offices). Both the University of New Mexico Institutional Review Board and the Navajo Nation Human Research Review Board had oversight of the study and reviewed and approved all protocols and consent documents.

Out of the 136 children who completed the comprehensive 4- to 5-year-old neurodevelopmental assessment battery, data on at least one of the eye-tracking tasks were obtained for 125 children (68 females; _{Mage for total sample} = 51.96 months; SD_{age for total sample} = 5.5), and data from the CBCL/1½-5 questionnaire were available for 118 of those children. Seven children only had eye-tracking data, and 11 children only had CBCL/1½-5 data. Missing CBCL/1½-5 data were due to mothers not completing or returning the surveys. Missing eye-tracking data resulted from the unavailability of the eye-tracking equipment due to scheduling conflicts (n = 6), difficulty calibrating the equipment when testing a highly active child (n = 2), or a child’s reluctance to complete the task (n = 3).

Illinois Kids Development Study Sample.

IKIDS children were a subset of the children born to 565 pregnant women who enrolled in the IKIDS cohort between December 2013 and March 2020. Women were recruited for the study from two obstetric clinics in the Urbana-Champaign, Illinois area, late in their first trimester of pregnancy. They were eligible to participate if they did not have a previous child born into the IKIDS cohort, were between 18 and 40 years of age at the time of enrollment, were not in a high-risk pregnancy, were not carrying multiples, were fluent in English, resided within a 30-minute drive of the University of Illinois at Urbana-Champaign campus, and were planning to remain in the area until the child’s first birthday. A maternal interview was completed at the time of the woman’s enrollment, between 10 and 14 weeks of gestation, to obtain baseline sociodemographic, medical, and reproductive history, and lifestyle information (Eick et al., 2021). Children were enrolled in the cohort at birth and completed their first assessment for the study at the hospital before they were discharged to go home.

Following the hospital assessment, mothers were invited to bring their children for periodic follow-up and cognitive assessments at the IKIDS research laboratory at the University of Illinois campus, including between 46-48 months of age or just around the time of the child’s 4^th birthday. This later lab visit included a battery of neurodevelopmental assessments of the child and several maternal surveys about the child, including the CBCL/1½-5 and the two eye-tracking tasks to assess attention biases to emotional stimuli. Starting in fall 2019, IKIDS participants began to be recruited to participate in the ECHO program in addition to their participation in IKIDS. The study was approved by the Institutional Review Boards at the University of Illinois at Urbana-Champaign and ECHO. Participants completed 3 written informed consents for their participation in IKIDS: at the time of enrollment in pregnancy, at the first research lab assessment in infancy, and at the 4-year assessment. Consent to participate in ECHO was completed only once and it happened at the IKIDS enrollment visit for the pregnant women who had not yet enrolled in IKIDS, or at the first available opportunity after that for those who had already enrolled in IKIDS.

A total of 80 children whose mothers consented to be both in IKIDS and ECHO came to the IKIDS research lab for the 4-year-old assessment between September 2018, when the eye-tracking assessments started, and March 2020, when in-person assessments were suspended due to the COVID-19 pandemic. Out of these children, 70 had eye-tracking data (42 females, M_{age for total sample} = 48.12 months; SD_{age for total sample} = .44). CBCL/1½-5 questionnaires were completed by the mothers of 67 children. Most CBCLs were completed during the lab visit, but some were completed shortly before or shortly the visit. Ten children did not have usable data on the eye-tracking tasks for the following reasons: (a) the child was too tired and unwilling to attempt the task (n = 3), (b) the child was uncooperative and unable to sit still for the testing (n = 3), (c) the child had trouble wearing the eye-tracker target sticker on their forehead and thus calibration was not possible (n = 2), (d) the child was shy and unwilling to do any testing with the researcher (n = 1), or (e) data from the testing could not be retrieved from the computer due to technical issues (n = 1).

Procedures

NBCS sample.

Children were brought to the designated testing sites across the Navajo Nation for the neurodevelopmental assessment, during which mothers completed the CBCL/1½-5 questionnaire. Following the completion of the neurodevelopmental assessments, children completed the two eye-tracking tasks by sitting 65 cm away from a laptop screen with a size of 38.1 × 21.59 cm and a resolution of 1600 × 900 pixels. A portable eye-tracker camera and a near infrared illuminator (SR-Research EyeLink Portable Duo) were installed on the laptop keyboard. The use of a portable eye-tracker allowed us to complete the eye-tracking tasks across the community sites. The near infrared light, invisible to the human eye, was directed to one of the child’s eyes to create reflections on the cornea, which the eye-tracker’s high-resolution camera tracked at a sampling rate of 500 Hz. The position and movements of the eye were tracked by the camera through these corneal reflections. A booster seat was used with some children to allow the eye-tracker’s camera to capture their eyes properly.

Prior to starting the eye-tracking tasks, the eye-tracker was calibrated by asking children to follow a focal point (a circle) as it appeared at the center and each of the four corners of the screen. The calibration was validated by asking children to repeat this procedure to ensure that children’s new gazes matched the initial ones at each of the five locations. Calibration was repeated until the eye-tracker software indicated that there was concordance between calibration and gaze validation at each of the five points based on the software standardization protocol. Specifically, the calibration deviations for each participants should have been between 1.0° and 1.5° with worst point error being less than 2.0°, as indicated by the eye-tracking software Following calibration, children completed the free-viewing and visual search tasks, in which pictures of emotional faces were paired with neutral faces. The pictures of facial expressions were taken from the NimStim-MacBrain Face Stimulus Set developed by the Research Network on Early Experience and Brain Development (Tottenham et al., 2009), which has been used in previous studies of young children’s face and emotion recognition. The original stimuli set includes facial expressions portrayed by 43 female and male actors from White, African American, Asian American and Hispanic backgrounds (Tottenham et al., 2009). Facial expressions in the free-viewing task were different from those used in the visual search task. In each task, equal numbers of pictures portrayed by female and male actors were used, with representation from all race/ethnicities used in the original stimuli set.

IKIDS sample.

Children were brought to the IKIDS research laboratory at the University of Illinois at Urbana-Champaign campus. Stimuli (e.g., sizes of stimuli, background color), screen resolution (i.e., 1600 × 900 at 60 Hz), testing procedures (e.g., calibration procedures), and sampling rate (500 Hz) were similar to those used in the NBCS. The only differences in the two cohort settings were relatively small variations in the degrees of visual angles for viewing the presented stimuli (i.e., a visual angle difference of less than 1.5 degrees) due to different monitor dimensions and distances between the child and the monitor’s screen. Because a stationary infrared eye tracker (SR-research EyeLink 1000 Plus), instead of a mobile eye tracker was used in the IKIDS cohort, the stimuli were presented on a large TV (102 × 57 cm), instead of a laptop screen although the resolution of the screen was the same across cohorts. Thus, while children’s distance to the eye-tracking camera was kept at 65 cm as with the NBCS sample, children sat further away from the display screen, at a distance of 1.5 m. Because of these differences, the degree of visual angle for viewing pictures from the free-viewing task in the IKIDS sample was 9.69 × 12.47 compared with 8.36 × 10.76 in the NBCS sample, and the degree of visual angle for viewing pictures from the visual search task in the IKIDS sample was 7.3 × 9.35 compared with 6.29 × 8.07 in the NBCS sample.

The procedure and protocol used, including those related to calibration and validation procedures, were the same across samples. In the IKIDS cohort, ten children completed tasks after having poor initial calibration according to the eye-tracking software (i.e., calibration deviations > 1.5° with worst point error > 2.0°). Data for these ten children were not excluded from our analyses because our sensitivity analysis did not show any differences between eye-tracking variables for these children and the children that met the calibration criteria.

Measures

Measures administered included 1) two eye-tracking tasks, free-viewing and visual search, for assessing attention biases, and 2) CBCL/1½-5 questionnaire for assessing children’s anxious and socially withdrawn behaviors. The detailed description of two eye-tracking tasks and CBCL/1½-5 are described below.

Free-Viewing Task.

In this task, children completed 42 (NBCS) or 48 (IKIDS) trials. Facial expressions from two actors, resulting in loss of six trials, were removed from the task used in NBCS but were kept in the IKIDS due to differences in protocols between the two sites. Each trial comprised one of three emotional faces (angry, sad, or happy) paired with a neutral face of the same person. Each emotional face appeared in 14 (NBCS) or 16 (IKIDS) trials, which were presented in a random order. Positions of emotional faces in each picture also were counterbalanced across trials. Trials lasted for 2,000 ms, sufficient for capturing young children’s attention and allowing for examination of attentional processes (Buckner, Maner, & Schmidt, 2010). Before each trial began, a fixation point or attention getter was displayed in the center of the screen, until the child looked at it for 500 ms. If the child did not meet this criterion, a drift check procedure was completed in which the child’s attention was drawn to a point at the center of the screen to check how far the recorded gaze was from the actual center of the screen. If the error was too large, the calibration procedure was repeated before the test proceeded.

The following eye movement variables were measured during each of the trials in the free-viewing task: (a) detection latency of each face, (b) first fixation duration on each face, and (c) dwell time or cumulative fixations on a face during the entire trial. The fixation was determined by eye-tracking software developed by SR Research (Kanata, Ontario, Canada) and was defined as a relatively short period of time, 200 ms or shorter, during which the gaze is relatively stable on one location (“EyeLink® 1000 User Manual,” 2005-2009). The purpose of the free-viewing task was to determine whether the type of emotion displayed influences these attention bias components. For example, significantly shorter detection latencies of emotional faces compared with neutral faces indicate vigilance toward an emotional stimulus. Significantly longer first fixations and dwell times on the emotional faces suggest difficulty disengaging from the emotional stimuli, whereas significantly shorter first fixations and dwell times on emotional faces indicate avoidance of emotional stimuli (Fu, Nelson, Borge, Buss, & Perez-Edgar, 2019; Seefeldt, Kramer, Tuschen-Caffier, & Heinrichs, 2014).

Visual Search Task.

In this task, NBCS and IKIDS children viewed 48 trials comprising an emotional face (angry, sad, or happy) paired with three different neutral faces of the same person. Each emotional face appeared in 16 trials, which were presented in a random order. Each test trial lasted for 2,000 ms. The same procedures used in the free-viewing task for drawing the child’s attention to the center of the screen before each trial and for drift check correction were repeated in this task. Two eye-tracking variables were calculated for each trial: (a) detection probability of emotional faces, defined as whether the child fixated on the emotional face, and (b) detection latency of the emotional face for trials in which the child detected the emotional face. The purpose of the visual search task was to determine whether and how quickly children detected an emotional face when it was shown among other neutral faces. Higher detection probability and shorter detection latencies of emotional faces, compared with neutral faces, suggest a greater attentional vigilance to emotional stimuli.

Child Behavior Checklist.

Mothers reported on children’s symptoms of behavior problems using the CBCL/1½-5 (Achenbach & Rescorla, 2000). The CBCL/1½-5 consists of 100 items, each describing behaviors that are symptoms of externalizing behavioral problems, such as aggression and acting out, and of internalizing behavioral problems, such as anxiety and social withdrawal. The child’s mother was asked to rate how frequently they observed each of these behaviors in the child within the past two months using the following scale: 0 = absent, 1 = occurs sometimes, and 2 = occurs often. Children’s internalizing behavioral problems are measured by summing up the scores from four subscales: 11 items measuring emotionally reactive behaviors (e.g., “Whining,” “Shifts between Sad-Excited”), 8 items measuring anxious/depressed behaviors (e.g., “Nervous”), 11 items measuring somatic complaints (e.g., “Aches”), and 8 items measuring socially withdrawn behaviors (e.g., “Withdrawn” and “Avoids Eye Contact”). Our analysis focused on the scores in the “anxious/depressed” and the “withdrawn” subscales (i.e., sum of scores for items in each subscale) because of previous work suggesting an association between children’s attention biases to emotional information and anxious or socially withdrawn behaviors (Lisk, 2020; Morales, Beekman, Blandon, Stifter, & Buss, 2015; Morales, Perez-Edgar, et al., 2015). The total scores can be categorized within the normal range, the concerned range, or the clinical range based on cutoff scores and percentiles from the normative sample (Achenbach & Rescorla, 2000). The normative study to validate the CBCL/1½-5 has integrated data collected from children from multiple and diverse United States (U.S.) populations (Achenbach & Rescorla, 2000). The CBCL/1½-5 also has been widely used in research and clinical practice with children from various populations living in rural areas and outside the U.S., including children from South America (e.g., Peru, Chile), Asia (e.g., Taiwan, China), the Middle East (e.g., Iran, Israel), and Europe (e.g., Finland, Denmark, France) (Hope & Bierman, 1998; Ivanova et al., 2010; Kariuki, Abubakar, Murray, Stein, & Newton, 2016). However, this measure, to our knowledge, has not been used or validated in an Indigenous population in the U.S. Thus, in the current study, we evaluate the factor structure of the internalizing behavior problem scale and its four subscales (i.e., emotionally reactive, anxious/depressed, withdrawn, somatic complaints) using data from the NBCS sample.

Data Preparation

Eye-tracking data were extracted using the EyeLink Data Viewer software, version 3.248. Given that we were interested in measuring attention to faces and consistent with previous research (Nozadi, Spinrad, et al., 2018), areas of interest (AOI) for both tasks were selected to be around the faces in the Data Viewer software. The AOIs are regions of the displayed stimulus for which the researcher is interested in obtaining eye-movement variables (Hunter, Roland, & Ferozpuri, 2020). Variables of interest (e.g., detection latency, duration of first fixation and dwell time on each face) were chosen based on previous eye-tracking studies measuring attention in children, with fixation defined as the time (in milliseconds) during which the child’s gaze was relatively stable on an AOI (i.e., a face) (Carter & Luke, 2020). Trials were excluded if (a) the child did not look at any of the faces (i.e., had no fixations on any faces) or (b) the first fixation on either of the AOIs occurred less than 80 ms from the stimulus onset, indicating that the child was already looking at the area where the face was about to appear, instead of focusing on the central attention-getting stimulus. The 80-ms threshold was selected based on previous research indicating that this period gives enough time for face processing (Armstrong et al., 2010; Nozadi, Spinrad, et al., 2018). These exclusion criteria were applied to both eye-tracking tasks. Approximately 10% and 13% of trials in the free-viewing and visual search tasks (respectively) were excluded for NBCS sample, compared to 8.8% and 8.6% of trials in IKIDS sample across free-viewing and visual search tasks.

Statistical Analyses

We first compared the descriptive statistics for demographics across the two samples using the Chi-Squared and Fisher’s Exact tests for the categorical variables and independent two-sample t-tests for the continuous variables. Next, a series of linear mixed-effects models were run to examine general patterns of attention bias toward emotional versus neutral faces within the same trial, and patterns of attention bias toward angry and sad faces compared with happy faces across trials using data from each cohort separately. The goal of these analyses was to evaluate whether the descriptive patterns of attention biases were similar across cohorts and to determine if we replicated the general patterns of attention to emotional faces reported in previous research. Because some children did not fixate on emotional faces in all of the trials, mixed effects models allowed us to include data from all trials, regardless of whether the child had looked at the emotional face or not, while taking into account the unequal numbers of trials per child. Indeed, use of linear mixed modeling has been recommended for handling eye-tracking data with inherent variability and noisiness to increase the statistical power and precision (Toker, Conati, Steichen, & Carenini, 2013).

All attention bias indicators, excluding detection probability, were continuous outcomes. For continuous outcomes, linear mixed models (LMM) were conducted using maximum likelihood method by the function “lme” from R package ‘nlme’. For detection probability, a generalized linear mixed model (GLMM) was conducted using penalized quasi-likelihood approach by the “glmmPQL” function from R package ‘MASS’. The distribution of reaction time measures were skewed. Thus, all reaction time measures were transformed using the square root transformation procedure before the model fitting.

The regression model started with a full model that included all covariates, i.e., age, sex, emotion type, child’s birth weight, maternal age, maternal education, maternal employment status, and household annual income. The stepwise backward selection procedure was conducted for variable selection, which led to a final working model. The Akaike Information Criterion (AIC) value guided the variable selection for LMM fitting such that variables were removed until the lowest AIC was attained. The variables retained in the final working models included emotion type, age and sex. For the GLMM that fitted the outcome of probability of face detection in the visual search task, the stepwise backward selection only retained the predictors with statistical significance less than 0.2. Age and sex were included as covariates because differences have been reported in some studies with more consistent differences based on age than sex in attention biases, indicating that children’s biases toward emotionally charged faces may increase with age (Stuijfzand et al., 2020).

After testing patterns of attention biases in each cohort, cohort differences in attention biases toward emotional versus neutral faces and negative versus positive faces were examined using a combined data set from the two cohorts. In these analyses, age and sex again were included as covariates while controlling for the cohort effect and the interaction between cohort and emotion type. The two cohorts have very different age distributions. The median age in NBCS was 4.27 years old ranging from 3.55 - 5.65 and covering a two-year range, while the median age in IKIDS was younger with all children being clustered around 4 years old. Thus, in all models, we considered the interaction term between age and cohort.

Because the CBCL/1½-5 has never been used with Navajo children, we tested (a) whether CBCL/1½-5 items related to the internalizing subscales measure the same constructs in Navajo children and (b) whether the four syndrome subscales loaded onto the construct of internalizing behavior problems as proposed in the general population (Achenbach & Rescorla, 2000). These evaluations were necessary before examining associations between CBCL/1½-5 subscales and attention biases in NBCS and were conducted using a series of confirmatory factor analysis (CFA) and Mplus software, version 8.5 (Muthén, 1998-2017). The Standardized Root Mean Square Residual (SRMR), Root Mean Square Error of Approximation (RMSEA), and Comparative Fit Index (CFI) were used to evaluate the fit of CFA models. Values lower than .08 for SRMR and RMSEA and greater than .90 for CFI indicate an adequate model fit (Hu, 1999; Kline). A Diagonally Weighted Least Squares estimation was used for confirmatory factor analyses to test the factor structure of the subscales, which involved the categorical variables, and a Robust Maximum Likelihood (RML) estimation was used for the CFA analysis of the internalizing behavior problems scale with the continuous variables (i.e., raw continuous scores for each subscale).

In addition to testing the factor structure of CBCL/1½-5 subscales, the internalizing behavior problem scale, and related subscales in the NBCS, we examined the equivalence of factor structures for the internalizing behavior problems subscale in two cohorts (configural invariance model) to assess whether the same items load on the same construct. The configural invariance models, the least strict invariance models, were run using multi-group CFA analyses in Mplus (IKIDS was coded as 0 and NBCS coded as 1) and the fit of model(s) was tested using the previously mentioned fit indices. Because of small sample sizes for the two cohorts resulting in low variability in responses, we only considered testing for the invariance model at the configural level.

Because the full measurement invariance beyond testing for the similarity of factor structure (i.e., testing for the equivalence of factor loadings and intercepts) were not examined, associations between CBCL/1½-5 subscale scores and indicators of attention bias were conducted using data from each cohort separately. In these analyses, CBCL/1½-5 scores on the “withdrawn” and “anxious/depressed” subscales were treated as the outcome in models as previously conceptualized (Britton et al., 2015). Indicators of attention bias toward each emotion were computed by averaging the values across trials for each child and were used as predictors in the models. Because use of mixed-effect modeling was not plausible as it pertains to repeated measure outcomes (Hoffman & Rovine, 2007), regression analyses were conducted to determine associations between attention biases and CBCL/1½-5 scores. Because of the large variability in attention bias summary scores compared with CBCL/1½-5 scores, we considered implementing a piecewise regression analysis if the diagnostic test suggested a non-linear association between CBCL/1½-5 scores and attention biases. Using the piecewise regression analysis allowed us to examine whether the slope for testing the association was homogeneous for all possible ranges of values or if there were breakpoints in the data at which the slopes differed across different ranges of response (Wainer, 1971). Depending on the variability of linear associations, one or segmented linear models were fitted by specifying various breakpoints on the range of responses. Davies’ tests were employed to examine whether slopes in each model were non-constant. If the Davies’ test p value was > 0.1, a general linear regression model was fitted and otherwise a piecewise regression model was fitted.

Results

Descriptive statistics for the demographic variables for the eye-tracking samples from two cohorts are reported in Table 1. Compared with the NBCS children, IKIDS children came from families with higher incomes and higher maternal and paternal education levels, ps <.001. Although the majority of women were married, a larger number of mothers in the NBCS samples were single (ps <.001). Additionally, the majority of women from the NBCS cohort were unemployed, whereas the majority of IKIDS mothers were employed (p < .001). Children in the NBCS also had a slightly lower gestational age and birthweight and were older than children in IKIDS, ps <.01. Mothers in the NBCS were also younger than mothers in the IKIDS, and higher percentage of mothers in the NBCS were not married or living with a partner compared to mothers in the IKIDS, ps <.001.

Table 1.

Demographic Characteristics of Participants in the NBCS and IKIDS Eye-Tracking Samples.

Variables	NBCS (n = 125)	IKIDS (n = 70)	p-value
Annual Income			< 0.001
Less than $20,000	70 (56%)	2 (2.9%)
$20,000-$39,000	14 (11.2%)	5 (7.1%)
$40,000-$69,000	10 (8%)	13 (18.6%)
$70,000 and above	5 (4%)	50 (71.4%)
Missing	26 (20.8%)	0 (0%)
Maternal Education			< 0.001
Below high school	34 (27.2%)	0 (0%)
High school	65 (52%)	0 (0%)
Associate degree	11 (8.8%)	7 (10%)
Bachelor’s degree or higher	3 (2.4%)	63 (90%)
Missing	12 (9.6%)	0 (0%)
Paternal Education			<.001
Below high school	24 (19.2%)	0 (0%)
High school	68 (54.4%)	1 (1.4%)
Associate degree	11 (8.8%)	13 (18.6%)
Bachelor’s degree or higher	2 (1.6%)	56 (80%)
Missing	20 (16%)	0 (0%)
Marital Status			<.001
Not married nor living with a partner	26 (20.8%)	1 (1.4%)
Married or living with a partner	89 (71.2%)	69 (98.6%)
Missing	10 (8%)	0 (0%)
Mother Employment			<.001
Employed	35 (28%)	58 (82.9%)
Unemployed	80 (64%)	12 (17.1%)
Missing	10 (8%)	0 (0%)
Child’s Birth Weight			0.0014
Mean (SD), lb	7.3 (1.08)	7.88 (1.12)
Median [Min, Max], lb	7.28 [4.34 - 10.27]	7.97 [4.94 - 10.78]
Missing	14 (11.2%)	10 (14.3%)
Child’s Gestational Age at Birth			< 0.001
Mean (SD), weeks	38.89 (1.62)	39.79 (1.4)
Median [Min, Max], weeks	39 [34 - 42]	39.71 [36.00 - 42.14]
Missing	14 (11.2%)	1 (1.4%)
Childs’ Age at Assessment			< 0.001
Mean (SD), week	51.96 (5.5)	48.12 (0.44)
Median [Min, Max], year	51.25 [42.61, 67.79]	48.22 [47, 48.77]
Missing	0 (0%)	0 (0%)
Maternal Age at Birth			< 0.001
Mean (SD), years	27.88 (6.03)	31.95 (3.64)
Median [Min, Max], years	27.02 [16.1 - 43.12]	32.15 [23.11, 41.26]
Missing	10 (8%)	0 (0%)

Note. Data shown are n (%) except where indicated. IKIDS = Illinois Kids Development Study; NBCS = Navajo Birth Cohort Study. Chi-square tests and two-sample t-tests were used for comparing the distributions of categorical or continuous variables (respectively) between NBCS and IKIDS samples.

Attrition Analyses

The NBCS children included in our analysis (n = 125) were compared to all NBCS children at the time of enrollment who had demographic data (n = 491) in terms of family’s annual income, maternal and paternal education, marital status, maternal employment, maternal age at the time of childbirth, child’s gestational age and birth weight. No significant differences were found between the two groups as indicated by chi-square tests and two-sample t-tests, ps > .14.

Results from attrition analysis for the IKIDS cohort showed that children included in our analyses (n = 70) had a father with a slightly higher education level compared to children in the original cohort (n = 495). Specifically, 80% fathers of IKIDS children included in our analyses had Bachelor’s or a graduate degree compared to only 68.9% of fathers in the original cohort (p=.043). Further, the mean for gestational age at birth was slightly higher for children who were included in our analyses (Mean = 39.79) compared to children in the original cohort (Mean = 39.22), p <.01. No other differences were found between the IKIDS sample in our analyses compared to the original cohort.

General Patterns of Attention Biases

Patterns of Attention biases Towards Emotional Versus Neutral Faces

Free-Viewing Task.

Results from the mixed models for testing the patterns of attention biases toward emotional faces compared with neutral faces (first four columns of Table 2) showed that the patterns were mostly comparable across the two cohorts, despite differences in culture and demographic characteristics. In the free-viewing task, children in both samples had longer dwell times and shorter detection latencies for emotional versus neutral faces (ps < .001), suggesting that children were more attentive to emotional versus neutral faces. The dissimilarity in patterns of attention biases was observed only for the length of first fixation in the free-viewing task. While NBCS children had longer first fixations for all three emotional faces compared with neutral faces (ps < .01), children in the IKIDS cohort only had longer first fixations for angry faces.

Table 2.

Examination of Attention Bias Patterns Toward Emotional Faces Compared With Neutral Faces and Attention Bias Patterns Toward Negative Faces Versus Happy Faces.

	Emotional Versus Neutral Faces						Emotional Versus Happy Faces
		NBCS		IKIDS		Similar Pattern	NBCS		IKIDS		Similar Pattern
	Variables	Estimate	SE	Estimate	SE		Estimate	SE	Estimate	SE
Free-Viewing Task
Dwell Time	Intercept	17.64***	2.12	−14.30	24.13		22.28***	2.51	−34.65	31.78
	Age	0.06	0.04	0.75	0.50		0.04	0.05	1.21	0.66
	Sex	0.20	0.44	0.16	0.42		−0.02	0.51	0.18	0.56
	Happy	3.42***	0.35	2.10***	0.38	✓
	Angry	3.87***	0.36	3.71***	0.38	✓	0.43	0.43	1.61***	0.46
	Sad	4.45***	0.35	2.55***	0.38	✓	1.01**	0.42	0.45	0.45
Face Detection Latency	Intercept	27.84***	1.47	35.35*	17.60		25.26***	1.80	36.81	27.09
	Age	−0.04	0.03	−0.18	0.37		−0.03	0.03	−0.23	0.56
	Sex	−0.12	0.30	0.30	0.31		−0.24	0.37	0.03	0.47
	Happy	−2.06***	0.28	−1.23***	0.29	✓
	Angry	−1.79***	0.28	−1.02***	0.29	✓	0.28	0.32	0.19	0.34	✓
	Sad	−1.75***	0.27	−1.33***	0.29	✓	0.32	0.31	−0.08	0.34	✓
First Fixation Duration	Intercept	22.18***	1.71	36.30	25.41		24.19***	1.89	36.86	28.56
	Age	−0.06	0.03	−0.38	0.53		−0.08*	0.04	−0.39	0.59
	Sex	0.45	0.35	0.27	0.45		0.73	0.39	−0.08	0.50
	Happy	0.93***	0.22	0.21	0.22
	Angry	1.39***	0.23	0.77***	0.22	✓	0.45	0.28	0.57*	0.27
	Sad	0.69**	0.22	0.36	0.22		−0.26	0.27	0.15	0.27	✓
Visual Search Task
Face Detection Latency	Intercept	27.03***	1.11	−2.43	16.93		27.40***	1.84	8.72	24.98
	Age	0.01	0.02	0.66	0.35		0.00	0.03	0.42	0.52
	Sex	0.50*	0.24	0.10	0.31		0.26	0.39	0.91*	0.45
	Happy	−0.39	0.32	−0.31	0.42	✓
	Angry	−0.68*	0.32	−0.20	0.41		−0.28	0.40	0.10	0.54	✓
	Sad	−0.30	0.31	0.60	0.41	✓	0.10	0.40	0.91	0.53	✓
Probability of Face Detection	Intercept	−1.46***	0.38	3.74	5.50		−1.56**	0.49	7.13	7.83
	Age	0.03***	0.01	−0.07	0.11		0.04***	0.01	−0.14	0.16
	Sex	0.15	0.08	0.03	0.10		0.25*	0.10	0.12	0.14
	Happy	0.37***	0.06	0.27**	0.08	✓
	Angry	0.41***	0.06	0.37***	0.08	✓	0.05	0.08	0.09	0.11	✓
	Sad	0.47***	0.06	0.39***	0.08	✓	0.10	0.08	0.12	0.11	✓

Note. IKIDS = Illinois Kids Development Study; NBCS = Navajo Birth Cohort Study. Except for ‘Visual Search: Probability of Face Being Detected’ all the other outcome variables were transformed by square root calculation before the modelling analyses. The significance is indicated by *p < .05, **p < .01, ***p < .001. The estimates for ‘Sex’ variables are for females (male is the reference value). The estimate of a covariate for ‘Visual Search: Probability of Face Being Detected’ is on the degree of logit of the probability.

Visual Search Task.

In the visual search task, the overall pattern for detection probabilities across two samples was the same such that the detection probabilities of emotional faces were higher than neutral faces across both samples (ps < .001). The general patterns for the detection latencies in the visual search task were not the same. While NBCS children were quicker to detect angry faces that were paired with neutral faces in the visual search task (p <.05), this result was not replicated for children in the IKIDS sample.

Patterns of Attention Biases Towards Negative Versus Happy Faces

Free-Viewing Task.

A comparison of negative (i.e., angry and sad) faces to positive faces (last four columns of Table 2) showed that patterns of attention biases across the two samples for the detection latency were again similar. In both samples, no differences were found in detection latencies for negative versus happy faces. However, the general patterns of dwell times and first fixation durations on negative faces, when paired with happy faces, were not the same across samples. While NBCS children spent more time looking at sad versus happy faces, IKIDS children had longer dwell times on angry versus happy faces (ps< .01). IKIDS children also had longer first fixations on angry versus happy faces (p< .01), but this result was not replicated in the NBCS sample.

Visual Search Task.

The general descriptive patterns of attention bias indicators were the same across samples. Specifically, no differences were found between detection probabilities and detection latencies of negative versus happy faces, ps >.10.

Age and Sex Differences in Attention Biases

For the NBCS sample, older age predicted shorter first fixations on negative faces versus positive faces (p < .05). Additionally, older age was associated with higher detection probabilities of negative faces versus positive faces and emotional faces versus neutral faces (ps <.001). The same effects were not found in IKIDS sample. In the NBCS sample, females also had higher detection probabilities of negative faces versus happy faces and had quicker detection of negative versus neutral faces in the visual search task (ps < .05).

In the IKIDS sample, female children had a quicker detection of negative versus positive faces (p < .05). No other age or sex differences were found in attention bias indicators among the IKIDS children.

Comparative Analysis for Detecting Cohort Differences in Attention Biases

Results from analyses testing cohort differences in indicators of attention biases during the free-viewing and visual search tasks are presented in Tables 3 and 4, respectively. In each table, the first four columns show estimates for attention biases toward emotional faces versus neutral faces, followed by estimates for attention biases toward negative faces versus positive faces. Three cohort differences were observed for attention bias indicators during the free-viewing task (two of which were related to differences in the magnitude of bias) but no cohort differences were observed for indicators of attention bias during the visual search task.

Table 3.

Cohort Differences for Attention Biases Toward Emotional Faces Versus Neutral Faces and Negative Faces Versus Positive Faces During the Free-Viewing Task.

		Emotional Versus Neutral Faces			Negative Versus Positive Faces
Outcome	Covariate	Estimate	Std.Error	t.value	Estimate	Std.Error	t.value
Dwell Time	(Intercept)	−14.10	24.07	−0.59	−34.20	31.73	−1.08
	Cohort: NBCS vs. IKIDS	31.79	24.14	1.32	56.40	31.80	1.77
	Age	0.74	0.50	1.48	1.20	0.66	1.83
	Gender: Female vs. Male	0.17	0.30	0.57	0.07	0.38	0.18
	Happy	2.10***	0.40	5.23
	Angry	3.71***	0.40	9.21	1.61***	0.48	3.33
	Sad	2.55***	0.40	6.36	0.45	0.48	0.94
	Int: (NBCS vs. IKIDS) on Happy	1.32*	0.53	2.51
	Int: (NBCS vs. IKIDS) on Angry	0.17	0.53	0.31	−1.18	0.64	−1.85
	Int: (NBCS vs. IKIDS) on Sad	1.90***	0.53	3.61	0.56	0.63	0.89
	Int: (NBCS vs. IKIDS) on Age	−0.68	0.50	−1.36	−1.16	0.66	−1.76
Face Detection Latency	(Intercept)	36.43*	17.63	2.07	37.52	27.07	1.39
	Cohort: NBCS vs. IKIDS	−8.82	17.68	−0.50	−12.34	27.11	−0.46
	Age	−0.20	0.37	−0.55	−0.25	0.56	−0.44
	Gender: Female vs. Male	0.08	0.22	0.39	−0.14	0.29	−0.47
	Happy	−1.23***	0.31	−3.99
	Angry	−1.02***	0.31	−3.33	0.19	0.35	0.54
	Sad	−1.33***	0.31	−4.28	−0.08	0.36	−0.22
	Int: (NBCS vs. IKIDS) on Happy	−0.84*	0.41	−2.06
	Int: (NBCS vs. IKIDS) on Angry	−0.77	0.41	−1.86	0.09	0.47	0.19
	Int: (NBCS vs. IKIDS) on Sad	−0.43	0.41	−1.05	0.40	0.47	0.86
	Int: (NBCS vs. IKIDS) on Age	0.16	0.37	0.43	0.21	0.56	0.38
First Fixation Duration	(Intercept)	36.35	25.37	1.43	35.32	28.79	1.23
	Cohort: NBCS vs. IKIDS	−14.10	25.41	−0.55	−10.81	28.84	−0.37
	Age	−0.39	0.53	−0.73	−0.36	0.60	−0.60
	Gender: Female vs. Male	0.38	0.28	1.36	0.43	0.31	1.38
	Happy	0.21	0.24	0.87
	Angry	0.77**	0.24	3.20	0.57	0.30	1.90
	Sad	0.36	0.24	1.48	0.15	0.30	0.50
	Int: (NBCS vs. IKIDS) on Happy	0.72*	0.32	2.26
	Int: (NBCS vs. IKIDS) on Angry	0.62	0.32	1.90	−0.12	0.40	−0.29
	Int: (NBCS vs. IKIDS) on Sad	0.33	0.32	1.02	−0.41	0.39	−1.04
	Int: (NBCS vs. IKIDS) on Age	0.33	0.53	0.62	0.28	0.60	0.46

Note. IKIDS = Illinois Kids Development Study; NBCS = Navajo Birth Cohort Study. Except for ‘Visual Search: Probability of Face Being Detected’ all the other outcome variables were transformed by square root calculation before the modelling analyses. The significance is indicated by *p < .05, **p < .01, ***p < .001. The estimates for ‘Cohort’ are for NBCS (IKIDS was coded as zero and NBCS was coded as 1). The estimates for ‘Sex’ variables are for females (male is the reference value). The estimate of a covariate for ‘Visual Search: Probability of Face Being Detected’ is on the degree of logit of the probability.

Table 4.

Cohort Differences for Attention Biases Toward Emotional Faces Versus Neutral Faces and Negative Versus Positive Faces During the Visual Search Task.

		Emotional Versus Neutral Faces			Negative Versus Positive Faces
		Visual Search Task
Outcome	Covariate	Estimate	Std.Error	t.value	Estimate	Std.Error	t.value
Face Detection Latency	(Intercept)	−3.56	16.95	−0.21	9.63	26.27	0.37
	Cohort: NBCS vs. IKIDS	30.74	16.98	1.81	17.49	26.32	0.66
	Age	0.68	0.35	1.94	0.40	0.55	0.74
	Gender: Female vs. Male	0.35	0.19	1.86	0.52	0.30	1.74
	Happy	−0.31	0.45	−0.69
	Angry	−0.20	0.44	−0.46	0.10	0.56	0.18
	Sad	0.60	0.43	1.39	0.90	0.56	1.61
	Int: (NBCS vs. IKIDS) on Happy	−0.08	0.54	−0.15
	Int: (NBCS vs. IKIDS) on Angry	−0.47	0.54	−0.88	−0.39	0.69	−0.56
	Int: (NBCS vs. IKIDS) on Sad	−0.91+	0.53	−1.71	−0.81	0.68	−1.19
	Int: (NBCS vs. IKIDS) on Age	−0.67+	0.35	−1.90	−0.40	0.55	−0.73
Probability of Face Detection	(Intercept)	3.39	5.52	0.61	6.73	7.86	0.86
	Cohort: NBCS vs. IKIDS	−4.80	5.53	−0.87	−8.25	7.88	−1.05
	Age	−0.07	0.11	−0.58	−0.13	0.16	−0.81
	Gender: Female vs. Male	0.10	0.06	1.60	0.21*	0.08	2.49
	Happy	0.27**	0.08	3.30
	Angry	0.37***	0.08	4.43	0.09	0.11	0.83
	Sad	0.39***	0.08	4.73	0.12	0.11	1.09
	Int: (NBCS vs. IKIDS) on Happy	0.10	0.10	1.00
	Int: (NBCS vs. IKIDS) on Angry	0.05	0.10	0.49	−0.04	0.13	−0.29
	Int: (NBCS vs. IKIDS) on Sad	0.08	0.10	0.81	−0.02	0.13	−0.13
	Int: (NBCS vs. IKIDS) on Age	0.09	0.11	0.82	0.17	0.16	1.03

Note. IKIDS = Illinois Kids Development Study; NBCS = Navajo Birth Cohort Study. Except for ‘Visual Search: Probability of Face Being Detected’ all the other outcome variables were transformed by square root calculation before the modelling analyses. The significance is indicated by *p < .05, **p < .01, ***p <. 001. The estimates for ‘Cohort’ are for NBCS (IKIDS was coded as zero and NBCS was coded as 1). The estimates for ‘Sex’ variables are for females (male is the reference value). The estimate of a covariate for ‘Visual Search: Probability of Face Being Detected’ is on the degree of logit of the probability.

Cohort differences in the free-viewing task were as follows. Although in both samples children had longer dwell times and shorter detection latencies for emotional versus neutral faces, the dwell times on happy and sad faces were slightly longer and latencies to detect happy faces were shorter in NBCS children compared to IKIDS children. In other words, while the patterns of attention biases were the same in both cohorts, sensitivity to attend to specific emotional faces was stronger in NBCS than IKIDS children. Further, NBCS children had slightly longer first fixations on happy versus neutral faces, a pattern not replicated in IKIDS children.

We conducted two sets of sensitivity analyses to examine whether older aged children or the presence of children with elevated social withdrawal scores in the NBCS sample contributed to differences across the two samples. Specifically, in the first set of analyses, we removed NBCS children who were categorized into the clinical/concerned group from the combined data set. In the second set of analyses using the combined data set, we only included NBCS children who were in the same age range as children in the IKIDS. The results of these analyses were comparable to the results shown in Tables 3 and 4, suggesting that a wider age range and the presence of a greater proportion of CBCL/1½-5 “withdrawn” children did not account for significant cohort differences in attention biases.

Factor Structures of the CBCL/1½-5 Internalizing Behavior Problems Scale and Related Subscales

CBCL Descriptive Summaries and Reliabilities.

The individual item frequencies for the internalizing behavior problems subscales (e.g., “withdrawn”, “anxious/depressed”) showed that only a small percentage of parents in IKIDS reported the highest categories for the subscale items (i.e., category 1 and 2, indicating that a behavior “occurs sometimes,” and “occurs often,” respectively) compared with parents in the NBCS sample. For example, less than 1.5% of IKIDS parents reported the highest categories for 7 out of 16 items that measure socially withdrawn and anxious behaviors. None of the IKIDS parents reported the highest categories for two items, item 43 (“Little interest”) from the “withdrawn” subscale and item 71 (“Looks unhappy”) from the “anxious/depressed” subscale, compared with 11% and 37% of NBCS parents who reported the highest categories for these two items, respectively. Supplemental Table S1 summarizes the proportions of children in NBCS and IKIDS whose scores exceeded CBCL/1½-5 “clinical” or “concerned” categorical cut-offs that were derived from the normative sample for syndrome subscales.

The internalizing behavior problem scales showed good reliability in both samples indicated by estimating Cronbach Alphas (αs > .80 in both samples). While the reliabilities of withdrawn subscale in NBCS and anxious withdrawn subscale in IKIDS were adequate (αs>.70), reliabilities for anxious/depressed subscale in NBCS (α = .65) and withdrawn subscale in IKIDS (α = .49) were less acceptable. The “withdrawn” subscale was not considered for IKIDS in further analysis.

Confirmatory Factor Analysis For CBCL Internalizing Behavior Problem Scale.

Given the low rate of concerns reported for the subscale items and unacceptable reliability of “withdrawn” subscale in IKIDS, CFAs were conducted only for the NBCS sample to evaluate the factor structures of subscales that involved item-level data. Supplemental Table S2 shows the results from CFAs examining the factor structure for the internalizing behavior problems scale in both cohorts and the subscales for NBCS. The fit of resulting measurement models to test the factor structures of internalizing behavior problems in each sample was good, as indicted by fit indices. In the NBCS sample, the fit of measurement models for “withdrawn” and “emotionally reactive” subscales were also good.

Testing Measurement Invariance for CBCL Internalizing Behavior Problems Across Two Samples

Testing measurement invariance models for the “withdrawn” and “anxious/depressed” subscales across the two cohorts was not viable because some items within these two subscales were not reported by any of the IKIDS parents, and we could only test for the configural invariance of the internalizing behavior problems scale (involving continuous scores from subscales) across cohorts. The fit of this model was adequate (CFI = 1.00; RMSEA = .00; and SRMR = .01), suggesting that that the higher order domain of internalizing behavior problems underlying the four syndrome scales was the same across the two cohorts, as was the basic factor structure of the scale.

Associations Between Attention Bias Indicators and CBCL/1½-5 Scores

Given that only the NBCS cohort showed variability in scores for the “withdrawn” subscale with a larger proportion of NBCS children in the “clinical” or “concerned” range, the association between CBCL/1½-5 and indicators of attention biases was assessed only for NBCS children. The results from piecewise regression analyses are presented in Table 5. The Davies’ tests indicated non-linear associations between CBCL/1½-5 socially withdrawn scores and (a) dwell times on emotional faces versus neutral faces (free-viewing) and (b) detection latencies and probabilities (visual search). Therefore, for these models, piecewise regression models were conducted with one breakpoint estimating slopes for high versus low response ranges. Results showed that for NBCS children with the highest range of socially withdrawn scores, the increase in socially withdrawn behaviors was associated with a decrease in average dwell times on happy and sad faces (relative to neutral faces), suggesting a pattern of attentional avoidance to emotional faces. Further, for NBCS children with the highest range of socially withdrawn scores, an increase in socially withdrawn behaviors was associated with an increase in average detection latency for angry versus neutral faces. A significant association with the detection probability for angry faces was also suggested, but the estimate was small (0.03 with a 95% confidence interval of 0- 0.05).

Table 5.

Association Between Indicators of Attention Bias and CBCL/1½-5 Withdrawn Score in the NBCS Cohort.

	Happy	Angry	Sad
FV: Dwell Time	0-1.2: 68.7 (−18.68, 156.08) 1.2-14: −26.58 (−42.38, −10.78)	0-14: −6.75 (−18.43, 4.93)	0-2.7: 23.61 (−15.56, 62.77) 2.7-14: −23.12 (−44.05, −2.19)
FV: Latency to Detect Face	0-14: 5.37 (−11.61, 22.35)	0-14: 1.94 (−14.87, 18.75)	0-14: −3.58 (−20.25, 13.09)
FV: First Fixation Duration	0-14: −2.78 (−11.53, 5.97)	0-14: 8.69 (−1.04, 18.42)	0-14: 1.11 (−8.36, 10.57)
VS: Latency to Detect Face	0-14: 5.05 (−12.18, 22.29)	0-4: −16.75 (−59.19, 25.69) 4–14: 41.12 (3.81, 78.42)	0-14: −0.04 (−18, 17.92)
VS: Probability of Face Detection	0-14: 0 (−0.02, 0.01)	0-4.2: 0.03 (0, 0.05) 4.2-14: −0.03 (−0.07, 0)	0-14: 0 (−0.01, 0.02)

Note. NBCS = Navajo Birth Cohort Study. In a table cell, the numbers before the colon are the range of ‘Withdrawn’ scores, and the estimated slope with a 95% confidence interval follows the colon.

Discussion

The goal of the current study was to examine the cross-cultural applicability of eye-tracking technology in assessing attention biases toward emotional stimuli in two samples of preschool children between (Means _age = 51.96 and 48.12) from very different racial, cultural, and demographic backgrounds, one of which (the Navajo sample) had never been previously included in this type of research. Building upon previous research on children’s attention to emotional information, we used faces as our stimuli. The results showed that, for most indicators of attention bias, children in both samples showed comparable patterns of attention to emotional faces and the qualitative differences between samples were negligible. Consistent with existing hypotheses and replicating findings from previous studies (Fu & Perez-Edgar, 2019; Peltola, 2009), both groups of children spent more time dwelling on emotional faces versus neutral faces. Children across both groups also were more likely and faster to detect emotional faces than neutral faces and had longer first fixations on angry faces versus neutral faces. Cohorts showed significant differences in a few indicators of attention biases toward emotional faces versus neutral faces as measured during the free-viewing task but not during the visual search task. Specifically, the observed effects for dwell times and detection latencies when viewing emotional faces versus neutral faces were more pronounced in NBCS children than IKIDS children, although the general patterns were the same in both cohorts. Further, NBCS children had longer first fixations on happy faces, an effect consistent with findings from previous studies but not found for IKIDS children. Lastly, in NBCS children, higher symptoms of socially withdrawn behaviors were associated with attentional avoidance of emotional faces. Given the comparability of patterns for most components of attention biases across two independent samples, our results suggest that eye-tracking, and specifically the eye-tracking tasks used in this study with preschool-aged children, provides a non-biased tool for assessing normative attention biases toward emotional faces in young children across different populations. The two tasks used in the current study were relatively short, taking about 10 minutes to complete, and have the potential flexibility to be administered using a portable eye-tracker. Therefore, these tasks can be incorporated in studies that use a variety of settings and among populations that have been historically underrepresented in this line of research (e.g., children from culturally, linguistically and geographically diverse populations) to examine children’s development of typical and atypical attention biases and their roles in anxiety problems in laboratory, clinical, and field settings.

Previous studies of infants and young children have shown that processing emotional information over neutral information is prioritized given the significant survival and developmental functions of emotionally charged information (LoBue, 2017; Mastorakos & Scott, 2019). This pattern of attention biases pertains to both non-social (e.g., objects) and social stimuli (e.g., faces), particularly when stimuli signify a threat, such as a “snake” or “angry faces.” Using two independent samples, we replicated previous findings by showing that children from two cohorts were more vigilant toward the detection of emotional faces and spent more time processing emotional faces over neutral faces. In addition, our findings extend previous research by testing a more diverse sample and expanding the age range of the children tested. Specifically, we tested this hypothesis in a wider age range group (3.5-5.5 years of age) across two different populations with substantially different cultural, lifestyle, and demographic characteristics.

Despite similarities in general patterns of attention biases across the two samples, three cohort differences were observed between NBCS and IKIDS children, two of which were related to magnitude of attention bias. Although children in both samples, had longer dwell times on and shorter detection latencies for emotional faces relative to neutral faces, NBCS children had longer dwell times on happy and sad faces and showed quicker detection of happy faces compared to IKIDS children. Further, NBCS children had longer first fixation on happy faces versus neutral faces, but this effect was not found with IKIDS children. Differences between the two cohorts, such as having children with a broader age range and more socially withdrawn behaviors in the NBCS cohort, did not account for this significant cohort difference in attention biases. Thus, other behavioral, sociocultural, or demographic variables not measured in the study could be explored in future research in relation to these attention biases. For example, one explanation for increased attention to emotional faces in Navajo children can be due to potential differences in emotion recognition across samples, which we did not assess in the current study. It is possible that NBCS children spent more time looking at emotional faces due to novelty of non-native faces portrayed by actors in the stimuli set. Although the NimStim set contained facial expressions portrayed by actors from different ethnicities (e.g., Black, Asian, Hispanic), indigenous faces are not included in this set. Another alternative explanation for cohort differences may be due to potential differences in social display of emotional expressions. There is considerable evidence to show that culture affects perceptual processing of emotions through several mechanisms including impact on norms for social display of emotional expressions (Engelmann & Pogosyan, 2013; Park & Huang, 2010). For example, in collectivistic cultures (e.g., Indigenous cultures) with an emphasis on group’s well-being, the explicit social display of emotions (particularly negative emotions) may be prohibited (Crivelli & Gendron, 2017; Crivelli, Jarillo, Russell, & Fernandez-Dols, 2016). Thus, individuals from collectivistic cultures may take longer time to encode and process facial expressions portrayed by individuals outside their cultures, as these facial expressions may appear novel. These cultural differences in emotion recognition and links to attention biases should be investigated in future research.

In the NBCS cohort, which included children from a wider age range, age was associated with indicators of attention biases. In older children, detection probabilities increased for emotional and negative faces relative to neutral and happy faces. Yet, increase in age was related to quicker disengagement from emotional faces following face detection in Navajo children, reflecting a faster processing speed of emotional faces. These age differences for different aspects of attention biases, including a higher probability of face detection at the onset of stimulus presentation and shorter first fixation, perhaps reflect changes that occur in children’s executive function during these early years (Jones, 2003). As children age, they become more aware of various emotions and their consequences, yet they have more ability to regulate, redirect, and control their attention. Thus, although older children are more likely to detect emotional faces faster, they are also able to redirect their attention to regulate their moods.

For the NBCS sample, which had a larger proportion of children with elevated scores outside the normal range on the CBCL’s withdrawn subscale, we were able to examine associations between indicators of attention biases and children’s high versus low withdrawn scores. This association could not be conducted in the IKIDS sample due to low variability in responses for items measuring socially withdrawn behaviors. Children with higher degrees of mother-reported withdrawn behaviors showed an attentional avoidance pattern for angry, sad, and happy faces, characterized by spending less dwell time on happy and sad faces and longer detection time on angry faces. The strategic redirection of attention as a regulatory mechanism has been observed in children with early anxiety and withdrawn symptoms (Brown et al., 2013; Stirling, 2006) and has been thought to protect them against the development of anxiety disorders later in life. In fact, some studies have shown that only a subgroup of children with early risk factors who do not develop adequate regulatory skills may end up having attention biases and anxiety later in childhood (Morales, Perez-Edgar, & Buss, 2016). Thus, our results are consistent with previous findings of an association between withdrawn behaviors and attention biases.

The current study has several strengths, such as (a) the inclusion of two samples with substantial differences in demographic, cultural, and lifestyle characteristics, and (b) the inclusion of Indigenous children living in rural areas and tribal lands who have not been traditionally included in developmental and clinical research in more than population-representative percentages (i.e. 1-2% of normative samples). Additionally, we were able to replicate some previous reports on what are considered adaptive patterns of attention biases using two relatively short eye-tracking measures across two independent cohorts, suggesting that these two measures can be used in future research to study attention biases. However, small cohort differences also were observed that may challenge the idea of universality of adaptive attention biases. This should be further investigated using other diverse samples and particularly outside the U.S. to examine the extent to which adaptive patterns of attention biases are truly pan-cultural.

The current study has several limitations that should be noted. First, we only relied on a single parent-reported questionnaire, the CBCL, to assess children’s anxiety and socially withdrawn behaviors. Previous studies on associations between anxiety and attention biases in children, albeit limited, have utilized several measures, including clinician-administered measures, to assess children’s anxiety. Second, the CBCL/1½-5 has never been validated in a Navajo or Indigenous sample to our knowledge. The results from a series of CFAs showed that CBCL/1½-5 items relating to “withdrawn” and “anxious/depressed” subscales loaded on the same constructs in the NBCS as those in the general population and that the basic factor structure of internalizing behavior problems was the same across the two cohorts. However, we were not able to test for the equivalence of specific subscales (i.e., withdrawn and anxious/depressed subscales) across two cohorts or to test for all of the different types of measurement invariance given the small sample sizes and low variability of responses in the IKIDS cohort. Despite finding similar CBCL/1½-5 general factor structures in the NBCS sample and samples from other studies, we cannot draw any confident conclusion about the validity of the CBCL/1½-5 in the Navajo population. Hence, our results involving associations between CBCL/1½-5 and attention biases in the NBCS should be interpreted with caution because it is possible that categorizing more children in the clinical/concerned group was due to cultural norms rather than clinical problems. An analysis of CBCL/1½-5 subscales and scales using data from all ECHO cohorts, including NBCS, which is currently underway, can elucidate the functionality of this measure in the Navajo population compared with other cohorts. Lastly, very few children across both samples, and specifically in the IKIDS sample, had high scores in the “anxious/depressed” and “withdrawn” subscales. Thus, future research on this topic should focus on examining the associations between attention biases and anxiety/socially withdrawn status in a group of children who have been referred for services and/or have been diagnosed with anxiety.

Nevertheless, the results of this study showed that patterns of attention biases were mostly comparable across two very different samples. One of these samples included children from a rural Indigenous population, who has been historically segregated from the rest of U.S. population, both geographically and economically. Despite a few cohort differences, mostly related to stronger attention biases towards emotional faces, our result suggest that eye-tracking, may provide robust and non-biased measures of attention biases in young children across cultures. Understanding how adaptive patterns of attention biases may develop during early childhood using eye-tracking, which does not rely on behavioral and verbal responses from children, provides a basis for better understanding of what components of attention biases can be universal and their roles in anxiety disorders.

Data Availability Statement

The datasets for this manuscript are not publicly available because, per the NIH-approved ECHO Data Sharing Policy, ECHO-wide data have not yet been made available to the public for review/analysis. Requests to access the datasets should be directed to the ECHO Data Analysis Center, ECHO-DAC@rti.org.