An investigation of Head Start preschool children's executive function, early literacy, and numeracy learning in the midst of the COVID-19 pandemic

Abstract

The COVID-19 pandemic's impact on preschool children's school readiness skills remains understudied. This research investigates Head Start preschool children's early numeracy, literacy, and executive function outcomes during a pandemic-affected school year. Study children (N = 336 assessed at fall baseline; N = 237-250 assessed in spring depending on outcome; fall baseline sample: mean age = 51 months; 46% Hispanic; 36% Black Non-Hispanic; 52% female) in a network of Head Start centers in four states (Nevada, New Jersey, Pennsylvania, and Wisconsin) experienced low in-person preschool exposure compared to national pre-pandemic norms. Children experienced fall to spring score gains during the pandemic-affected year of 0.05 SD in executive function, 0.27 SD in print knowledge, and 0.45-0.71 SD in early numeracy skills. Descriptively, for two of the three early numeracy domains measured, spring test score outcomes were stronger among children who attended more in-person preschool. We discuss implications for future research and policy.

1. Introduction

Beginning in 2020, the COVID-19 global pandemic precipitated abrupt and influential changes in the lives of young children. There is growing concern among researchers and policymakers that the pandemic may have harmed young children's development and acquisition of key school readiness skills (Bassok et al., 2021). Many children either experienced disruptions to their early care and education (ECE) participation during COVID-19, or they stopped attending or never began ECE at all. Even when ECE centers began reopening after initial closures, their enrollments were substantially lower than they were pre-pandemic, particularly for low-income, Black, and Hispanic children (Barnett & Jung, 2021; Bassok & Shapiro, 2021; Weiland et al., 2021; Zhang et al., 2023).

Little research has directly assessed how preschoolers’ academic skills and executive function development have fared during the pandemic. The current research contributes to the evidence base by investigating the early literacy, numeracy, and executive function outcomes of children in Head Start preschools during the COVID-19 outbreak. Our study examined the following questions:

• 1.To what extent did children in Head Start preschools experience growth in early literacy, numeracy, and executive function during the COVID-19 outbreak?
• 2.How was in-person attendance at Head Start preschool centers during COVID-19 associated with early literacy, numeracy, and executive function development during the pandemic?

We leveraged a unique dataset that includes fall and spring direct child assessments of Head Start preschoolers conducted via a virtual assessment methodology, as well as attendance data. With this study, we aimed to present among the first empirical research based on direct child assessments of preschoolers’ development that was conducted during the height of the COVID-19 pandemic.

The current research makes three key contributions. First, the setting is important and understudied. Head Start programs have provided federally-funded free child care to over 37 million low-income children in the United States to date, and the United States spends over $10 billion annually on Head Start and Early Head Start (National Head Start, 2022). Research on how COVID-19 influenced Head Start participants is quite limited, despite the fact that Head Start enrollees are a critical population of predominantly Black, Hispanic, and low-income children whose communities faced disproportionate pandemic impacts. Second, most prior studies of the impact of COVID-19 in ECE settings have relied on teacher and parent surveys, which are subject to self-report bias. Very few studies have directly assessed preschoolers’ learning during COVID-19 (for an exception, in executive function, see Ahmed et al., 2022), and to our knowledge, few or none have data from more than one point in time that would allow an examination of growth. We used direct child assessments conducted in both fall and spring of the pandemic-affected school year 2020-21, as described below. Third, while informative research has explored preschool absenteeism in non-pandemic school years (e.g., Ansari & Purtell, 2018), the volume of in-person preschool that children have missed during the pandemic dwarfs the levels examined in prior research. Thus, new research specifically using attendance data and addressing the pandemic period is needed to examine how children's development progressed under a regimen of substantially reduced access to preschool.

The following section situates the current investigation in the context of the existing literature base. This section is organized as follows. First, a review of the literature on the importance of the outcomes examined in this study, namely, preschoolers’ early literacy skills, numeracy skills, and executive function, is presented. Second, we review the pre-pandemic research literature on the role of preschool absences in children's learning and development. Lastly, we describe the specific context of COVID-19, contextualizing how children from minoritized racial and ethnic communities and low-income backgrounds may have been disproportionately affected by the pandemic.

1.1. The importance of early academic skills and executive function for preschoolers

Early literacy and numeracy skills are important for both kindergarten readiness and later performance in reading and mathematics, two core school subjects (e.g., Clements et al., 2013; Lonigan, Burgess, & Anthony, 2000). Federal policy guidelines also highlight the value of these skills for building mathematics and literacy competencies in the preschool years (e.g., National Mathematics Advisory Panel, 2008; U.S. Department of Health and Human Services, 2015).

Among the most important predictors of children's reading skills is early print knowledge, defined as knowledge of the way print is organized (Language and Reading Research Consortium & Chiu, 2018; Piasta et al., 2012). Print knowledge is comprised of pre-reading concepts including knowledge of letters, words, and conventions of books and printed materials (Sawyer et al., 2014; Storch & Whitehurst, 2002). Competencies contained under print knowledge include understanding the direction that text is read, distinguishing the cover of a book, knowing the purpose of punctuation, and identifying letters of the alphabet and their sounds (Wilson & Lonigan, 2009). Preschoolers’ print knowledge as measured by the Test of Preschool Early Literacy (TOPEL; Lonigan et al., 2007) is significantly correlated with later-grades reading skills, as well as phonological awareness, which is linked to decoding skills (Lonigan & Farver, 2008).

Early numeracy is another core domain of preschoolers’ development (Clements & Sarama, 2007; Duncan et al., 2007; Watts et al., 2014). Number sense is an essential dimension of children's mathematical skills, pertaining to the knowledge and capacities to make sense of numbers, operations, and number relations (NCTM, 2000). A range of skills developed in early childhood contribute to number sense development (Hojnoski et al., 2022), including, among others, identifying written forms of numerals (Geary, 2006), comparing the magnitudes of quantities (Jordan et al., 2007), and counting in sequence (Gelman & Gallistel, 1986). Preschoolers’ number sense skills are important predictors of their subsequent mathematics performance. Early measures of oral counting are correlated with mathematics achievement in later grades (Aunola et al., 2004; Laracy et al., 2016), and early magnitude comparison skills have been shown to predict later performance in fact retrieval and arithmetic (Desoete et al., 2012). Meanwhile, early challenges with identifying written numbers have been identified as a risk factor for mathematical difficulties (Gersten et al., 2005).

Executive function comprises a set of capacities involved in the regulation of thoughts and behaviors (Baggetta & Alexander, 2016), which support a range of academic skills, as well as future behavioral and social outcomes (e.g., Blair & Razza, 2007; Evans & Rosenbaum, 2008; Moffitt et al., 2011). Three of the core components of executive function are inhibitory control, referring to the ability to modulate or override one's automatic or dominant responses, actions, and/or thoughts; cognitive flexibility, referring to the ability to shift back and forth among multiple tasks, ideas, or goals; and working memory, referring to the ability to maintain specific information in mind, while simultaneously and quickly incorporating new information or discarding irrelevant information (Baggetta & Alexander, 2016; Miyake & Friedman, 2012). Children's executive function skills in pre-K and preschool are associated with their performance in subsequent grades in domains including mathematics and literacy (e.g., Fuhs, Nesbitt, Farran, & Dong, 2014; Fuhs, Nesbitt, Farran, & Dong, 2014; McClelland et al., 2013). These links may be driven in part by the ways that executive functioning supports the development of positive learning behaviors, which are useful for future academic tasks (Nesbitt et al., 2015).

1.2. The role of preschool absences in children's learning and development

The importance of school attendance for student learning has been well-documented (e.g., Gottfried, 2009, 2010), and reducing absenteeism is increasingly recognized as a public policy goal (FutureEd, 2017; Schanzenbach et al., 2016). In theory, strong attendance in the earliest grades may be expected to ease children's transition into formal schooling, as participating in consistent daily school routines may pave the way for better adjustment and learning in elementary school (Anderson & Romm, 2020). A growing body of research has documented important links between preschool attendance and students’ school performance in the later grades. For example, using data from pre-K programs operated under the Chicago Public Schools, Ehrlich, Gwynne, & Allensworth (2018) found that children with more pre-K absences tended to have lower scores on kindergarten readiness measures of mathematics, letter recognition, pre-literacy, and behavior in spring of the pre-K year. In another study conducted with preschoolers in rural Appalachia, children with higher attendance rates were found to have significantly better end-of-year literacy skills (Rhoad-Drogalis & Justice, 2018). At the national level, research using nationally representative data on Head Start participants found that missing days of Head Start preschool was linked to lower fall to spring gains in both mathematics and literacy (Ansari & Purtell, 2018).

Studies have come to different conclusions regarding whether specific student and classroom characteristics moderate relationships between preschool attendance and later school outcomes. In some cases, preschool attendance appeared more consequential for children from low-income families (Anderson & Romm, 2020; Fuhs, Nesbitt, & Jackson, 2018; Fuhs, Nesbitt, & Jackson, 2018) for children with lower levels of baseline skills (Ansari & Purtell, 2018; Ehrlich, Gwynne, & Allensworth, 2018), and in classrooms with high instructional quality (Ansari & Purtell, 2018; Fuhs, Nesbitt, & Jackson, 2018). On the other hand, Ehrlich et al. (2018) did not find differential associations between pre-K absences and end-of-year outcomes depending on children's neighborhood poverty level nor racial/ethnic background, and Rhoad-Drogalis and Justice (2018) found that instructional quality was not a significant moderator of relationships between attendance and literacy skills.

Overall, the finding of a negative link between absences and achievement gains is consistent with literature documenting the importance of school exposure for learning (e.g., Raudenbush & Eschmann, 2015). As such, lower-than-typical exposure to Head Start during the pandemic may have been expected to predict weaker academic skills outcomes.

1.3. The learning and development of preschoolers from low-income and minoritized racial and ethnic communities during the COVID-19 pandemic

Policy underinvestment and the effects of racism have long operated to constrain opportunities for low-income children and children from minoritized racial and ethnic communities (Massey & Denton, 1993). Discriminatory housing and employment policies, among other issues, have contributed over time to reduced wealth accumulation in Black, Latinx, and low-income communities (e.g., Bertrand & Mullainathan, 2004; Green et al., 2021). These factors in turn inhibit access to neighborhoods with well-resourced schools (e.g., Blair & Raver, 2016; Duncan & Murnane, 2011; Osher et al., 2020). Meanwhile, families often work to marshal resources to counteract these inequities, drawing upon social networks, family relationships, and community organizations such as religious, cultural, and civic associations to enrich their children's academic achievement and well-being (e.g., Chin & Phillips, 2004; DiMaggio & Garip, 2012; Osher et al., 2020; Yoshikawa, 2011).

We note that it is not obvious to what extent we would expect preschool-aged children participating in Head Start to experience learning and developmental setbacks during the pandemic. On the one hand, there are several reasons to predict reduced learning for young children during COVID-19, particularly in low-income and racially minoritized communities. First, as described above, high preschool absenteeism has been identified as a general risk factor for weaker learning outcomes in pre-pandemic research studies, with some studies suggesting more negative effects of absenteeism among preschoolers from low-income families or with lower baseline skills; thus, the substantially higher absence rates seen during COVID-19 could be expected to predict decreased academic outcomes for Head Start participants.

More specific to the pandemic, spells of virtual learning may have contributed to placing young children at academic risk. According to teachers, preschoolers struggled under virtual learning due to inadequate instructional provisions, parents’ difficulties supervising online preschool while they worked, preschoolers’ need for physical activity and difficulty sitting in front of computers, and the difficulty preschoolers have learning social and school readiness skills when not in-person (Bassok et al., 2021). Such conditions could also have harmed children's executive functioning outcomes, which are sensitive to access to high-quality instructional interactions (Fuhs, Nesbitt, & Jackson, 2018). One signal of the potential for social-behavioral setbacks due to the pandemic is that parents reported worsened home behaviors during remote learning (Hanno et al., 2022).

Meanwhile, systemic inequities affecting children from low-income and minoritized racial and ethnic backgrounds already existed prior to the pandemic, including caregivers’ disproportionate representation in jobs that did not allow for remote work options (Goldman et al., 2021) and limited access to high-quality health care (Abedi et al., 2021). Such inequities resulted in minoritized communities being disproportionately affected by the pandemic (Solomos, 2021). As a result, individual families and children in these communities were likely exposed to added stresses and challenges. On the other hand, it is conceivable that various influences could have ameliorated learning risks to preschoolers during the pandemic, such as local regulations that allowed ECE centers to remain open more frequently than public schools (e.g., Grimm, 2020), and/or intensive compensatory efforts by parents and ECE teachers to bolster preschoolers’ learning while they were at home. Thus it is unclear but important to know how Head Start preschoolers’ learning and development have fared during the pandemic's progression.

2. Method

2.1. Participants

Study participants were a sample of children enrolled in a network of Head Start centers located across four states (Nevada, New Jersey, Pennsylvania, and Wisconsin) during the 2020-21 school year. As we described previously in (Thomas et al., 2022), data collection proceeded as follows. We originally set the goal of assessing up to 30% (or 600) of all Head Start children across the four study locations. Program staff at the organization that operates the centers used the random function in Excel to randomly select 600 students. Of these, baseline (fall 2020) assessments were ultimately collected for 336 children. The analytic samples for the listwise deletion analyses ranged between 242-250 when examining the associations between exposure to in-person schooling and academic learning outcomes. The number of children assessed was lower than that recruited for several reasons. Scheduling difficulties for virtual assessment appointments during this phase of the pandemic were substantial, as centers and classrooms frequently closed due to COVID-19 outbreaks and local public health ordinances, and children were frequently absent due to COVID illnesses and exposure-induced quarantines. In addition, as is frequently the case in research studies, a number of parents/caregivers declined participation in assessments both initially and throughout the year, while other children withdrew from the Head Start centers and were not subsequently assessed. We could not and did not track specific reasons for parents/caregivers’ nonparticipation during pandemic data collection.

Table 1 presents descriptive information for the sample of children who completed baseline assessments; comparisons between the full enrollee population at the participating Head Start centers, the students randomly selected for assessments, and the completed baseline assessment sample; as well as comparisons to nationally representative Head Start samples. Children in the baseline assessment sample were identified by their caregivers predominantly as Black (36%) or Hispanic (46%). Mean age was 50.82 months. Most children's (91%) highest level of parent education was less than a bachelor's degree; and 35% had a home language besides English (see Table 1 for other sample statistics).

Table 1.

Descriptive statistics for study demographics.

	Enrollee Population Mean	Random Sample Mean	Final Samplea				National Sampleb
			Mean	SD	Min	Max
Female	0.52	0.50	0.52	0.50	0	1	0.50
Black Non-Hispanic	0.36	0.41	0.36	0.48	0	1	0.33
Hispanic	0.51	0.40	0.46	0.50	0	1	0.35
White Non-Hispanic	0.07	0.10	0.07	0.26	0	1	0.20
Asian Non-Hispanic	0.04	0.04	0.05	0.22	0	1	0.02
Other Race Non-Hispanic/Race Missing	0.02	0.03	0.05	0.21	0	1	0.10
Age (Months)	51.45	51.39	50.82	6.07	36	60	–
Age 3 or younger	0.43	0.37	0.29	0.46	0	1	0.49
Age 4 or older	0.57	0.63	0.71	0.46	0	1	0.51
Home language not English	0.39	0.31	0.35	0.48	0	1	0.21
Parent Education
< High School	0.20	0.17	0.19	0.40	0	1	0.22
High School Diploma	0.40	0.42	0.39	0.49	0	1	0.34
Some College / AA Degree	0.32	0.32	0.32	0.47	0	1	0.34
BA +	0.09	0.08	0.09	0.29	0	1	0.10
Single Parent Household	0.60	0.64	0.64	0.48	0	1	0.64
Number of Days Attended In Person	–	–	88.69	53.63	0	168	–
N	1823	600	336

^aFinal sample refers to the sample of children that completed baseline assessments (N = 336). Complete data were available for demographic and attendance variables.

^bAll statistics in this column except parent education are from Kopack Klein et al. (2021) and weighted to represent all children enrolled in Head Start in fall 2019. Parent education data are weighted to represent all children enrolled in Head Start in fall 2014 (Kopack Klein et al., 2018).

Compared with a nationally representative Head Start sample, study participants in the baseline assessment sample were similarly likely to be Black non-Hispanic or Asian non-Hispanic, more likely to be Hispanic, less likely to be White non-Hispanic, and more likely to have a home language besides English. Study children's fall executive function scores were 0.10 SD higher than the national mean among children enrolled in Head Start in fall 2019, based on data reported in Kopack Klein et al. (2021) (see Table 2 for descriptive statistics for all child assessments). Study children were also somewhat more likely to be age 4 than age 3 as compared to the national pre-pandemic sample; Table 1 suggests this was likely driven in part by random sampling variability, although it also may have been the case that parents of 4-year-olds (who were often program returners from the prior year) were more likely to enroll their children in Head Start than parents of 3-year-olds during the pandemic. As we describe below, all child assessment scores were scaled to be age-adjusted, such that each child's gain scores may be compared to typical growth of a same-age norming sample; thus comparisons of conclusions about preschoolers’ score growth across studies are not expected to be substantially affected by small variations in the age compositions of the studies’ samples.

Table 2.

Descriptive statistics for child assessments.

	Final Samplea
	Mean	SD	Min	Max	% Missing
Print Knowledge, Fall	94.79	14.66	70	144	6%
Print Knowledge, Spring	99.82	14.60	65	144	20%
Oral Counting, Fall	11.42	9.95	0	60	2%
Oral Counting, Spring	18.86	14.83	0	85	18%
Number Naming, Fall	16.13	14.41	0	63	3%
Number Naming, Spring	23.24	17.44	0	60	18%
Quantity Comparison, Fall	7.58	4.39	0	20	2%
Quantity Comparison, Spring	10.45	4.76	0	22	18%
Executive Function, Fall	97.17	8.89	61	126	7%
Age 3 and Under	96.23	9.39	61	120
Age 4 and Over	97.56	8.67	61	126
Executive Function, Spring	98.34	6.39	80	114	18%
Age 3 and Under	97.34	6.47	81	113
Age 4 and Over	98.77	6.33	80	114

^aFinal sample refers to the sample of children that completed baseline assessments (N = 336).

2.2. Measures

2.2.1. Child outcomes

We assessed print concepts understanding as an indicator of early literacy using the Print Knowledge subtest of the Test of Preschool Early Literacy (TOPEL; Lonigan et al., 2007), which captures alphabet knowledge and knowledge of written language conventions and forms. The Print Knowledge subtest comprises 36 multiple-choice items. Assessors shared with children a digital version of the test flipbook using Zoom's screen share feature. Assessors read each prompt aloud, asking children to select the best response (e.g., “Which is the letter /k/?”). Scores were converted to standard scores that permit same-age peer comparisons. Internal consistency reliability (Cronbach's alpha) for the analytic sample was α = 0.96 for the fall assessment and α = 0.95 for the spring assessment. According to Lonigan et al. (2007), the TOPEL assessment was validated for use with Black, Hispanic, and White samples of children.

Early numeracy was measured using three subtests of the Individual Growth and Development Indicators of Early Numeracy (IGDI-ENs; Hojnoski & Floyd, 2012). The Oral Counting subtest indexes the child's ability to count fluently. Children are asked to count aloud in sequence beginning at one; the score is the highest number the child reaches in one minute before making an error. Number Naming measures the ability to identify numerals. Children are asked to identify the numerals 1 to 20, presented in random order; the score is the number correctly identified. Quantity Comparison indexes the ability to judge differences in the quantity of object groups. Children are shown a series of images which each display two dice faces showing 1-6 dots and asked to identify the die ‘with more.’ The child's score equals the number of images identified correctly within one minute. As above, digital flipbooks and the Zoom screen share feature were used to conduct the assessments. Raw scores were used in analyses.

Cronbach's alpha for the analytic sample for the Quantity Comparison subtest was α = 0.90 for the fall assessment and α = 0.90 for the spring assessment; traditional sample internal consistency reliability statistics were not estimable for the Oral Counting and Number Naming subtests due to the way scores for these assessments are constructed; however, publisher-reported test-retest reliabilities for these measures ranged from 0.71-0.88. Prior research comparing remote and in-person administration of IGDI assessments showed that the two assessment modalities yielded similar scores (Greenwood et al., 2022); however, note that the research was conducted with infants and toddlers using different subtests. The IGDI-EN has demonstrated validity evidence for use in measuring numeracy growth in Head Start (Floyd et al., 2006), including for Black children (Hojnoski et al., 2009), although we are unaware of specific validity data for Hispanic children.

We assessed executive function, including working memory, inhibitory control, and cognitive flexibility, using the Minnesota Executive Function Scale (MEFS; Carlson & Zelazo, 2017). MEFS has been used in prior federally-funded Head Start research (Kopack Klein et al., 2021). MEFS is an adaptive assessment administered via a tablet app (publisher-reported test-retest reliability = 0.93; as above, sample internal consistency reliabilities were not estimable given the nature of the reported scores). Assessors read prompts aloud to children, and asked children to select one of two options that best matched the prompt (for example, “We are going to play the color game. This is a blue box and this is a green box. All green bicycles go in the green box, all blue bicycles go in the blue box. Where does this one go?”). Scores were derived from the publishers’ guidelines for age-adjusted scores that permit same-age peer comparisons. MEFS has been validated for use with Head Start preschoolers identified as Black, Hispanic, and White (Nguyen et al., 2022).

2.2.2. Exposure to in-person preschool

Attendance and child demographic data were collected from administrative files. We operationalized children's exposure to in-person Head Start preschool as number of days attended per calendar month, calculated by dividing the total number of days the student was present in-person by their total months enrolled in the program.

Beyond routine absences, children were absent in the focal school year for several pandemic-specific reasons. First, children missed in-person days due to parents’ preferences for virtual learning. Children whose parents opted for virtual learning at the beginning of the fall semester were enrolled in virtual Head Start programming. The virtual programming, which was the same across the four states, comprised weekly one-on-one teacher-child Zoom meetings, and encouragement to access a website with educational links. Parents could change their programming preference (virtual versus in-person) whenever they chose, and many parents did so. In centers where more parents expressed interest in in-person enrollment than there were spaces available, children were placed on a waitlist. Waitlisted children began the year in virtual learning, and were offered an in-person spot as soon as one became available. Second, classrooms closed intermittently due to virus outbreaks and public health ordinances. Third, children were excluded from school when they were being quarantined. Head Start records do not permit us to parse the relationships between various reasons for child absences and student outcomes separately; this notwithstanding, we view absences for all reasons as contributors to the construct of interest: Children's ultimate level of exposure to in-person preschool during the pandemic-affected school year 2020-21. In a sensitivity check, analyses using raw days of in-person attendance yielded the same pattern of findings as those below.

2.3. Procedures

2.3.1. Virtual assessments

Child-level literacy, numeracy, and executive function assessments in fall and spring were conducted one-on-one via a virtual assessment model (see Authors [2022] for a detailed discussion). At the beginning of the school year, Head Start centers provided tablets and Internet hotspots to all families that needed them. The assessments we used are traditionally administered in-person. Due to public health restrictions in place during data collection, we used a virtual model for all assessments. All participating children, including both those attending Head Start in-person and those attending virtually (i.e., at home), were administered the assessments remotely. Children attending preschool in-person at assessment time completed the assessments using Internet-connected devices in their classrooms. Those enrolled in the virtual model at the time of the assessment completed the assessments via Internet-equipped devices at home. Trained assessors, all of whom were remote while conducting the assessments, used the screen share feature in Zoom to show children publisher-provided digital versions of the test materials.

All assessments were conducted in English. Two Spanish-speaking assessors conducted all assessments with children whose home language was Spanish. The Spanish-speaking assessors conducted assessments in English, but could clarify directions or converse with parents in Spanish as needed. For the children whose home languages were other than English or Spanish (N = 24), translators who were trained to translate instructions verbatim to the child were present for assessments.

2.3.2. Analytic approach

We begin by presenting descriptive information on children's in-person Head Start attendance, as well as fall and spring school readiness skills, during the pandemic-affected school year. We present both raw scores as well as effect sizes translated to percentile ranks using a calculation from Lipsey et al. (2012).

To examine the relationship between preschool exposure and learning outcomes, we fit a series of regression models predicting children's spring outcomes in early print knowledge, numeracy, and executive function, respectively, as a function of their in-person attendance, controlling for fall scores and demographic indicators. Our primary models were fit with center fixed effects and robust standard errors, and controlled for child demographics including race, gender, age, English as a home language, parent education, single parent household, and assessment location (home versus school). In our primary specifications, we fit all models using multiple imputation to account for missing data (Royston & White, 2011). We imputed values for variables with any percentage of missingness (executive function, numeracy, and literacy scores from the spring and fall; and whether the student was in-person or virtual during the spring semester) using multiple imputation by chained equations (MICE, Royston & White, 2011). We generated 20 datasets containing imputed values, incorporating information from auxiliary variables without missingness (gender, race/ethnicity, age, home language, parent education, single parent household indicator, and number of days of Head Start attended in-person). We then obtained estimates from the 20 imputations using Rubin's combination rules (Rubin, 1987); the resulting estimates are shown in our main modeling results.

We conducted a series of sensitivity checks to examine the robustness of the findings to missing data and modeling variations. First, we refit all models using listwise deletion in place of multiple imputation. Second, we fit models including a dichotomous indicator indexing children who attended no in-person days, to examine potential differences for never-attenders on the measured outcomes, controlling for baseline scores and the other demographic indicators noted previously. Lastly, we refit the above models using negative binomial regression, a common approach for over-dispersed count data (Hilbe, 2011) such as in the case of pandemic attendance. It is important to note that student attendance is endogenous in most research studies and contexts (e.g., Liu et al., 2021) and as we described above, the pandemic was no exception. We could not and did not document the reasons for different absences. Thus the data we have do not permit us to draw causal conclusions about the effects of pandemic preschool attendance. Rather, our aim is to illuminate growth patterns and predictors of child academic and executive function outcomes within a descriptive framework.

3. Results

3.1. In-person preschool attendance

Head Start preschoolers in the baseline achievement sample who began the year in virtual learning attended an average of 6.31 days per month of in-person Head Start, while those who began the year enrolled in the in-person model attended an average of 12.66 days per month. Among study children who attended in-person for at least one day during the school year, mean days of in-person preschool was 11.93 days per month. Fig. 1 plots the distribution of monthly in-person attendance for each of those groups.

Fig. 1.

Histogram of In-Person Preschool Days per Calendar Month Attended among Children who Began the School Year in In-Person Learning (blue); Virtual Learning (Red); and those who Attended at Least One Day of In-Person Preschool (‘Ever-attenders’) (Green)

3.2. Did Head Start preschoolers experience gains in school readiness skills during the pandemic-affected school year?

Descriptively, examining the sample of children with complete raw (non-imputed) fall and spring assessments for each measure, children experienced mean gains in school readiness skills during the pandemic-affected school year (Table 3 ). In print knowledge, children achieved fall to spring gains that were 0.27 SD greater than expected for their age, with mean scores increasing from the 40^th percentile of the test norming sample in the fall to the 50^th percentile at spring posttest. In early numeracy, children gained an average of 0.71 SD in oral counting, 0.45 SD in number naming, and 0.71 SD in quantity comparison. In executive function, children experienced growth that was 0.05 SD higher than expected for their age, with average scores increasing from the 44^rd percentile of the assessment norming sample in the fall to the 46^th percentile in the spring.

Table 3.

Descriptive results of children's academic achievement and executive function during the pandemic-affected school year.

	Child Outcomes
	Fall Score Mean (SD)	Spring Score Mean (SD)	Effect Size
Print Knowledge	96.12 (14.91)	100.11 (14.35)	0.27a
Oral Counting	11.74 (10.29)	19.02 (15.21)	0.71b
Number Naming	16.81 (14.95)	23.51 (17.43)	0.45b
Quantity Comparison	7.28 (4.53)	10.48 (4.72)	0.71b
Executive Function	97.68 (7.95)	98.42 (6.45)	0.05a

Notes: Statistics are from the sample of children with complete fall and spring scores for each assessment.

^aFall-spring score difference divided by the assessment norming sample SD (15).

^bFall-spring score difference divided by fall sample SD.

3.3. Associations between in-person preschool exposure and child outcomes during the pandemic-affected year

Table 4 provides the results of multivariate analyses predicting spring child outcomes as a function of in-person attendance, controlling for prior scores and background characteristics as well as center fixed effects and using multiple imputation to account for missing data. The results of these models indicate that on average, children who attended more in-person preschool during the pandemic year had significantly better learning outcomes in print knowledge (b = 0.48; p < 0.05), oral counting (b = 0.42; p < 0.05), and number naming (b = 0.61; p < 0.01) than those who attended fewer days of preschool in-person. Spring outcomes in executive function and quantity comparison were not significantly associated with exposure to in-person preschool in the multivariate models.

Table 4.

Multivariate results of child outcomes as a function of in-person (versus Virtual) preschool attendance during the pandemic-affected school year, using multiple imputation.

	Child Outcomes
	Print Knowledge	Oral Counting	Number Naming	Quantity Comparison	Executive Function
In-person preschool attendance	0.480*	0.421*	0.609**	-0.002	0.097
N	336	336	336	336	336

Note. Raw (unstandardized) coefficients shown. Separate linear regression models were estimated for each outcome, using multiple imputation. All models controlled for center fixed effects and child demographics including race, gender, age, English as a home language, parent education, single parent household, and assessment location (home versus school). Models also controlled for child's fall scores on the five academic and executive function outcomes. Robust standard errors displayed. + p < 0.10, * p < 0.05, ** p < 0.01, *** p < 0.001.

Next, we describe the results of sensitivity checks conducted to assess the sensitivity of the primary findings to model variations and missing data. Table 5 shows that refitting the primary models using listwise deletion rather than multiple imputation yields results similar to those above. Table 6 shows the results of adding to the main models a dummy indicator indexing children who did not attend any days of in-person programming, to examine the extent to which these children differed on the measured outcomes from their counterparts, controlling for baseline scores and the other indicators. The coefficient on this indicator is statistically significant only for the oral counting variable. When this indicator is added to the models, the associations between oral counting and number naming with in-person attendance remain statistically significant; the coefficient for print knowledge remains positive in sign but is no longer statistically significant. Models refit with negative binomial regression and site fixed effects did not achieve convergence; the pattern of findings from models refit using negative binomial regression trimmed of site fixed effects is similar to that observed in the model including the indicator for non-attenders (see Table 7 ). Although these differences may be stochastic given the data limitations inherent in the relatively small sample, they are consistent with a conclusion that the association between attendance and print knowledge may have been driven by weaker print knowledge outcomes among children who never attended in-person, while associations between in-person attendance with oral counting and number naming were robust to the inclusion of children with no in-person attendance days.

Table 5.

Multivariate results of child outcomes as a function of in-person (versus Virtual) preschool attendance during the pandemic-affected school year, using listwise deletion.

	Child Outcomes
	Print Knowledge	Oral Counting	Number Naming	Quantity Comparison	Executive Function
In-person preschool attendance	0.467*	0.459+	0.830**	0.051	0.013
N	242	247	247	247	250

Note. Separate linear regression models with center fixed effects were estimated for each outcome. All models controlled for child demographics including race, gender, age, English as a home language, parent education, single parent household, and assessment location (home versus school). Models also controlled for child's fall scores on the five academic outcomes. Robust standard errors displayed. + p < 0.10, * p < 0.05, ** p < 0.01, *** p < 0.001.

Table 6.

Multivariate results of child outcomes as a function of in-person preschool attendance during the pandemic-affected school year, controlling for a dichotomous indicator indexing whether the child never attended Head Start in-person.

	Child Outcomes
	Print Knowledge	Oral Counting	Number Naming	Quantity Comparison	Executive Function
In-person preschool attendance	0.145	0.659***	0.503*	-0.047	0.051
Never-attender	1.578	5.884*	1.897	-0.744	-0.337
N	242	247	247	247	250

Note. Unstandardized regression coefficients shown. Separate linear regression models were estimated for each outcome. All models controlled for child demographics including race, gender, age, English as a home language, parent education, single parent household, and assessment location (home versus school). Models also controlled for child's fall scores on the five academic and executive function outcomes. Robust standard errors displayed. + p < 0.10, * p < 0.05, ** p < 0.01, *** p < 0.001.

Table 7.

Replication of multivariate results of child outcomes as a function of in-person preschool attendance during the pandemic-affected school year, using negative binomial regression.

	Child Outcomes
	Print Knowledge	Oral Counting	Number Naming	Quantity Comparison	Executive Function
In-person preschool attendance	0.077	0.299*	0.559**	-0.019	0.065
N	242	247	247	247	250

Note. Marginal effects shown. Separate linear regression models were estimated for each outcome. All models controlled for child demographics including race, gender, age, English as a home language, parent education, single parent household, and assessment location (home versus school). Models also controlled for child's fall scores on the five academic and executive function outcomes. Robust standard errors displayed. + p < 0.10, * p < 0.05, ** p < 0.01, *** p < 0.001.

We also explored the potential for heterogeneous results by fitting models examining the interactions between attendance and child race/ethnicity; of the 20 hypothesis tests conducted, none of the interaction terms were statistically significant. Due to small sample sizes, interaction models are underpowered, and caution is warranted in interpretation. Finally, we fit interaction models to explore whether relationships between in-person attendance and outcomes differed significantly by baseline skills; none of the interactions examined were statistically significant.

4. Discussion

Overall, study children in Head Start experienced a fraction of the in-person preschool that would have been expected in a pre-pandemic year based on prior research. This was true even among children who began the year in in-person learning. Ansari and Purtell (2018) found that Head Start children in a nationally representative sample in 2009-10 were absent for an average of 5.48% of the school year, or the equivalent of attending roughly 170 days in a hypothetical 180-day school year. Extrapolating this calculation to the current sample, children in the present study who began the year in in-person learning would have attended the equivalent of roughly 111 in-person preschool days in a hypothetical 180-day school year, while those who began the year in virtual learning would have attended approximately 55 days.

Despite relatively low levels of in-person preschool attendance, children experienced mean gains in school readiness skills over the course of the year. Data did not exist to compare growth during the pandemic to prior growth for the study children. For the purposes of discussion, we attempted to locate pre-pandemic research studies of growth seen across the preschool year on the same assessments in Head Start or other predominantly low-income samples. We emphasize that comparisons between growth seen in our sample and that reported in these pre-pandemic studies are imprecise. However, in the absence of direct comparands for our sample in pre-pandemic data, we present them as broadly suggestive of how the trends we observed may have compared to others seen in earlier research.

Specifically, we make reference to recent studies that assessed preschoolers in Head Start or other predominantly low-income settings, and used the TOPEL, IGDI, and MEFS measures. For print knowledge, the Head Start network involved in the current study assessed children on the same measure (TOPEL) during the most recent pre-pandemic school year (2018-2019), allowing us to compare growth for our sample to this prior year sample. Using the same calculation methods as above, we found that during 2018-19, children enrolled in the same network of Head Start centers as the study children achieved fall-spring TOPEL gains 0.24 SD greater than expected for their age, similar to study children's gains of 0.27 SD greater than expected for their age. In numeracy, the IGDI assessment developers found that Head Start preschoolers experienced fall to spring gains of 0.48 SD (oral counting), 0.44 SD (number naming), and 0.78 SD (quantity comparison), respectively (Hojnoski et al., 2009). Study children experienced mean numeracy gains in the same general span (approximately 0.45-0.71 SD across subtests). In executive function, a recent pre-pandemic longitudinal study that examined preschoolers’ fall to spring MEFS executive function gains in a primarily low-income sample found average gains of 0.06 SD (Anderson et al., 2020), in the same range as those observed in the current study (0.05 SD). These findings are consistent with the notion that study children's gains were not outside the general range of those observed in these pre-pandemic research studies; however, as noted above, these should only be considered suggestive of potential comparisons meriting note.

The observation that study children experienced school readiness gains during the pandemic-affected school year leads to questions about the mechanisms that supported this growth. On the one hand, this finding may seem surprising, given that research conducted with older students has documented large pandemic-induced learning setbacks (e.g., U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 2022; West and Lake, 2021). However, child care centers tend to operate under different regulations than public school districts, and in many localities ECE centers were able to reopen in-person during periods when public school buildings remained closed (Kim et al., 2022). As a result, children enrolled in preschool may have missed less school on average than older students, contributing to fewer learning setbacks. We are unable to identify research that has examined this comparison directly; however, as one point of reference, as of spring 2021, 25% of fourth-graders nationally were still reportedly enrolled in fully-remote schooling (Institute of Education Sciences 2021), while 11% of preschoolers were reported to be in fully-remote programs (Jung and Barnett, 2021). Another possibility is that parents may have found it easier to ‘home teach’ preschool concepts, such as oral counting, as compared with the more advanced mathematics and other academic content that older students were expected to learn, and as such parents may have been better able to mitigate the effects of school closures on preschoolers’ than older students’ learning outcomes.

To summarize the attendance findings, overall, we found that for two of the three early numeracy domains measured, outcomes tended to be stronger among children who attended more in-person preschool. However, relationships were not consistent across models for print knowledge, and we did not observe statistically significant associations between attendance and executive function outcomes in our data.

One possible explanation for this pattern of findings may be that during the pandemic-affected year, an amalgamation of selection effects and exceptional efforts by parents and ECE centers yielded the combined result of buffering the impacts of the pandemic on young children's school readiness skills. Head Start parents that were better-positioned to compensate for preschool closures with home learning supports may have opted to keep their children home more frequently during the COVID-19 outbreak. In this scenario, the lack of a consistent association between pandemic preschool absences and child outcomes may be explained by the fact that in descriptive research, researchers observe relationships in equilibrium. For example, if parents whose children are assigned to teachers whose instruction is weak disproportionately enroll their children in tutoring programs to compensate, observed correlations between child test scores and teachers’ instructional quality will be attenuated (Hill et al., 2011). Head Start parents and caregivers whose children were home more frequently may have responded to reduced preschool access by intensifying their own efforts to teach their children early skills, thus mitigating the immediate effects of preschool absences on their children's development. Teachers and ECE staff also provided supports that may have mitigated the effects of absences, including parent outreach, links to web-based learning materials, and, for children enrolled in virtual programming, weekly Zoom videoconference meetings.

At the same time, perhaps children attending in-person may have benefitted from unexpected affordances, such as smaller class sizes than they would have experienced during a non-pandemic year. Like many Head Start centers and other ECE programs (e.g., statewide in California; Kim et al., 2022), anecdotally, the partner organization reported that centers reduced class sizes to accommodate social distancing. Smaller class sizes in Head Start predict lower teacher job stress (Friedman-Krauss et al., 2014), stronger child literacy outcomes, and increased teacher-child interaction time (Francis & Barnett, 2019), a hypothesized support for executive function development (Neitzel & Stright, 2003). Future research comparing measures of classroom instructional quality before and during COVID-19 would be helpful to shed light on how preschoolers’ classroom experiences were affected by the pandemic, including potential classroom-level supportive factors that may have strengthened pandemic learning outcomes.

However, even if the above hypotheses prove correct, parents’ and caregivers’ taking of unusual steps to support preschoolers during the pandemic almost certainly would have had costs to families not visible in these data, and such efforts would likely be unsustainable in the long term. Many parents, particularly mothers, reduced work hours or delayed seeking new employment after a layoff in order to care for children at home during the pandemic (Alon et al., 2020). Reallocating parents’ time comes with costs (Robinson et al., 2022). Such shifts may have bolstered preschoolers’ learning, but with negative side effects such as lower household financial stability and parental stress from ‘homeschooling.’ Extra work undertaken by teachers may also have buttressed children's learning, but with a cost of increased staff burnout risk (Hanno et al., 2022).

4.1. Limitations

Several limitations to the current study are important to note. Firstly, the analyses used are descriptive in nature. As noted above, the relationships identified between attendance and learning outcomes are correlational, not causal. Second, the current study sample was relatively small and limited by attrition, as data collection with families during the height of the pandemic posed challenges. Participating Head Start centers were not representative of all Head Start sites nationally, limiting the generalizability of the findings. Further studies with larger and nationally representative samples, along with studies following preschoolers’ progress post-pandemic, are warranted.

From a measurement perspective, we note that we were unable to locate evidence that the IGDI-EN measure was explicitly validated for Hispanic samples. Information was also not available to analyze race match between participants and the individuals administering the IGDI, TOPEL, and MEFS assessments, nor to analyze children's comfort level with technology as a potential contributor to their scores on the virtual assessments. Such investigations, along with replication studies using alternative measures, would be valuable to conduct in future research. More broadly, our measures captured important aspects of academic achievement and executive functioning, yet more research is needed to capture other domains of child well-being during the pandemic. From an ecological perspective (Bronfenbrenner, 1979), preschool closures and virtual preschool affect the ecology of the family, including parents’ ability to work and family stress. These factors may have been expected to influence other dimensions of children's skills beyond those we measured, including social and behavioral adjustment (e.g., Mistry et al., 2004), and more studies are needed to address those dimensions.

4.2. Conclusions and future directions

However, despite the noted limitations, we were able to compile evidence based on direct child assessments of preschoolers’ learning progress and development during one of the school years most heavily disrupted by COVID-19. The current study points towards potential avenues for fruitful future research. More research is warranted to understand how parents of low-income preschoolers and preschoolers from minoritized racial and ethnic communities responded to the pandemic, including qualitative interview and survey studies of parents whose children attended more and fewer days of in-person Head Start, as well as parents who declined to send their children to Head Start altogether. Such research could shed light on how families altered their household dynamics to accommodate Head Start absences during the pandemic, and potential side effects of these shifts on children that may implicate needs for future supports.

Future research could also examine sources of strength that families drew on to support children's learning amidst child care disruptions. Families from minoritized racial and ethnic communities and low-income families have at their disposal and draw upon many and varied sources of strength to support preschoolers’ learning and early care, depending on contexts; such sources range from family and social networks; to church and faith-based supports; cultural institutions and civic organizations; and culturally-specific assets that foreground elements of home- and community-based learning (González and Moll, 2006; National Research Council, 2000; Small, 2006). As one example, researchers have documented strengths among Latino families in fostering preschool children's sense of family belonging, such as via consistent family mealtimes (Calzada et al., 2010; Murphey et al., 2014), which may contribute to the strengths in approaches to learning that Latino children possess on average in preschool (Bustamante & Hindman, 2020). The presence of such positive family routines may have buffered children when the pandemic disrupted other predictable facets of their daily lives. As another example, in a study of preschoolers’ home literacy practices, researchers found that low-income African American mothers actively promoted their preschoolers’ literacy skills via a range of practices including book reading, writing, homework supervision, and games, with other family members serving as a frequent source of support (Jarrett & Coba-Rodriguez, 2017). Such existing assets in active home literacy practices could have helped young children to continue learning in the midst of preschool closures. Given the vast diversity of Latinx, Black, and low-income communities in the United States, as well as regional variations across the country in COVID-19 restrictions and public policy responses, specific research is needed to examine the strengths that families and communities brought to navigating the pandemic in a range of geographic localities, contexts, and settings.

The findings also suggest implications for public policy. Our findings confirm prior reports indicating lower preschool attendance during the pandemic. In response, it may be valuable for early childhood centers to increase their parent outreach to rebuild preschool attendance, particularly in the Head Start context. Strategies such as fostering parent relationships, communicating attendance goals, and engaging community partners show promise (Kalil et al., 2021; Katz et al., 2016; Sommer et al., 2020). Moreover, offering children more instructional time, such as via kindergarten-preview summer programs, can strengthen their academic skills and self-regulation (Duncan et al., 2018), comprising another research-based avenue to support school readiness. The observation that children with lower in-person preschool exposure tended to have somewhat lower numeracy outcomes suggests that these children may particularly benefit from support for missed learning opportunities in this domain.

In summary, this study aimed to provide new evidence on preschoolers’ experiences during COVID-19. Producing this knowledge is a critical step in determining the impacts of the pandemic on young children's development, as well as identifying needed resources to support children's pandemic-related recovery and educational opportunities in the future.

CRediT authorship contribution statement

Kathleen Lynch: Conceptualization, Methodology, Writing – original draft, Writing – review & editing. Monica Lee: Conceptualization, Methodology, Formal analysis, Writing – review & editing. Susanna Loeb: Supervision.

Data availability

• The data that has been used is confidential.