Using educational transitions to estimate learning loss due to COVID-19 school closures: The case of Complementary Basic Education in Ghana

Abstract

Learning loss is expected for millions of children who have been out of school as a result of the current COVID-19 pandemic. Unfortunately, it is uncertain how much learning will be lost and how wide the gaps may be for disadvantaged children. This paper uses a unique longitudinal dataset to estimate learning loss during a three-month transition from Complementary Basic Education to government schools in Ghana. Our results show an average learning loss of 66 % of previous learning gains in foundational numeracy during this transition period. More importantly, we estimate widening gaps in learning loss according to lack of home learning support, as well as lack of home learning resources. Our results have implications for the provision of learning activities and support at home, not just during current school closures due to COVID-19, but also during transitions between academic years.

1. Introduction

Across the world, countries are facing unprecedented and challenging times in trying to support the education of millions of children outside of school. Various methods for reaching the most marginalised students have been implemented in diverse countries, ranging from off-line remote learning solutions including the use of radio and television in locations of limited internet penetration, to online provision for some of the better resourced schools and communities (Asim et al., 2020). Yet, many of the most marginalised children remain unable to access any form of educational support. In addition, most children who received educational materials, whether off-line or online, lacked the teacher guidance needed to utilise these for learning. While the learning solutions provided are expected to enhance some form of skill acquisition in children, they are by no means expected to replace learning in the classroom. How much children will learn during this time remains unknown, although it is expected that the poorest will be hit the hardest.

Providing a deeper understanding of the inequalities in learning which are expected as a result of school closures is central to the debate around educational provision in the post-COVID-19 era. As governments ease restrictions on school closures, and as schools begin to reopen under new social distancing rules, it is expected that not all children will return to education. Those who do return will most likely have significant learning loss resulting from school closures. For the most marginalised children, the magnitude of their expected learning loss and the factors expected to protect children from such loss, remain empirical questions.

A number of recent blogs have indicated that forms of marginalisation experienced by children are likely to be connected to increased learning losses due to school closures (e.g. Kim and Rose, 2020; McClain-Nhlapo, 2020; Parsitau and Evelyn, 2020; Tibebu, 2020). Recognising the potential higher risks that might be faced by marginalised populations and the potential consequences on their learning, it is important to establish how much learning is likely to be lost as a result of school closures, and the extent to which these populations are disproportionately affected. It is also important to explore the impacts of support at home for learning or the availability of learning materials, activities and resources, as these supports are likely to be less available for those most in need. Furthermore, empirical evidence on the impact of unintended school closures on learning is currently extremely limited.

There are at least three ways in which researchers have engaged with estimating the impact of school closures on learning loss. First, some researchers have used the impact of natural disasters which has caused children to spend time away from schools. Using the case of Pakistan, recent research by Andrabi et al. (2020) traced the impact of school closures lasting 14 weeks on average for children who were affected by the 2005 earthquake. The authors estimated a learning loss equivalent to 1.5 years of education in the areas most affected by the earthquake. The authors indicated that educated mothers were able to mitigate learning losses, but not losses in other factors of human capital accumulation such as nutrition. The authors also highlighted that there was a greater loss in learning when children returned to school, perhaps because the curriculum had not been adjusted to account for lower student starting points when children re-joined school. These findings also resonate with Sacerdote’s (2012) research, which found that students impacted by school closures and displacement resulting from Hurricane Katrina (in the United States) suffered sharp declines in test scores in the year following the disaster (approximately 0.10 standard deviations).

Another way to estimate learning losses is to use simulations about the potential impact of the pandemic more generally. These simulations take into account the direct impact of school closures on learning which is measured as learning that is not taking place while schools are closed, learning that is forgotten, as well as the indirect impact on learning for children not returning to school due to the detrimental effect of the pandemic on household income (Azevedo et al., 2020; Cummiskey et al., 2021). These simulation exercises also consider potential mitigation effects of off-line and online educational provision for some children at their homes during school closures. Based on a series of modelling assumptions and data from the learning adjusted years of schooling (LAYS) from 157 countries, Azevedo et al. (2020) simulate a global learning loss of 0.6 years of schooling adjusted for quality due to school closures that last around 5 months. This learning loss could be as high as 0.9 years of schooling or as low as 0.3 years of schooling, depending on the mitigating circumstances.

A third way to estimate learning losses from time out of school is to examine the extent to which learning changes because of long holidays, summer breaks or transition time. Many school systems provide a break somewhere between 6 to 9 weeks between the end of a school year and the beginning of the next academic year. Most schools remain closed during this time, although some offer holiday clubs, cultural, artistic or sport activities for children. Evidence examining learning loss during transition time does not typically consider efforts to encourage students to engage in remote or home learning by parents, school actors, communities and/or governments, which are more likely to have occurred during school closures resulting from the pandemic (see for example Brossard et al., 2020; Kamei et al., 2020; Mishra et al., 2020). On the one hand, that means that these estimates may serve as an upper bound indication of the expected loss (which could be mitigated by home-learning opportunities). However, the gaps that emerge in learning loss during transition time are likely to be exacerbated during the pandemic, as a result of inequalities in access to online and offline learning resources.

Several studies from the Global North have estimated learning loss as a result of time away from school (for a recent review see Education Endowment Foundation, 2020), although there are important methodological debates around the ways in which loss has been calculated (Von Hippel and Hamrock, 2019; Dumont and Ready, 2020). In the UK, Shinwell Jackie and Defeyter, Margaret Anne (2017) estimated loss in spelling for children between the ages of 5 and 10 years, in areas of low socioeconomic affluence, when they returned to school immediately after a 7-week summer holiday. The authors found a small but statistically significant change in mean scores for spelling, changing from 26.6 to 25.4 from the beginning to the end of the summer (although no significant effects were found for their performance in reading words). In the USA, summer learning loss has been studied extensively. One of the earliest reviews of the issue found that summer losses equated to approximately one month of schooling, on average (Cooper et al., 1996). A more recent study found that students from low socioeconomic backgrounds were more likely to fall behind in mathematics, roughly between 4 to 5 points on a maths test per month away from school relative to children from higher socioeconomic backgrounds (McAlister, 2014). Other studies conducted in the USA have suggested that the impacts of extended school breaks without learning increase over time, and that children from disadvantaged backgrounds fall further and further behind their more affluent peers who have the opportunity to engage in learning activities throughout these periods (Terzian et al., 2009; Blazer, 2011).

Empirical evidence from the Global South on learning loss as a result of grade transition has not been as well documented. Slade et al. (2017) used literacy assessments in Malawi to estimate learning loss during long breaks in the academic year. Their results show that across grade transitions from primary 1–2 and 2–3, children performed lower on all literacy subtasks. For example, children who transitioned from grade 2 to grade 3 in 2015 lost “12.7 letters per minute, 10.2 syllables per minute, 5.7 words per minute and 5.4 words per minute in connected text during the transition (Slade et al., 2017, p 469)”. They found no differences by gender in learning loss for their sample. Using the learning trajectories of out of school children in Ghana, who completed an accelerated learning programme known as Complementary Basic Education (CBE) between October 2016 and June 2017 and then transitioned into government schools in October 2017, Akyeampong et al. (2018) also found significant learning losses during the three month transition period. For example, the authors found a 20 percentage point drop in number identification and 23 percentage point decline in reading comprehension, on average during this transition period. Akyeampong et al. (2018) did not find relative gender differences in learning loss, on average, during this period.

Further empirical evidence from the same case of out of school children in Ghana has shown wide inequalities in learning loss for low performing girls, as well as for children whose language of instruction changed from CBE to government schooling (Carter et al., 2020a, b). Girls who were low performers (i.e. with results in the lowest quartile of scores) at the beginning of the CBE programme in 2016−17 were more likely to remain low performers throughout the 2 academic years than their low performing male counterparts, who managed to shift from their initial weak position overtime (Carter et al., 2020b). With respect to language, Carter et al. (2020a) found that children who changed language of instruction from mother tongue during the CBE programme to one of the official languages of the Ghanaian Education Service in public schools significantly lost language gains achieved during the CBE programme. The average learning loss for children who moved into government schools with a different language of instruction was 33 percentage points in letter sound identification and 37 percentage points in reading comprehension relative to children for whom the language of instruction remained the same. Carter et al. (2020a) further found that these differences varied by grade of transition, as this was related to language of instruction in Ghana, but no gender differences were found according to losses during the transition period based on language of instruction.

Given the richness of the data, the timely importance of this research due to school closures, and the lack of evidence from the Global South regarding learning loss due to time away from school, we use the learning trajectories utilised by Carter et al. (2020a; and 2020b) to estimate learning loss during the transition between CBE and government schooling. Additionally, we investigate the extent to which factors related to availability of home learning support or availability of home learning resources predict the widening or narrowing of the learning loss gap during the transition. With respect to home learning support, we focus on whether children receive support to study at home or if they ask for help from family members with their school work. For availability of home learning resources, we consider whether children have access to educational materials or activities at home, as well as availability of television, radio and mobile phones, which are currently considered as central to supporting children’s learning during school closures. As mentioned above, it is expected that parental engagement during the transition time may be different from what has occurred during pandemic-related school closures and we reflect on this issue in the discussion of results. Empirically, we assess learning at the beginning of the academic year in government schools conditional on learning at the end of the CBE programme during the previous academic year, thus raising the question: how did students who have access to different levels of resources or support at home perform in numeracy after three-months of not being in school?

This is one of the first analyses to estimate expected learning loss due to school closures for a population of disadvantaged and previously out of school students. Furthermore, our paper highlights the extent of learning loss and whether this is associated with key factors related to the home learning environment, which has not been explored before. Moreover, children who participated in the CBE programme are unlikely to have had access to educational programmes between school years, as could have been the case for children in more economically affluent areas. Of course, we do expect that other forms of learning took place, and this is acknowledged as a limitation of our measurement of learning via foundational numeracy test scores. Finally, the authors of this paper were not involved in any aspect of CBE delivery or implementation but did work on the independent evaluation of the programme.

2. Objective and research questions

The overall aim of this study is to estimate learning loss as a result of time out of school, measured by the transitional period between graduating from the CBE programme at the end of June 2017 and enrolling in government school in September 2017. For the empirical estimation of this research we use four rounds of data collection which were completed as follows: round 1 took place at the beginning of the CBE programme in October 2016; round 2 took place during the last month of the CBE programme in June 2017; round 3 took place in government school in October 2017 (about 1 month after lessons had started); and finally round 4 took place during the month of June 2018, almost at the end of the first year in government school. We extend previous work in Ghana and Malawi which focused on learning loss by gender and language of instruction to include differences according to: (1) availability of home learning support (2) availability of home learning resources. The overall research question is: what is the learning loss experienced by marginalised children during the transition period? The following two sub-questions are also addressed:

• aTo what extent do learning losses in the transition period depend on the availability of home learning support (i.e. the willingness of students to ask for support from adult members of their household when they found learning challenging; whether children were given enough time to study at home)?
• bTo what extent do learning losses in the transition period depend on the availability of home learning resources? (i.e. learning materials and activities in the home environment, as well as basic resources such as radio, television and mobile phones)?

For all these questions, we are interested in examining the size of the learning loss between groups. To account for the fact that students had different starting points in their numeracy ability, we estimate learning loss relative to learning gains achieved throughout CBE.

3. Methodology

3.1. Description of the CBE programme and sample

Complementary Basic Education (CBE) is a programme initiated by School for Life (SfL) in 1995 and scaled up in 2013, with support from the Department for International Development (DFID1 ) and the United States Agency for International Development (USAID). The CBE programme provides nine months of accelerated learning in basic literacy and numeracy in eleven mother tongue languages. Classes are set up in remote and deprived areas for children who would normally be unable to attend school. Children are generally aged between 8–14 years and the curriculum aims to equip them with the knowledge and skills equivalent to those learnt in the first three years of formal school. Following completion of the CBE, children transition into nearby primary schools at a grade level compatible with their achievement at the end of the programme.

Data used for this paper consists of learners who took part in the CBE programme in the 2016/2017 academic year and were tracked longitudinally over two years. There are four rounds of data available for these learners, including baseline and endline assessments on foundational literacy and numeracy during the CBE programme, as well as repeated measures for the start and end of the first year after transition into government schools. These data also include background socioeconomic characteristics of children as well as their opinions about learning support received at home. Data on learning loss refers to the period between the end of the CBE in June 2017 and the beginning of the government school year in October 2017.

The original sampling took place in September 2016, when 2360 students were selected from over 40,000 students enrolled in the CBE programme using a stratified random sampling approach intended to provide proportional representation by gender, language, region, district and provision by implementing partners. The original sample consisted of 53 % boys. 66 % of the sample were located in the Northern region, 12 % in Upper West, 11 % in Upper East, 9% in Brong Ahafo and 2% in Ashanti. From the original sample, 29 % of the children responded they had access to a light bulb during the night, whereas 55 % used a torch light. Similarly, little over half of the children in the original sample indicated they go hungry some days, whereas the rest indicated that they did not. Finally, 79 % of these children had never been to school prior to enrolling in the CBE programme in October 2016 and 21 % had some school experience but had already dropped out.

As demonstrated by Carter et al. (2020a), there was attrition between the original sample of 2360 children and the estimated sample which contains children with full information over the 4 time periods of data collection (1166 children). Their overall conclusion was that the “estimation sample contains a larger proportion of children who are high performers, missed fewer school days, and engage more with learning activities at home compared with the full sample" (Carter et al., 2020a, p. 4). The implication of sample attrition for our paper is that our estimates of the learning loss are likely to be lower-bound estimates for the CBE student population overall.

3.2. Assessment of numeracy skills

We use foundational numeracy skills over time to measure learning loss during the transition from CBE into government schools. The learning assessment used for the four rounds of data collection were based on the Early Grade Mathematics Assessment (EGMA). EGMA was designed to provide information about basic mathematics competencies which are typically mastered in the very early grades of primary school, and important to achieve higher academic competencies.

The assessments administered during the CBE programme (rounds 1 and 2 of data collection) were different from the standard EGMA instruments, which were used during transition into government schools (rounds 3 and 4 of data collection). The assessments administered during the CBE programme were modified by the Directorate of Research Innovation and Consultancy (DRIC) of the University of Cape Coast in Ghana, to reflect the specific numeracy competencies learners were expected to acquire in the CBE programme2 . Due to these adaptations, the assessments used during the CBE phase of data collection contained a few key differences from the EGMA used in the latter phase of data collection. These included differences in the numbers of items in each task as well as the subtask constitution of the instrument. These differences including the number of assessment items (in brackets) are shown in Table 1 .

Table 1.

Differences in test instruments between modified and standard EGMA.

Modified EGMA Instrument (Start and end of CBE)	Standard EGMA Instrument (Start and end of first year of formal school)
Number identification: One-digit (50)	Number identification: One and two-digit (20)
Number identification: Two-digit (40)	Number discrimination (10)
Missing number (5)	Missing number (10)
One-digit addition (2 mechanical; 1 word problem)	One-digit addition (20 mechanical)
One-digit subtraction (2 mechanical; 1 word problem)	One-digit subtraction (20 mechanical)
Two-digit addition (4 mechanical)	Two-digit addition (5 mechanical)
Two-digit subtraction (4 mechanical)	Two-digit addition (5 mechanical)
Problem solving: Multiplication (3)	Word problems (6)
Problem solving: Division (3)

Notes: Numbers in parenthesis indicates the number of items for each sub-task.

Due to differences between the subtask constitution of instruments used in the first and second year of data collection, only some items could be selected for comparison over time. In order to ensure comparability, the analysis that follows therefore includes a combined measure of numeracy using missing number identification, two-digit addition and two-digit subtraction. Since tests at the end of the CBE programme and the beginning of government schools measure the same competencies, our results are not distorted by difference in assessment measurement such as rescaling or changes in test form (Von Hippel and Hamrock, 2019).

3.3. Key factors related to learning loss

We extend previous work by Carter et al. (2020a, 2020b) to measure the extent to which learning loss varies according to the availability children have to home learning support and home learning resources which could foster learning during the transition period. Assets such as television, radio and mobile phone are included due to their importance for delivering education in the home during school closures due to COVID-19. Our aim is to provide a deeper understanding of the expected impact of the pandemic on learning losses by estimating both the average learning loss and the relative change in this average loss according to the availability of home learning support and home learning resources (see Table 2 for descriptive statistics on these factors plus control variables).

Table 2.

Descriptive statistic of sample to estimate learning loss.

Variables		Description	Descriptive Statistic
Home learning support	Time to study	% have time to study at home	68.7
	Asking for support	Never ask adult (%)	35.7
		Sometimes ask adult (%)	42.9
		Most of times/always ask adult (%)	21.5
Home learning resources	Activities at home	% with reading of counting activities at home	73.1
	Reading Materials	% with books or reading materials at home	72.6
	TV	% with TV	15.6
	Radio	% with radio	52.2
	Mobile Phone	% with mobile phone	72.5
Controls	Gender	% female	49.2
	Language	% no change in language of instruction	53.6
	Lessons easy	% found most of the lessons easy during the CBE	35.8
	Effort	% most of the times tried hard during CBE	53.4
	Ability	% thought they were good at maths during CBE	29.4
	Age	Average Age (sd)	10.3 (2.2)
	HH size	Average household size (sd)	9.9 (5.7)
	Attendance	Average missed days at school (out of 5) and (sd)	1.1 (1.2)
	Electricity	% access to electricity at home	33.7
	Poverty	% with less money than others in village	63.6
	Sample size	Number of observations	1166

Note: Descriptive statistics refer to proportion or average. For average, standard deviations are in parenthesis. Source: CBE Monitoring and Evaluation 2016–2018.

3.3.1. Availability of home learning support

Indicators related to availability of home learning support included whether children asked an adult at home for help with school work and whether children were given enough time to study. Support at home from an adult was measured from child reports by combining the following statements: “when I did not understand things at school I asked my mother or female adult” and “when I did not understand things at school I asked my father or male adult”, each recorded on four item scales (i.e. never, sometimes, most of the time, always). We created three categories based on these questions: 1) those who never asked an adult for support (35.7 % of the sample); 2) those who sometimes asked at least one adult for support (42.9 %); and 3) those who most of the time or always asked at least one adult for support (21.5 % of the sample). Our second indicator related to whether the child was given enough time to study at home was measured as binary to differentiate ‘never’ from the rest.

3.3.2. Availability of home learning resources

Indicators related to availability of home learning resources included whether children had access to activities involving reading, writing or counting, as well as the availability of books or other reading materials. Interestingly, nearly three-quarters of children had access to reading materials or activities related to reading, writing or counting (Table 2). Access to television, radio and mobile phone are explored as potential devices to bring schooling into children’s homes. As shown in Table 2, only 15.6 % of children in the sample had access to television in the home, and little over half to radio. 72.4 % of children indicated they had a mobile phone in their homes.

3.4. Estimation method

In order to estimate the relative learning loss during the transition we use difference-in-difference (DID) estimation techniques. Our DID model compares the combined numeracy attainment before and after the transition, for children who have different levels of home learning support and home learning resources. In all our estimations, we also include other controls which are important for learning trajectories (and potentially learning loss). These control variables include gender and age of the child, self-rated opinions on school effort and difficulty of lessons in school, self-concept of mathematics ability, whether the child had to change language of instruction from the language in which they learned during the CBE programme, school attendance in the 5 days prior to the survey at end of CBE, household size, whether the household had access to electricity and relative poverty (whether the child ranked their household among the poorest in the community or not). Control variables are measured at the end of the CBE programme to condition out factors which take place prior to or at the start of the transition period (see Table 2).

A generic equation for the DID estimation we utilise in this paper to estimate the relative magnitude of learning loss for each of the factors is:

$$N_{it}={\beta }_0+{\beta }_1{F}_i+{\beta }_{2}Time+{\beta }_3{F}_i|Time+\gamma X_{it}+e_{it}$$ (1)

where N is a combined measure of numeracy for child i in time t; F is a vector containing the factors for which we are interested in measuring relative differences (in some cases it is a dummy variable and in others a categorical variable); Time is a dummy variable to indicate the pre-transition and post-transition; and F|Time is the interaction term which denotes the relative difference in learning loss between children who have benefitted (or not) from such factors before and after the transition (i.e. our main coefficient of interest). The matrix X contains control variables.

We propose to undertake the following empirical strategy in order to respond to our research questions. First, we examine correlations of the factors for home learning support and home learning resources to check for collinearity and to ensure that all cells had sufficient sample sizes for estimation (i.e. at least 30 children per group for all analyses). Then, we estimate the model described by Eq. (1) for factors related to home support. These factors are entered in the model with the time interaction to estimate the DID parameters. Other factors related to home resources are included as controls. Then, we estimate the DID for factors related to home resources and include as controls factors related to home support. Our final, parsimonious model includes all the DID parameters for all factors previously identified as important for the widening or narrowing of the learning loss gap.

In order to estimate the relative learning loss of the transition, we use the magnitude of the estimated parameters and adjust to the relative gains during the CBE programme. In other words, we consider the relative loss as a function of the relative gains prior to the transition. This is an estimate of the percentage loss relative to gains.

4. Results

4.1. What is the learning loss experienced by marginalised children during the transition period?

Fig. 1 presents the percent score in numeracy over time, including the transition period. The learning loss during the transition period shown in Fig. 1 is substantial, represented by the downward sloping line in the middle of the figure. Overall, gains in numeracy scores during the CBE programme constituted around 22.4 percentage points, whereas the loss during the transition was 16 percentage points. Therefore, about 66 % of the previous learning gains during the CBE programme were lost during the transition period. We also examined these trends by gender but found no significant differences for boys and girls.

Fig. 1.

Percent score achieved in numeracy subtasks over time.

On average, the magnitude of the learning loss per month out of school is sizeable. Every month out of school had a learning loss equivalent to around 20 % of what was learned during the previous school year. Considering that many children will spend somewhere between 4 to 6 months not in school as a result of the COVID-19 school closures, the learning gains obtained from foundational numeracy skills before the pandemic could be completely lost in the absence of successful supports for learning at home.

4.2. To what extent do learning losses in the transition period depend on the availability of home learning support?

To answer to this question, we focus on what we defined as support at home, namely 1) whether children asked adults in the home for help with school work, and 2) whether children were given enough time to study at home. Table 3 column 1 shows results for the relative learning loss due to the transition period according to availability to home learning support factors. The average learning loss in Table 3 column 1 refers to children who never asked for support and who were not given enough time to study at home. For these children, the average learning loss was 20.5 percentage points.

Table 3.

Learning loss during transition time: difference-in-difference estimator for home support and home resources and parsimonious model.

	Home Learning Support	Home Learning Resources	All Factors Combined
VARIABLES	[1]	[2]	[3 = Parsimonious]
Average learning loss	−20.517*** (2.147)	−30.827*** (2.941)	−30.051*** (2.581)
Time to study at home	3.136 (2.012)	2.338* (1.328)	2.306* (1.324)
DID: time to study relative to no time	−1.001 (2.598)	--	--
		--	--
Sometimes ask adult help	1.103 (2.126)	0.473 (1.442)	0.516 (1.324)
Most times ask adult help	−2.050 (2.049)	6.223*** (1.808)	0.405 (2.543)
DID: sometimes ask relative to never ask	3.138 (2.737)	--	−0.118 (2.817)
		--
DID: most times ask relative to never ask	14.386*** (2.798)	--	11.487*** (3.301)
		--
Literacy/numeracy activities	3.556** (1.761)	−1.938 (2.671)	−2.446 (2.486)
DID: Learning activities relative to none	--	11.686*** (3.313)	12.929*** (2.812)
	--
Reading materials	1.590 (1.694)	−0.547 (2.466)	2.042 (1.653)
DID: Reading materials relative to none	--	5.149 (3.238)	--
	--		--
TV/Radio/Mobile	0.878 (2.197)	0.758 (2.216)	1.121 (1.528)
DID: TV, Radio or Mobile at home relative to none	--	0.717 (2.887)	--
	--		--
Other controls	Yes	Yes	Yes
Constant	13.950** (5.808)	22.106*** (5.853)	30.203*** (5.809)
Observations	1995	2027	2027
R-squared	0.331	0.328	0.333

Note: Robust standard errors in parentheses. DID (difference-in-difference parameters) indicates the relative loss within factors. Each model is estimated conditioning on the control variables shown in Section 3.4 (results not shown here). Asterisks *, **, *** indicate statistical significance at 10, 5, 1 and 0.1 % level. Source: CBE Monitoring and Evaluation 2016–2018.

Then, for each of the home support factors we show the relative difference in performance at the end of the CBE programme, which is indicated by estimated parameters on time to study at home or asking for help from adults in the household. We also show the DID estimator, which indicates whether the learning loss differs between two groups of children (for example those who had time to study at home relative to those who did not). It is important to note that positive coefficients represent decreases in the amount of learning loss. Results show that there is no statistical difference in learning loss between children who were given time to study relative to those who were not given time (Table 3 column 1).

The most relevant result, however, relates to receiving support from adults when children did not understand lessons at school. Children who reported mostly or always asking for support from adults had 14.4 percentage points less learning loss in numeracy scores than children who never asked for help. This result is even more important if we consider that at the end of the CBE programme, children who reported asking for support and those who never asked for support did not differ in their numeracy scores. Therefore, the impact is solely a result of the differences during the transitional period.

4.3. To what extent do learning losses in the transition period depend on the availability of home learning resources?

Table 3 column 2 shows results for the model estimating relative learning loss due to home learning resources. Given that television, radio and mobile phone are resources that can bring learning into the home, and due to the low cell count for children living in households with some of these resources, but not others, we combined availability of these three factors into one.3 In other words, for estimation purposes we differentiate between children living in households with access to either television, radio or mobile phone (79.4 % of the sample) from those without any of these resources (20.6 % of the sample).4 Our results show that the average learning loss for children who had no learning activities at home or reading resources as well as no television, radio or mobile phone was 30.8 percentage points. It is important to highlight that this is a larger average learning loss than previously explored for children who did not have access to home support.

With respect to the relative learning loss, our results show that children who had access to reading, writing or counting activities in the home had a smaller learning loss compared with children who did not have access to these activities (see Table 3 column 2). Overall, children who had access to learning activities at home had 11.7 percentage points lower learning loss relative to children who did not have access to these activities. Importantly, at the end of the CBE programme, the average score across all numeracy subtasks was similar for children who had access to learning materials and for those who did not.

For reading materials at home, we did not find statistical evidence of a relative learning loss due to the transition period by having reading resources at home—nor did we find any statistically significant differences for children who had access to at least one of these assets at home and those who did not (Table 3 column 2).

4.4. Combining factors: learning loss due to availability of home learning support and availability of home learning resources

In our final model, we include all significant factors from the previous models to determine their relative impacts on learning loss. As shown in the final column of Table 3, the average learning loss during the transition for children in our reference group was 30 percentage points, based on the comprehensive model.

In terms of relative loss, we find that all previously estimated significant factors remain statistically significant but the size of the relative differences change slightly. For example, if we compare the estimated learning loss for children who asked adults for help most of the time in column 1 and column 3, we see a reduction from a 14.4 percentage point gap to around an 11.5 percentage point gap relative to those who never asked for help. For those children with learning activities at home, the estimate increased slightly from 11.7 percentage points (column 2) to 12.9 percentage points (column 3).

It is also important to provide a benchmark for the size of the estimated gap in learning losses for these groups of children. The gap of 11.5 percentage points difference between children who asked for help and those who did not is equivalent to 43 % of the average learning gains achieved during the CBE programme. For those who did not have access to learning activities, the learning loss during the transition relative to those who did have access is equivalent to 35 % of the average learning gains previously obtained during the CBE programme.

To understand more about these findings, we also examined the overall trajectories of children from the start of CBE through the end of the first year of formal schooling, by both of the key factors in the final model (see Fig. 2 ). There are several findings to highlight from these figures. First, in both cases presented, the gap in numeracy achievement either remained the same or narrowed during the CBE programme. Secondly, and consistent with our analysis, these trajectories in numeracy scores show a widened gap in attainment during the transition period. Finally, the gap narrows again during the first year in government schools, which points to a lasting impact of CBE as opposed to effects that simply fade out after a shortened, accelerated programme. This latter finding also points to the important role of schools (whether formal or informal) as ‘equalisers’ of attainment; although this is only on average as previous analyses by Carter et al. (2020b) point out that low achieving girls do not have the same opportunity for equalising their attainment as low achieving boys do. Furthermore, our results point to the importance of support at home during the transition and by implication during time away from school due to COVID-19 school closures.

Fig. 2.

Learning trajectories for children with access to support at home and activities.

5. Conclusions

There is an urgent need to provide evidence on the learning loss that might be expected as a result of school closures during the COVID-19 pandemic. This study was motivated by this urgency and provides estimates of learning loss experienced by disadvantaged children from Northern Ghana who benefited from one year of accelerated complementary education before starting formal education in government schools. During the transition, these children spent around three-months not in formal education. We estimate that the learning loss in foundational numeracy suffered over a three-month period is about 66 % of the gains attained during the prior year. This equates to an approximate 20 % loss in learning gains per month that students are out of school.

Results from this study additionally underscore the critical role that household factors play in learning loss for students from marginalised backgrounds in the Global South. As shown, being unable to ask for help from primary caregivers or adults in the household, as well as a lack of opportunities to engage in learning activities at home led to the largest relative losses for students. At a time when education is witnessing a surge in the use of digital platforms for learning, whether it is language apps, video conferencing tools or online learning software, this study reminds us that the basics matter most. This is particularly the case for students from remote and disadvantaged circumstances who struggle to gain access to books or any support at home, let alone to a computer or reliable internet. Without home-based support, these students will continue to fall further behind their peers and widen the gap that will have to be addressed by teachers once schools reopen.

With regard to technology, we found that children who did not have access to a television, radio or mobile phone at home did not have a learning loss over and above that of children who had access to these devices during the transition period. However, it is likely that this is driven in part by the fact that these devices were not being utilised for educational purposes at the time, as is the case now with the current support during the COVID-19 crisis. In other words, the lack of these resources may be more associated with widening learning loss when they constitute a major component of the provision of out of school learning opportunities. This is particularly problematic given that an estimated 20 % of children in our sample did not have access to any of these devices at home, which would have major implications for any distance learning programmes that rely on such technologies. In our sample of around 40,000 learners who were enrolled in the CBE programme in the academic year 2016−17, this represents 8000 learners. Supporting these learners with printed educational materials and basic technology-free learning activities should be seen as a priority.

Overall, learning while not in school during a typical transition year may be different from learning at home during the pandemic. For example, in many contexts affected by the pandemic there has been an increased effort to support parents and family members, school actors, communities and governments to facilitate learning at home through family members or with the provision of educational distance learning resources.5 These are indeed mitigating factors that one would expect to ameliorate the learning losses during the pandemic. At the same time, however, they can also create very stressful circumstances for parents and children alike, which can impact on learning. This is indeed very different from the situation of a transition time prior to the pandemic where children were involved in diverse activities, without the need for social distancing, and regular school work. Yet, we estimated that there are inequities in learning losses during the transition time, with greater losses suffered by those children without access to support or educational resources at home. As the situation for those at risk is likely to remain unchanged, or potentially even worsen during the pandemic, we could expect a widening gap in learning losses as a result of the pandemic, despite (and in some part resulting from) the increased focus on distance learning. This latter result is potentially the most important application of our estimates of learning loss to what is expected during the current pandemic.

Whilst this study represents the first attempt to look at factors impacting learning loss due to time out of school during transitions, specifically for marginalised students from several regions of Ghana, it is important to consider the transferability of our results to other contexts within the Global South. Although it is difficult to establish the comparability of learning losses between and across studies, our findings resonate with evidence from the Global North that highlight learning losses particularly for children who do not have home support during the time out of school. Secondly, results shown in Fig. 2 indicate upwards learning trajectories for CBE learners after the transition which seem to narrow the gaps after one year in government schools. Carter et al. (2020a) established that CBE learners tended to catch up after a year in the formal school system when looking at learning trajectories of learners who changed language of instruction upon entering government schools. Overall, evidence points to the enduring impact of CBE on children’s learning and its appropriateness as a corollary for estimating losses from typical formal schooling opportunities (Casely-Hayford and Hartwell, 2010; Carter et al., 2020a, 2020b).

Given the gaps in learning loss, our study suggests the need for extra support for children from marginalised groups once they return to school. Compounding lower learning levels at the point of re-entry, students from disadvantaged backgrounds may face extra pressures in the aftermath of the COVID-19 crisis. For example, previous research has shown that changes in classroom routines and structure following crises can disrupt learning, affect concentration and lead to a negative attitude towards learning for students from challenging backgrounds (Mudavanhu, 2014). Social-distancing measures may very well bring about such changes in the classroom environment, which could lead to similar outcomes. Reports have also shown that because of food shortages after disasters, children may be required to help their families obtain food, which can impact students’ attendance and performance (Ayieko, 2006). These factors result in high failure, dropout and absenteeism. Financial constraints have also been found to cause families to withdraw students from schools (Mudavanhu, 2014). Whilst time will tell exactly what challenges will arise post-COVID-19, the financial devastation already being felt will likely continue even after the crisis has passed, potentially limiting many children’s opportunities to learn, even while at school.

Ultimately, we are not certain how schooling will continue to be impacted by COVID-19 but we are confident that formal education in schools will have to be only one way to continue to support children’s learning. Learning at home and in communities has to be reimagined if gains are to be achieved and losses are to be mitigated in the post-COVID 19 era.

CRediT authorship contribution statement

Ricardo Sabates: Conceptualization, Writing - original draft, Supervision. Emma Carter: Software, Data curation, Investigation, Writing - review & editing. Jonathan M.B. Stern: Methodology, Formal analysis, Writing - review & editing.