Learning loss and learning recovery

Abstract

In 2020, most countries closed schools. Two years after the pandemic began, the evidence strongly indicates that school closures result in learning loss. A decrease in learning could decrease future employment prospects and lower future earnings. This means that schooling matters. One promising policy option for mitigating learning losses during closures as well for subsequent learning recovery and acceleration is tutoring. While tutoring is effective, the replicability was demonstrated during the COVID-19 school closures. These online experiments were very cost-effective, showing that it is possible to provide quality instruction across the cost spectrum in different contexts.

Introduction

By mid-April of 2020, 192 countries closed schools and universities, affecting more than 90% of the world’s learners; or 1.5 billion children and young people, according to UNESCO (https://en.unesco.org/covid19/educationresponse). This challenged education system across the globe and caused a shift to emergency online remote learning. It was not planned, and the distance and online systems were not designed for mass education. Hybrid or blended learning was out of the question at the beginning of the pandemic. In most countries, if they had online education, it was mostly broadcast lessons, so interactions were minimal. The questions then are how did systems fare due to the school closures and how can systems recover post-pandemic?

Early forecasts were showing massive learning losses the longer systems remained shut. Any school closures or learning losses have grave consequences, not just for student’s progress, for also for future earnings. This is because every year of schooling raises earnings by 8 to 10% a year (Psacharopoulos and Patrinos 2018; Montenegro and Patrinos 2021). It is expected that the longer the school closures last, then the larger the learning and earning losses will be.

It is clear that the longer the closure, the bigger the loss in terms of learning. But schooling isn’t really closed since there is distance education. Therefore, it makes sense to ask the question, isn’t there mitigation against loss? But others would point out there is limited capacity to carry out high quality online/distance education. There is weak student connectivity; Internet penetration rates are low in some countries (Goudeau et al. 2021). Other issues include constrained interactions as the teaching tends to be one way, and often students don’t have the opportunity to interact. As the pandemic continued, distance education systems were plagued by poor attendance (Santibañez and Guarino 2021).

School closures result in learning loss. Early simulations forecasted losses of up to one year of schooling. Another simulation showed that if learning in the third grade is reduced by one-third, then learning levels in by the tenth grade would be a full year lower than would have been the case in the absence of COVID-19 (Azevedo et al. 2021; Kaffenberger 2021).

However, it is not just the time that schools are closed that is costly; it is the lost or foregone learning. A decrease in learning could decrease future employment prospects. Lower test scores can lead to significant reductions in earnings. Learning loss could lead to increases in poverty over time (Currie and Thomas 2001; Chetty et al 2014). Past pandemics led to reduction in human capital and well-being. It is estimated that loss in wages equivalent to between 5 and 9% was due to the 1918 influenza. School disruptions due to war and teacher strikes are associated with earnings losses up to 9% (Almond 2006; Ichino and Winter-Ebner 2004; Jaume and Willén 2019). Even back in the fourteenth century, the Black Death (1347–1352)—which resulted in 75 million deaths—led to reductions in labor but higher agricultural wages, while the 1918 influenza—100 million deaths—led to lower incomes overall and other negative social effects that lasted well into the 1980s. Therefore, pandemics reduce income and loss of GDP.

Learning loss is real

Robust studies—with before and after test scores, usually following up a cohort, with co-variates—suggest large early losses in most countries, with very few exceptions (see, for example, Birkelund and Karlson 2021), but even there, losses were typically proportionally higher for disadvantaged or low-income students. Long-term, nation-wide school closures result in learning losses (Donnelly and Patrinos 2021; Hammerstein et al. 2021; Storey and Zhang 2021; Zierer 2021; Moscoviz and Evans 2022). Students could lose 1/3 to a full year of education in terms of learning. The learning losses will lead to earning losses: for the current student when they enter the labor market and for countries in terms of lost productivity as it translates into income losses. Simulations suggest earning losses globally of $15–$17 trillion (Psacharopoulos et al. 2021; Azevedo et al 2021; Hanushek and Woessmann 2020).

This means that schooling matters. An exogenous event such as SARS-CoV-2 induced lockdowns which affect schooling. An absence of schooling led to lower test scores. It is known from the literature that schooling can improve cognitive ability (Schneeweis et al. 2014; Ritchie and Tucker-Drob 2018). Education reforms that that increase compulsory schooling can be used to estimate the effect of schooling on cognitive skills. In Norway, for example, a reform that increased compulsory schooling from 7 to 9 years in the 1960s was shown to significantly increase test scores (Brinch and Galloway 2012). Since learning has an impact on future earnings, one would expect less learning to lead to lower earnings in the future.

But the impact of the school closures wasn’t the same everywhere. This strongly implies that policy matters. For instance, if schools weren’t closed, then there were no learning losses (for example, Sweden didn’t close schools so there was no learning loss; see Fälth et al. (2021). In other countries that did close schools, some were able to mitigate learning losses. In the case of Denmark, students’ online reading behavior increased significantly during closures. This is despite the fact that inequality in reading behavior increased during the first lockdown. But there was only a short-term increase in inequality on children’s actual reading activity (Reimer et al. 2021). In France, early losses were addressed by constant measurement of learning outcomes by the Ministry of Education and helping parents with resources to make online learning more effective (Thorn and Vincent-Lancrin 2021).

Tutoring

One promising policy option for mitigating learning losses during closures as well for subsequent learning recovery and acceleration is tutoring. That is, direct academic support, usually provided by an expert, with deep knowledge or defined expertise in a particular subject or subjects. We are not referring to test preparation (Bray 2006, 2013); but rather high impact or high-dosage tutoring consisting of intensive engagement that occurs one-to-one or in very small groups on a sustained, daily basis, during the school day or after school to help students accelerate their learning (Fryer and Howard-Noveck 2020). Typically, private tutoring is most likely to be taken up by children from affluent households, further widening the disadvantaged gap in learning. But there are now many targeted programs that are publicly funded (Kirkebøen et al. 2021). Yet, it is still expensive. Individualized, intensive, in-school tutoring can cost over $4000 per participant a year in the USA (Fryer and Howard-Noveck 2020; Kraft and Falken 2021).

At the same time, tutoring is highly effective. Randomized controlled trials find that test scores could rise by 0.37 standard deviations—that’s about a year’s worth of learning (Goldstein and Paulle 2021). The effects persist into future years. The estimated benefit–cost ratios are high, from 4.9 to 6.0, making it comparable to many successful model early childhood programs. The literature distinguishes between low- and high-dosage tutoring (Dobbie and Fryer 2013). High-dosage tutoring occurs in groups of 6 or fewer for 4 or more days per week. Schools that implement high-dosage tutoring demonstrate marked treatment effects, with very large impacts. In terms of comprehension, the effects sizes can be as high as 0.55 standard deviations and over 1.6 standard deviations on basic skills. Therefore, high dosage tutoring can increase cognitive outcomes for students from disadvantaged backgrounds. Tutoring improves cognitive outcomes, especially those measured by test scores. Tutoring produces effect sizes of about one-third of a standard deviation. Most studies produced statistically significant positive effects—and no negative effects. The impacts are large by comparison with other interventions, including early childhood programs (Fryer 2017; Guryan et al. 2021; Nickow et al. 2020). It is important that the programs are rigorous; teachers and highly trained professionals are more effective than nonprofessionals. The impacts are found to be largest in the early grades of school. In addition, it matters when tutoring programs take place. On average, they are more effective if they are conducted during the school day.

But the positive effects are not limited to the USA. The experience is positive in Europe as well. Systematic reviews and meta-analyses of tutoring studies, mostly implemented through randomized controlled trials, find very high effect sizes showing that it is indeed possible to substantially improve educational achievement (see for example Dietrichson et al. 2017). In the Netherlands, high-dosage tutoring produces treatment effects of 0.28 standard deviations in math achievement scores after one school year. These treatment effects are sizable and can account for roughly 40% of the math achievement gap between low-income and high-income students (de Ree et al. 2021; see also Mischo and Haag 2002 for the positive experience in the case of Luxembourg).

The replicability of tutoring was demonstrated during the lockdowns and school closures due to COVID-19. There were several experiments that attempted to use tutoring and other similar approaches to mitigate learning loss during the pandemic. These experiments, all of which used technology to a large extent, produced highly cost-effective tutoring programs. These online experiments during the COVID-19 induced school closures showed that such programs could be used in other contexts and the effects could be maintained ever if the programs were not face to face.

Usually, high impact, high dosage tutoring, though very effective at improving learning outcomes, is very expensive, especially in high income countries. Online provides opportunities for more cost-effective tutoring. The COVID-19 experiments have been effective at raising learning outcomes. In Italy and Spain, online tutoring during the pandemic was shown to increase learning outcomes by 0.26 standard deviations in both countries. That is the equivalent of the learning losses that would have occurred otherwise (see, for example, Donnelly and Patrinos 2021).

During the school closure period in Italy, an experiment was conducted to provide free online tutoring to disadvantaged students. The program used volunteer university students as tutors. The tutors were randomly assigned to students from a list of potential beneficiaries compiled by school principals (Carlana and La Ferrara 2021). The program was highly effective in mitigating any learning losses. In Spain, the online tutoring program led to a 17% increase in end- of-year math grades, the equivalent of six months of learning. Children who took part in the program were 30% more likely to pass the subject (math) than children in the control group. The pupils given tutoring also experienced a 17% improvement in the standardized math test designed by the researchers’ pedagogical team (Gortazar et al. 2022). The Spanish model used regular public school teachers as tutors.

Moreover, these online experiments were very cost-effective. The implied costs for raising learning outcomes by 0.1 standard deviation are $21 in the case of Italy (Carlana and La Ferrara 2021) and $127 in the case of Spain (Gortazar et al. 2021). The unit costs are in the hundreds of dollars—$55 in the case of Italy—compared to the thousands of dollars for the face-to-face models in the USA. Also, online tutoring in the USA led to dramatically reduced cost to only $32 per student in one pandemic era online experiment (Kraft et al. 2022).

The use of technology for online education has also worked in low-income environments as well. A randomized control trial in Bangladesh of over-the-phone mentoring provided by volunteer university students was evaluated (Hassan et al. 2021). The program produced significant and large improvements in learning outcomes: 0.75 standard deviations. Effectiveness is said to have been enhanced by the direct of mothers. The program disproportionately benefited students from disadvantaged backgrounds. This was a very low-cost intervention at a total expenditure of just over $8,000 and a unit cost of less than $20. It is estimated that for each dollar invested, there was a 0.038 SD improvement in learning outcomes among treated children. A similar program in Botswana used SMS text messages and direct phone calls (Angrist et al. 2020). The interventions improved learning by 0.16 to 0.29 standard deviations at a unit cost of only $2 in the SMS group and $14 dollars in the phone and SMS group. It is estimated that these programs raise learning by a standard deviation for only $13 in the SMS-only group and $48 in the dual SMS and phone group.

These models show that it is possible to provide quality instruction across the cost spectrum in different contexts and substantially reduce the cost of tutoring. This is very important since cost is one of the biggest barriers to large-scale implementation. In times of crisis, it turns out that it is possible to reach students equitably and efficiently, even those located in remote regions, as long as digital means are available. But it does not have to be Internet/computers; even phones work. Tutoring online allows for the improvement in learning outcomes at very low cost. Recent research suggests a very high cost-effectiveness.

Conclusion

In 2020, most countries closed schools and universities, affecting more than 90% of the world’s learners. School closures can lead to learning losses, which will have grave consequences in the future. In fact, the longer the school closure, the bigger the loss in terms of learning. Two years after the pandemic began, the evidence strongly indicates that school closures result in learning loss. A decrease in learning could decrease future employment prospects and lower future earnings. Learning loss is real. In most countries, with very few exceptions, losses were typically proportionally higher for disadvantaged or low-income students. This means that schooling matters.

The fact that the impact of the school closures wasn’t the same everywhere strongly implies that policy matters. One promising policy option for mitigating learning losses during closures as well for subsequent learning recovery and acceleration is tutoring. Tutoring, especially online, has proven effective in mitigating learning loss. It should be part of any serious implementation of a national learning recovery program.

At the same time, it is important to protect the education budget. Learning recovery is costly, even if online tutoring is cost-effective. As schools re-open, teachers need support and professional development. Education systems need to be rebuilt and prepared for future shocks. Most importantly, before doing any of these things, it is imperative to measure learning outcomes. School systems that addressed learning loss quickly did this first by measuring losses and using assessment to gauge the results of their efforts.

Declarations

Conflict of interest

The author certifies that they have NO affiliations with or involvement in any organization or entity with any financial interest (such as honoraria; educational grants; participation in speakers’ bureaus; membership, employment, consultancies, stock ownership, or other equity interest; and expert testimony or patent-licensing arrangements), or non-financial interest (such as personal or professional relationships, affiliations, knowledge or beliefs) in the subject matter or materials discussed in this manuscript.