What have two years of interrupted schooling taught us about learning?

Covid-19 has disrupted education for children and young people around the world. Stalled progress in learning continues, as does widening inequality. Additional support for schools is urgently needed to reduce learning gaps.

Covid-19 and the policy efforts to contain it have radically disrupted the everyday lives of children around the world. School students’ learning became a concern early on, with the World Bank, the European Commission and other bodies predicting a sharp drop in progress (Azevedo et al, 2020; Di Pietro et al, 2020). This article highlights how these predictions have largely been borne out, and shows that efforts to reverse the damage have only had partial success.

The effects of the pandemic on education are commonly referred to as ‘learning losses’, but here we refer to the delays in learning as a ‘learning deficit’. This is more accurate given that in most cases, students will not come back knowing less than they did before the pandemic. More commonly, they will have made some progress, but less than they would have in a normal year.

Although the data are still coming in, evidence of a Covid-19 learning deficit is clear. Deficits opened up early in the pandemic and they have remained largely constant. Children and young people from socio-economically disadvantaged backgrounds have been most vulnerable.

Urgent policy action is needed to address these setbacks. This should target disadvantaged school students especially, but not exclusively.

What is needed to assess the Covid-19 learning deficit?

To understand the full extent of the learning deficit, it is important to ask first what kind of evidence is needed and why it is hard to find.

Frameworks to monitor school students’ progress are underdeveloped and many existing monitoring systems have been disrupted by the pandemic. A key lesson of the past two years is that we need to collect data on learning with the same commitment as we do on the economy or healthcare.

An early influential study from the Netherlands has become something of a gold standard (Engzell et al, 2021). The researchers studied learning during the initial months of school closure, with national exams that occur twice a year.

In 2020, these happened just before schools closed early in the spring, and then when students came back to the classroom in summer. This allowed the researchers to assess the progress that students had made at home during this time and compare it with the same period in previous years, when students were in school as normal.

The study reports two main findings. First, despite enormous efforts by parents and teachers, students learned very little during eight weeks of home schooling. Not only that, but students whose parents were less well educated suffered the largest setbacks.

Most other evidence on the learning deficit is incomplete because it does not follow the same students over time. What if only some students return to take tests after a period of school closures? How do we know how those students were performing before the pandemic? Were the types of tests used to evaluate students during the pandemic comparable to the tests used prior to the pandemic?

Many studies also use samples of schools that are not representative of the broader student population. Despite these limitations, most other studies tend to reach similar conclusions.

What is the evidence for the Covid-19 learning deficit?

Several other studies have shed light on the effects of Covid-19 on learning. A recent systematic review identifies more than 30 studies across 12 countries that use standardised test scores of students in primary or secondary schools (Betthäuser et al, 2022). To arrive at this number, the authors screened more than 5,000 scientific studies, and performed a search of studies citing or being cited by the relevant sources initially identified.

Figure 1: Estimates of learning deficits by date and country

Source: data compiled by Betthäuser et al, 2022

Figure 1 shows the estimated learning deficit that each study arrived at, separately by country and date of measurement. The same study often contains more than one estimate (for example, for different subjects or students at different ages). The horizontal axis shows the time of measurement, and the vertical axis shows the size of the estimated learning deficit. The size of the circles reflects the study sample size, with larger studies represented by bigger circles.

On average throughout the period, the estimated learning deficit is 0.17 standard deviations (SD) – a statistical measure of effect size. Students usually improve their performance by between 0.3 and 0.5 SD per year in normal circumstances.

Consequently, a 0.17 SD learning deficit can be interpreted as saying that students lost out on about 43% of a school year’s worth of learning (0.17/0.4). This is about twice as much as the Dutch study found after eight weeks of remote learning. Most estimates by studies from the UK are close to that average (see the list compiled by the Education Endowment Foundation).

The estimated learning deficit does not change with time: the average in Figure 1 is constant throughout the observed period.

What should we make of this? One interpretation is that the worst learning deficit occurred early on. This was a time of chaos when no one knew how long school closures would last and there was little opportunity to prepare. After that, teachers, students and parents have become better at coping with remote learning, although not enough to offset the initial setback.

What is the evidence for increased inequality?

One of the key findings of the study from the Netherlands was that students from less well-educated homes suffered the largest learning deficit.

Several other studies have examined how the pandemic has affected students from different backgrounds. This evidence is harder to summarise because different studies use different measures – such as eligibility for free school meals, family income or parents’ education – and the results cannot easily be compared on the same scale. Nevertheless, we can count the number of studies that found evidence for widened inequalities.

Figure 2: Evidence for effects of Covid-19 on educational inequality

Source: data compiled by Betthäuser et al, 2022

Figure 2 shows that there is strong evidence of increased socio-economic inequality in learning in the wake of Covid-19. Each circle or square refers to an estimate, and the results are separated by when student progress was measured, in what subject, and in what school grade.

The evidence is clear: the large majority of studies find that socio-economic inequalities have widened. This holds at each stage of the pandemic, for both mathematics and reading, across primary and secondary education, and independently of how socio-economic background was measured.

What are the wider lessons and what needs to be done?

What, then, did two years of interrupted schooling teach us about learning? If anyone doubted the value of the hard work of teachers and young people in classrooms around the world, they should have had their doubts dispelled.

The findings on the Covid-19 learning deficit highlight that like so many things that people often take for granted, the day-to-day functioning of our schools is fragile and depends on the continuous effort, dedication and goodwill of everyone involved.

There are three challenges ahead. First, in the vast majority of countries around the world we still lack data on the consequences of the pandemic on student learning.

We may be able to learn from the Netherlands in the way that researchers were able to monitor and assess the situation as it unfolded.

We should also ask what structures could be put in place to ensure that we protect the young in crises and give them the resources that they need. If governments are serious about schooling, they need to collect data on a continuous basis and make them available for researchers to assess.

The second challenge is related to limiting the long-term consequences of the learning deficit caused by Covid-19. The picture of a stable learning deficit is not as dramatic as many had feared. There were initial worries that small setbacks would cumulate into large losses over time (Kaffenberger, 2021; Fuchs-Schündeln et al, 2021).

That fear is not borne out by the data. But if we are to undo the initial damage of the pandemic, we need to step up our efforts. In the Netherlands, the government devoted €8.5 billion to an educational stimulus package, of which €5.8 billion is earmarked for primary and secondary education. This package contains many things that can be emulated by other countries, such as more school staff and increased mental health support and summer tutoring.

The third challenge is whether we can learn something from countries where the learning deficit was less pronounced or even absent.

One well-designed study from Denmark found surprising results (Birkelund and Karlson, 2021). Secondary school students saw a learning deficit close to that in the Netherlands, but younger students improved their performance by an even wider margin. As older students were kept home for a longer period, the younger ones benefited from increased resources and teacher density. Another factor may be that the Danish welfare state shielded parents from economic uncertainty.

Regardless, these results suggest that the Covid-19 learning deficit can be prevented or undone, if decision-makers show enough resolve.