What is the likely impact of remote learning on educational outcomes?

Even before the Covid-19 crisis, online learning was becoming increasingly prevalent in higher education. Now it is happening in schools during lockdown, what do we know about its impact on students’ performance? And is there a danger it will widen educational inequalities?

In response to the Covid-19 pandemic, we are seeing significant shifts towards online learning across the education sector. The lockdown will eventually come to an end, but remote learning is likely to play an increasingly prominent role in education. How effective might this new format be?

What does evidence from economic research tell us?

Online learning has been an accelerating trend in higher education for some time now, so the best evidence comes from this context. Research suggests no detrimental impacts on students’ performance from a mix of online and face-to-face teaching, although students tend not to perform so well in purely online courses.
But there seems to be variation in student responses. One concern is evidence that disadvantaged students are most likely to fare poorly with online instruction.
Versions of online learning are also happening in schools during lockdown. Uncertainty about the type of provision, together with a lack of evidence on the impact of online learning for younger learners, poses a challenge to assessing impact.
What is clear, however, is that family background is likely to be more important in a remote learning setting where parents are stepping into the role of teachers. Given pre-existing differences in family resources, this suggests a widening of educational inequalities during this period of remote learning for children.

How reliable is this evidence?

Even before the current crisis, online learning has become increasingly prevalent in higher education, and consequently its impact has been studied most extensively in this setting.

Online and blended learning at university

Research on the effectiveness of ‘blended’ learning – a mix of online and face-to-face teaching –generally finds no detrimental impacts on students’ performance compared to face-to-face instruction. This has been found in the context of introductory undergraduate courses, in particular, those that are quantitative in nature, such as economics and statistics (for example, Olitsky and Cosgrove, 2014; Bowen et al, 2014).

On the other hand, evidence focusing on purely online courses is less favourable. Research on ‘massive online open courses’ (MOOCs) consistently documents lower persistence and completion rates (Reich and Ruipérez-Valiente, 2019), although this is likely to be driven in part by self-selection into these types of courses. But more robust evidence finds that taking a course online, instead of in-person, reduces student achievement and the likelihood of staying enrolled in college (Bettinger et al, 2017).

In the context of first year undergraduates, one study finds that among a sample of introductory economics students, those engaging in a purely online format perform worse in a final exam relative to those receiving face-to-face classroom instruction (Alpert et al, 2016). Another study finds more equivocal impacts: attending lectures via live streaming instead of face-to-face lowers the achievement of low-ability first year undergraduate students and increases the achievement of high-ability ones (Cacault et al, 2019). This suggests different impacts for different students.

Indeed, in contexts where students have a choice between online and face-to-face versions of the same course, comparisons that do not account for differences between the types of students choosing online courses over face-to-face instruction – such as age and prior attainment – tend to find no difference in performance between the two formats (see, for example, Gratton-Lavoie and Stanley, 2009). This suggests that online learning can be effective for certain types of students but not necessarily for all. Younger students, lower ability students and male students, in particular, appear to perform poorly with online instruction (Xu and Jaggars, 2013).

As Covid-19 accelerates the adoption of remote learning, our best evidence tells us that the type of online learning provision is important, and that its impact is likely to be different for different students. To the extent that those performing poorly under online instruction are already low-achieving, there could be a risk of increasing educational inequalities.

But as the most robust evidence tends to be focused on economics, it is not clear how these formats would perform for different subjects. Moreover, except for the broad classification of blended or online, the quality of provision is largely unclear, which makes generalising from these findings more difficult.

Remote learning in schools

Widespread adoption of remote learning is taking place in schools in response to lockdown, although this is largely ad hoc in nature. Uncertainty around the type of provision, together with a lack of evidence on the effects of online learning on younger learners, poses a challenge to assessing impact.

For younger age groups, our best evidence on online learning comes from the study of online learning in addition to regular schooling and suggests mixed impact. For example, an evaluation of a personalised, online learning programme in the United States finds significant, positive impacts on the mathematics scores of 7th graders (ages 12/13), while an online programme aimed at boosting literacy produces mostly weak and insignificant results among a sample of primary school age children (Escueta et al, 2017).

Furthermore, it is consistently found that the introduction of technology in the school setting, in and of itself, provides no benefit to students (Education Endowment Foundation).

In the absence of evidence on the impact of remote teaching as a substitute for conventional schooling, it difficult to predict how current online versions of teaching will perform in comparison with its conventional classroom counterparts. But quality is likely to be important.

Some teaching practices that have good evidence for effectiveness such as one-to-one tutoring are likely to be amenable to taking online, but some practices may simply not be possible or may be watered down in a remote setting. For example, we may worry about a loss of opportunities for collaboration with peers, for group discussion or for active learning through activities and games.

In a review of the evidence, the Education Endowment Foundation finds consistently positive impacts of collaborative learning – that is, students working in small groups to complete a collective task, with particularly strong gains from approaches that promote talk and interaction between learners. This is likely to be more difficult to facilitate remotely, though it is worth noting that as teachers gain experience in the online setting and share experiences, no doubt pedagogies for remote learning will improve.

While it is not clear what exactly constitutes high quality provision in a remote setting, preliminary survey evidence suggests substantial variation in online provision among schools during this period of school closures. For example, one study finds that parents from more affluent households are much more likely to report their children’s schools providing online classes and access to online videoconferencing than less well-off parents (Andrews et al, 2020).

Another study shows how provision ranges from essentially ‘business as usual’ in private schools to significantly reduced teaching in the most deprived schools, though it is unclear how representative this survey evidence is (Montacute, 2020). This illustrates how many children are likely to experience multiple layers of disadvantage: poor support from schools on top of household deprivation risks pushing these students further behind their more affluent peers.

Student responses to remote provision

Even if receiving the same remote provision, we would expect there to be differences in students’ responses to distance learning. For both university and school students, individual characteristics and background are likely to matter, which suggests a widening of educational inequalities.

In the context of higher education, evidence suggests that existing abilities are important – only low-achieving students performed worse following a shift to remote streaming of lectures (Cacault et al, 2019).

Furthermore, non-cognitive skills that are strong predictors of educational performance are likely to influence the success of online teaching. Ability to self-motivate, attitudes towards learning and self-beliefs have all been shown to be important determinants of educational achievement (Cunha et al, 2010). For example, after controlling for past grades, college students performing far below expectations are significantly more likely to procrastinate, as measured by the amount of cramming before exams and the amount of delay before starting assignments (Beattie et al, 2016).

All of this suggests that students without skills that are conducive to self-regulated learning may fail to thrive under remote teaching. Insofar as non-cognitive competences vary systematically by socio-economic status (see, for example, Carneiro et al, 2007), some forms of online learning could exacerbate existing inequalities.

Parents and home learning

In the school setting, parental ability is likely to be an important determinant of students’ outcomes under distance learning, particularly for younger children who are both on a steeper learning trajectory and require greater support with learning. Although some parents will be able to step in and support their children’s learning effectively, evidence documents substantial differences between households in parental education, income and time investments, which all influence the ability to provide an enriching home learning environment (see, for example, Kelly et al, 2011).

Indeed, real-time survey evidence from the Netherlands (Bol, 2020) indicates that higher educated parents are more likely to feel capable at helping their children compared with less educated parents. Moreover, these perceptions matter with 70% of highly educated parents reporting helping their child with homework compared with 50% of the least educated parents.

Given that teachers are taking a back seat during this period of home schooling, these household characteristics are likely to be more important for children’s outcomes than schools’ remote teaching provision. As a result, it is likely that inequalities in child outcomes will widen further in this period.

What else do we need to know?

We have evidence from the evaluation of specific online programmes but lack understanding around the ‘active ingredients’ behind the success of online interventions. This makes it harder to generalise the findings to the current situation. It will be important for future research both to understand the effectiveness of the current forms of teaching and to attempt to distil the common elements of effective practice.

Some evidence suggests that low-ability students fare worse under online teaching, although it is unclear whether this is due to lower ability in and of itself or its correlates, such as low motivation. More generally, it will be important to understand the part that non-cognitive skills play in the success of distance learning.

Evidence is mixed as to whether small adaptations to remote online courses could be effective at raising engagement and attainment. One study finds that blocking distracting websites after a pre-agreed amount of time improves student performance and course completion (Patterson, 2015).

In contrast, evidence from a lighter touch intervention (whereby students are required to schedule study time at the start of the course, and are subsequently sent reminders of this self-designed timetable) finds weakly significant negative effects on course engagement, persistence and performance (Baker et al, 2016).

It will be useful for research to continue to explore how online courses can be designed more effectively.

Where can I find out more?

Distance and online learning: The Economics Network provide a collection of advice and resources for running courses at a distance and taking activities that were previously face-to-face online.

Teaching and Learning Toolkit: The Education Endowment Foundation provides an accessible summary of the evidence on teaching 5 to 16 year olds.

CTALE: The Centre for Teaching and Learning Economics at UCL provides online seminars and guides on the changes in education under Covid.