Coronavirus has led to high-stakes summer examinations being cancelled and the grades replaced by teacher predictions, or administered in a new online format. It is vital to understand the potential impact on the lives of young people affected by these disruptions.
The outcomes of examinations at key points in young people’s school and university careers have potentially lifelong consequences. How are their future prospects likely to be affected by the cancellation of this summer’s GCSEs and A-levels, and the shift of university assessments to new online formats?
What does evidence from economic research tell us?
- Changes to high-stakes exams matter because there is strong evidence that the educational qualifications that they confer have important effects on people’s economic and social outcomes over their lifetimes.
- Important school exams – GCSEs and A-levels – will be replaced this year by teacher predictions. The exam boards will then moderate the teacher grades based on the previous performance of pupils in that school. Overall, the assessments that teachers make are reasonably correlated with average exam performance; they are less reliable when it comes to making predictions about the performance of an individual pupil.
- One concern is what is known as `statistical discrimination’ – teachers’ knowledge about the average characteristics of a group affect the predictions they give for individual members of that group. For example, evidence suggests that some teachers exhibit bias towards ethnic minority and pupils with special education needs.
- Many university exams are still taking place, although in a new online format. Higher pass rates or greater uncertainty in scoring for these exams compared with normal exams could affect the ‘signalling’ value of qualifications to future employers.
- The pupils and university students most affected during this period of educational disruption may not be those facing exams in this year but those due to take exams in the following academic year.
- Lost teaching for year 10 (age 14-15) and year 12 (age 16-17) pupils could affect both their future exam performance and decisions around post-school transitions.
- Even if exam boards and universities award the same proportion of each grade the following year, if there is less learning, this may have longer-term implications for young people’s skills and job prospects.
How reliable is the evidence?
Performance in high-stakes exams matters for young people’s futures. For example, pupils who just fail to achieve a grade C in GCSE English are less likely to enter an upper-secondary high-level academic or vocational track or to attend tertiary education, compared with pupils who just pass the exam (Machin et al, 2018). They are also more likely to drop out of education before the age of 18 without some form of employment.
In a university context, there are differences in the financial returns for students based on the degree class that they achieve: in a sample of students born around 1970, the highest performing students (those attaining a first class or upper second class degree) earned around 7% more than students with a lower second class or lower degree by the age of 30 (Naylor et al, 2015). This highlights exam performance as a key determinant of the trajectories of lifelong earnings.
Teacher assessments
In response to the Covid-19 outbreak, high-stakes school exams – GCSEs and A-levels – have been cancelled and grades will instead be awarded on the basis of teacher predictions, moderated with statistical evidence on the performance of pupils at the school in previous years (Department for Education, 2020).
Evidence suggests that teacher assessments are reasonable predictors of future exam achievement. A review of over 75 studies reports a moderate to strong correlation between teachers’ judgments of pupils’ academic achievement and pupils’ performance on standardised achievement tests (Südkamp et al, 2012). In the UK context, teacher assessments are highly correlated with GCSE and A-level results (Rimfeld et al, 2019). Standardised test results are themselves not 100% reliable – in 2018, around 20% of all GCSE grades challenged were changed (Ofqual, 2018).
If accurate, teacher assessments could also have advantages over high-stakes exams due to reductions in exam-related stress and anxiety. Similarly, elimination of high-stakes exams could be beneficial for pupils who do not perform well in high-pressure environments. For example, there is a well-documented gender gap in performance in competitive environments (Gneezy and Rustichini, 2004).
In the context of exam performance, one study conducted in Spain finds that although girls perform significantly better than boys in classroom tests, in national exams, boys perform slightly better than girls (Azmat et al, 2015). Pupils who struggle to perform well in high-pressure exams could benefit from a shift to teacher assessment, although we might expect some gender bias with girls benefiting from this shift more than boys.
Potential biases
Although teacher assessments may perform well in terms of population-level correlations, their use in individualised prediction is less reliable. For example, there is evidence that teacher predictions can disadvantage certain groups of pupils due to teacher biases or stereotyping.
In the context of primary school pupils, there is evidence of systematic ‘under-assessing’ and ‘over-assessing’ of ethnic minority groups relative to their white peers (Burgess and Greaves, 2009). In particular, Black Caribbean and Black African pupils are more likely to out-perform their teachers’ expectations relative to white pupils, while Indian, Chinese and Mixed White Asian pupils are more likely to fall short of teacher predictions.
The data in that study support a model of stereotyping in which predictions of individual pupil performance are affected by teachers’ information on the past performance of members of that group from previous years. Indeed, there is evidence that teacher assessments can be influenced by a range of pupil characteristics. For example, there is evidence of teacher biases across multiple dimensions, including income level, gender, ethnicity and special educational needs status (Campbell, 2015).
The issue of statistical discrimination is all the more relevant for older pupils for whom predicted grades play a key role in university admission. UK university admissions data show that only 16% of pupils achieve the A-level grades predicted by their teachers, with the vast majority of pupils over-predicted (Murphy and Wyness, 2020).
Among the highest performing pupils, those from disadvantaged backgrounds receive lower predictions than more advantaged pupils. While the average impact of over-prediction will be moderated by the exam boards in their statistical adjustments, for individual pupils, there is clearly a risk that they will not be awarded the grade that they might have achieved if they had taken the exam.
Evidence on statistical discrimination in the classroom suggests that some groups of pupils are less likely to receive teacher assessments that are reflective of their true abilities. Placing greater weight on teacher assessment in the allocation of grades could therefore disadvantage some groups relative others, risking widening educational inequalities.
That said, with international students less likely to take up their places at university this coming year, even if some pupils do get lower grades than they might otherwise have done, there may well still be opportunities for them to go to university. Ensuring that young people are aware of their options even if they miss their required grades, along with encouragement to continue to university if they want to, will be important. This is particularly the case for young people from families who are not familiar with the university system.
Higher education
In the higher education setting, many exams have been shifted online. If the new format means that these exams become easier to pass, we could see cohorts of apparently more highly qualified graduates, which could have positive effects on labour market success.
For example, amid student riots in 1968, many exam procedures were abandoned in France, leading to higher than usual pass rates (Maurin and McNally, 2008). As a result, it became easier for young people to progress into higher education and the associated increase in years of education had a positive impact on later earnings and occupation levels.
On the other hand, if more young people pass or if scoring becomes more uncertain compared with normal exams, qualifications may lose their value as a signal of ability to future employers. Instead, firms may attempt to differentiate candidates by other means and issues could arise if employers turn to other characteristics or measures of ability that inadvertently disadvantage certain groups.
For example, the in the United States, ‘ban the box’ – a policy that prevented potential employers asking about a candidate’s criminal history – had the unintended consequence of reducing hiring among young, low-skilled black men, a group who are statistically more likely to have a criminal record (Doleac and Hansen, 2018).
Next year’s exams
The young people most affected during this period of educational disruption may not be those facing exams this year, but those due to take exams in the following academic year. Over this period of school closures, pupils in year 10 and year 12 will be missing critical teaching time and opportunities to accumulate knowledge ahead of next year’s assessments.
In the absence of changes to next year’s exams or compensatory teaching, this could affect both future exam performance and decisions around post-school transitions. For example, evidence points to educational attainment as the biggest driver of educational participation (Dickerson and Jones, 2004). This suggests that poor performance in next year’s exams could discourage young people from pursuing further education.
Moreover, peer effects, attitudes towards school and employment opportunities have been highlighted as important drivers of post-16 educational choices (Meschi et al, 2018). Time spent away from school and peers during this period as well as changing employment opportunities could serve to affect important educational decisions.
Of course, exam boards and universities can adjust grades statistically to ensure that similar proportions of pupils and students are awarded the same grades as previous years. But this too might have long-run implications in terms of young people’s long-term skills development and employment prospects.
What else do we need to know?
In the current climate, there are many factors shaping the choices facing young people at the ages of 16 and 18. For example, shifts in university teaching towards online provision may make universities less appealing, yet deteriorating labour market conditions may encourage young people to remain in education for longer. It will be important to understand the net effect of these shifts and how it may vary by young people’s characteristics.
Although cohorts graduating from school and university this year and the next may emerge with the same distribution of grades as previous years, it will be important to understand the impact of missed learning and missed experience with high-stakes testing on the development of their skills and employability.
Where can I find out more?
Why have we waited until now to improve the accuracy of predicted grades? Gill Wyness highlights concerns around the use of teacher predicted grades in university applications and reflects on the opportunity that the current crisis presents in improving grade predictions.
Lockdown and the 11-plus: Matt Dickson and Lindsey Macmillan look at the implications for socio-economic inequality of going ahead with the entrance exam for grammar schools (the 11-plus) this autumn.
The COVID-19 crisis and educational inequality: Anna Vignoles and Simon Burgess discuss the impact of learning loss during lockdown and exam cancellations on educational inequality, as well as policy responses to mitigate adverse effects for disadvantaged students.
Education and lockdown: skills, transitions and inequalities: In this webinar, Anna Vignoles and Simon Burgess consider the effects of school closures and exam cancellations on the accumulation of skills and on post-compulsory school transitions as well as short- and long-run policy responses to support young people.
