Discover peer-reviewed journal articles and reports that use ADR UK-funded flagship datasets. This collection is being expanded over time.
In 2020, COVID-19 forced the cancellation of all student end-of-school examinations in England. Schools were asked to provide centre assessment grades (CAGs), offering their best estimates for what students would have achieved had they sat their examinations. Although initially ignored in favour of grades calculated via an algorithm, students were eventually awarded their CAGs following widespread public outcry over the calculated grades. Whether CAGs were unfairly awarded across different student groups and schools in 2020 compared to previous years is a key question. However, existing analyses of bias in CAGs are limited by a lack of attention to potential interactions between student characteristics, and thus to hidden differential grade inflation across intersectional groups. We address this by examining student GCSE performance in 2018, 2019, and 2020 via a Multilevel Analysis of Individual Heterogeneity and Discriminatory Accuracy (MAIHDA) analysis of intersectional sociodemographic variation which we cross-classify with schools given their role in generating CAGs. Overall, a picture of stability emerges, where despite substantial overall grade inflation in 2020, the use of CAGs does not appear to have generated new or divergent intersectional relationships in comparison to previous years, suggesting CAGs showed a similar susceptibility to bias as normal examinations.
Dataset used: Grading and Admissions Data for England
Read more
The COVID-19 pandemic meant that, in 2020, students in England were unable to sit their examinations and instead received predicted grades, or “centre assessment grades” (CAGs), from their teachers to allow them to progress. Using the Grading and Admissions Data for England (GRADE) dataset for students from 2018 to 2020, this study treats the use of CAGs as a natural experiment for causally understanding how teacher judgements of academic ability may be biased according to the demographic and socio-economic characteristics of their students. A variety of machine learning models were trained on the 2018–19 data and then used to generate predictions for what the 2020 students were likely to have received had their examinations taken place as usual. The differences between these predictions and the CAGs that students received were calculated and then averaged across students’ different characteristics, revealing what the treatment effects of the use of CAGs were likely to have been for different types of students. No evidence of absolute negative bias against students of any demographic or socio-economic characteristic was found, with all groups of students having received higher CAGs than the grades they were likely to have received had they sat their examinations. Some evidence for relative bias was found, with consistent, but insubstantial differences being observed in the treatment effects of certain groups. However, when higher-order interactions of student characteristics were considered, these differences became more substantial. Intersectional perspectives which emphasise interactions and sub-group differences should be used more widely within quantitative educational equalities research.
