ADR UK Research Fellows: Grading and Admissions Data for England

The project aims to explore the following research questions: 

• What is the relationship between predicted grades and awarded grades, and how does it vary with individual-, subject- and school-level characteristics?
• How much did the assessment change within schools as a result of the pandemic, and how does any identified assessment modification vary by school-level characteristics?
• What is the effect of any modification in assessment on higher education application outcomes? Are external assessments important for student outcomes?

The methodology used in this study:

The research will take place over three phases, using different methods at each stage.

• For the first phase of the project, a multivariate regression model will be estimated for each cohort. This will show the relationship between UCAS (Universities and Colleges Admissions Service) predicted grades and awarded grades and how individual, subject and school characteristics shape this relationship.
• The second phase will test for the existence of a pandemic-driven modification in assessment within subjects and schools by considering potential changes in the gap between predicted and awarded grades between the pre-2020 and the 2020 cohorts.
• The third phase aims to use the quantitative measure of assessment modification to estimate its effect on higher education application outcomes using a difference-in-differences framework coupled with statistical matching techniques.  

Publications

• Blog: Predicted grades for university admissions and student application outcomes, December 2023
• Data Insight: Gender gaps in teacher grades for university admissions, February 2025

Duration: September 2022 – August 2023

Funding: £107,456

The GRADE fellowship is a partnership between Ofqual, the Department for Education, UCAS and ADR UK.

This project aims to explore the following research questions:

1. Are there systematic differences in how teachers predict the grades of different groups of students, for example by socio-economic status, ethnicity or gender? 
2. Do differences in teacher predictions across different groups of students differ by the subject of A-level qualification? 
3. Do teacher-predicted grades and centre-assessed grades (CAGs) affect the university course students ultimately enrolled in?

The methodology used in this study:

• This study will use a mostly descriptive approach to address research questions 1 and 2, estimating the distributions of predicted and achieved grades for the whole population and conditional on being in certain groups (e.g., by ethnicity, gender and socio-economic status). The project will compare these predicted and achieved grade distributions within and across groups and will also compare actual grades (2018 and 2019) with CAGs (2020) within groups. 
• These descriptive analyses will provide an overview of any discrepancies across groups in teacher predictions. However, if the underlying distributions of “true” grades are very different across groups, this exercise might be less insightful. To mitigate against this, a regression discontinuity design will be used focusing only on students who were ranked very close to grade boundaries by their teachers in 2020. Comparing the characteristics of the students (e.g., their ethnicity, gender, socio-economic status) who were just awarded a higher grade, to those who just missed out will provide insights on whether certain groups are systematically over- or underpredicted. The same approach will also be used to compare the outcomes of students who look similar across a range of characteristics (including being similarly ranked) but who were awarded different grades.

Funded value: £176,314 (full economic cost)

Duration: 15 August 2023 – 15 December 2024

Publications

• Blog: Exploring gaps in teacher judgements and the implications for university admissions, January 2024