Getting started with administrative data
Administrative data is an invaluable resource for public good, but what is it? And what makes it different from other types of data? Find answers to these questions here.
If you are interested to learn more, this page points to existing introductory courses (self-led and instructor-led). There are also publications to add to your reading list.
What is administrative data?
Administrative data is information created when people interact with public services, such as schools, the NHS, the courts or the benefits system, and collated by government and other public sector organisations.
These public bodies must keep records of these interactions for operational purposes: to enable them to carry out their day-to-day work, to monitor and improve their performance, and keep providing services in an effective way.
For instance, the education departments in England, Northern Ireland, Scotland and Wales collect data on educational attainment to identify schools and subjects where improvement is needed; HMRC collects data on tax and incomes to assess compliance and inform tax policies; and the NHS records details of admissions and appointments to monitor trends in hospital activity. Administrative data also includes basic information about people in the UK, such as notifications of births, deaths and marriages, the electoral register, and national censuses.
What makes administrative data different?
With so many different types of data out there, what makes administrative data special?
The UK’s wealth of administrative data is currently a largely untapped, information-rich resource. Most of this data was not originally created for research, but as a by-product of government services. It can provide powerful insights into our society and point to areas where change is needed.
Have a quick glance at some of the opportunities and challenges of using administrative data for research.
Opportunities
- Often covers difficult-to-reach populations
- Often covers timeframes not feasible in surveys
- Often allows analysis at both national and local levels
- Large sample size, which can cover underrepresented groups
- Can be linked to give a more complete picture of people’s lives
- Potential to understand cross-government programmes and research
Challenges
- Not collected for research purposes
- Lack of cross-government linking policy
- Requires computational skills to analyse
- Often messy, with missing data, biases, and poor metadata
- Subject to changes in collection practice due to e.g., change in policy
- Can involve complex legal gateways or long application process for access
In response to these challenges, ADR UK is working with government and data managers to smooth the researcher journey for data access, and to support the improvement of dataset documentation in line with the FAIR principles. Our flagship datasets are ‘research ready’, meaning that they meet a minimum standard of documentation and are suitably de-identified to protect the privacy of individuals.
Where can I learn more?
Many reputable organisations provide introductory learning resources and courses on what administrative data is and how you can use it. Check out some of the examples below.
CENTRIC
CENTRIC offers training for administrative data researchers in the form of a self-directed series of online modules. This includes an introduction to administrative data, administrative data in the study lifecycle, safeguarding public data, key regulatory considerations, application and approvals, and working with the public.
CLOSER: The home of longitudinal research
The CLOSER Learning Hub has a wealth of information and resources to help researchers better understand longitudinal population studies and how to use the data.
Health Data Research UK
The Health Data Research UK gateway is a platform where you can request access to health data and learn how to use it for research. Additionally, the virtual learning platform Health Data Research Futures is a free and flexible learning platform offering accredited continuing professional development courses. They also have a Data Utility Wizard, which helps users to improve the accuracy of their searches for health datasets for research.
National Centre for Research Methods (NCRM)
NCRM delivers a comprehensive programme of research methods training across the UK. This includes Introduction to Data Linkage theory and methods which is run periodically for those who might use linked data in their work.
Office for National Statistics (ONS)
ONS plays a crucial role in sourcing, linking, and curating public sector data for ADR UK. It also provides statistical training across its Government Statistical Service for statisticians and the wider community across government. Introductory courses include: Awareness in administrative data, Awareness in data linkage, and Quality Assurance of Administrative Data.
RADIANCE
Online pre-recorded lectures and live computer practical sessions for those interested in using administrative data - particularly biomedical and social data.
Scottish Centre for Adminstrative Data Research (SCADR)
SCADR, part of ADR Scotland, hosts the Introduction to Administrative Data Research and Analysis course - a mixture of pre-recorded lectures alongside a series of hands-on practical sessions using synthetic data. Topics covered include: what is administrative data and its key characteristics, sources of available administrative data, and how to deal with common problems working with administrative data.
UK Data Service
UK Data Service provides access to the country’s largest collection of economic, population and social research data, including administrative datasets. Amongst this collection, the Service offers secure access to linked and linkable datasets from the Centre for Longitudinal Studies (CLS) and NHS Digital, the National Pupil Database, the National Referral Mechanism and Duty to Notify Statistics, UCAS admissions service data and Ofsted data. You can also access free data training and resources via their Learning Hub [ukdataservice.ac.uk].
UK Longitudinal Linkage Collaboration
UK Longitudinal Linkage Collaboration (UK LLC) is the national trusted research environment for longitudinal research. It is a collaborative endeavour with many of the UK’s most established longitudinal studies. Its mission is to provide a data linkage resource to its partner studies and a simple one-application process to UK based researchers applying to access linked longitudinal data. Visit their website to see what longitudinal data is available for research and read their set of ‘How to Guides’ for researchers.
Datacise Open Learning – Precision Training for Data Science in Research
Datacise Open Learning aims to provide innovative training for a range of data science disciplines for those with an interest in health or administrative 'big' data. Datacise aims to support skills and learning development to enhance future research endeavours across social science, health and social care, healthcare and clinical research. Datacise Open Learning is the training and capacity-building arm of ADR Wales.
The data journey
For information on how to access data and the data journey, see Data Access.
Browse the slideshow below to learn about the data journey for access using the Digital Economy Act 2017. This slideshow has been provided by the UK Statistics Authority.
Glossary of terms
The world of administrative data can often seem complex and full of technical terminology! Explore the ADR UK glossary to learn about the key terms for administrative data research, as well as the social sciences in general.
Reading list
If you want to learn more about administrative data, consider adding some of the below to your reading list. This reading list will continue to be updated.
For a full list of ADR UK news and publications, head over to News & Blogs and Publications & Reports.