Widening the net: Changing how we think about administrative data

Janet Bowstead, a Postdoctoral Research Fellow at the Royal Holloway, University of London and Data Impact Fellow at the UK Data Service, explores how administrative data can support research and how a new approach to it may be needed.

There is increasing discussion about the use of administrative data in the UK – including the intention that even the census will be based on administrative data after 2021.

So, what is administrative data?

Government departments and agencies generate administrative data during their day-to-day activities. They routinely gather information on people, usually when delivering a service, and these records can increasingly be linked across different parts of government and service providers.

Typical administrative data includes social security payment records, educational attainment records, health records, court records, and tax records.

There are ongoing debates, at conferences and in publications, about the feasibility, ethics, rights and appropriateness of such developments in the use of these data.

The UK also has an ADR UK (Administrative Data Research UK) partnership dedicated to “transforming the way researchers access the UK’s wealth of public sector data”, so that such data can be used effectively towards “Good government policy – policy that solves social and economic problems and improves lives” . ADR UK argues that

“Administrative data is an invaluable resource for public good. Let’s use it.”

However, there are other types of administrative data that are often left out of these debates and developments.

These are not strictly government data, but are the kinds of service monitoring data that may be required by local or national government as part of a grant or ongoing funding programme. They may also be required by non-statutory funders, such as charities and trusts.

Some may be collected only for a short-term project, but others are part of longer-term funding arrangements; providing the potential for evidence over time. They also may be required of projects in different areas of the country; and therefore could provide evidence over geography, and different types of places. The datasets may be limited – even fragmentary – and have been collected for a specific purpose, and therefore may not include the exact variables that a researcher or policy-maker would ideally want.

These datasets, however, are often more extensive, and larger sample sizes, than any alternatives of survey or interview data.

I use such service monitoring data in my research on the journeys women and children make to escape domestic violence. These otherwise hidden and secret journeys away from abuse, can be safely researched using these de-identified datasets; which were originally required by government funding, and are now archived with the UK Data Service.

However, such types of administrative data is not often available for research. Administrative data held about individuals is rarely de-identified or archived, and is not usually made accessible for research outside the organisations that hold the data. Such datasets may also be regarded as commercially-sensitive in a competitive tendering environment.

When I presented on my research use of administrative data at the European Conference on Domestic Violence in Oslo in September 2019, delegates from other European countries were aware of service monitoring datasets in their contexts, but none had accessed such data for research. They anticipated similar problems to the UK whereby such data would be seen as too commercially sensitive, or that there was no culture or opportunity to archive such data securely. Data may reside in an organisation’s IT system for a few years, but be effectively lost in IT upgrades after the end of the funding programme that originally required the data collection.

Overall, I was left with the conclusion that we need to change how we think about administrative data.

The important work that is going on in terms of strictly “public sector data”, and how it can be safely and appropriately used for research, policymaking and practice, needs to be widened to address the mass of service monitoring data that are collected – but effectively lost – in such an ad hoc way.

This blog was originally published on the UK Data Service Data Impact Blog.