Splink, a cutting-edge, large-scale data linkage tool developed out of Data First – an ambitious data linkage project led by the Ministry of Justice (MoJ) and funded by ADR UK – has won the Methods Category of the Analysis in Government Awards 2020.
Within the Data First programme sits the Internal Data-Linking Team who developed Splink - their answer to modernising the MoJ approach to data linkage. Prior to Splink, existing methods were limited by technology, rendering them slow, unscalable, deterministic or hidden behind software.
With Splink, there is now an open-source solution to data linkage at scale, with improved transparency of methodology using a variety of diagnostic tools. Splink has the capacity to link a hundred times the number of records previously possible, and improved accuracy from the development and refining of several algorithms.
These improvements have hugely benefitted the Data First programme, enabling MoJ to link magistrates’ and Crown Court data, and generating smooth linkages as data becomes available, with minimal lag and greater accuracy. This new approach has been critical to enabling de-identified, research-ready datasets to be made available to accredited researchers via the Office for National Statistics’ (ONS) Secure Research Service, thereby creating the potential to generate important new insights to inform policy and practice.
Since its inception, Splink has been downloaded over 90,000 times, and is being tested within the Department for Work and Pensions, the Department for Education, and even the Colombian Department of Justice. It also features in the ONS cross-government review of linkage and is now part of their core methods library.
The Analysis in Government Awards received over 100 nominations this year. Splink has been recognised as an innovative method of analysis, winning the Methods category – one of five within the Awards. The judges felt that Splink is an impressive tool that will be greatly beneficial across Government for large data linking. It has been peer reviewed by academics and methodology experts at ONS and adopted by other departments. It has improved efficiency within MoJ by replacing existing data linking projects.
Dr Emma Gordon, Director of ADR UK, said: “This year has shone a light on the need for accurate data and improved methodology in data science and research. We are delighted that our investment had enabled the creation of Splink, which has not only benefitted data analysts, researchers and policymakers within MoJ, but is improving data methodology across other government departments – both here in the UK and abroad. This is a prime example of how ADR UK projects lead to sustainable benefits for government use of data across the board.”
Accepting the award on behalf of the Data First Internal Data-Linking Team, Robin Linacre, data scientist at MoJ, said: “We're thrilled for our team's work to be recognised. It's been an absolute pleasure to collaborate with so many talented people across the Data First programme from both within and beyond the department. This work was very much a team effort and would not have been possible without everyone's contributions. I'm confident that this work will pay dividends for many years to come in terms of the quality of our data, and the research it enables.”