ADR Scotland develops new infrastructure to enable faster, more efficient data access for research
10 November 2020
Enhanced data linkage infrastructure minimises burden on data controllers while supporting secure use of data for research in the public interest.
ADR Scotland, a partnership between the Scottish Government and the Scottish Centre for Administrative Data Research (SCADR), aims to make the data linkage process in Scotland more efficient by establishing and expanding a safe and secure environment to host Scottish public sector administrative data in a format that facilitates data linkage.
While Scotland’s data infrastructure has been enabling high quality research in a secure and ethical way for many years, the enhanced data linkage infrastructure developed by the partnership will provide a source of up-to-date, linkable data. This will reduce the time and effort required by both researchers and data controllers to implement successful data linkage and non-linked data projects.
Reduced administrative burden for data controllers
ADR Scotland’s approach is designed to ensure that data controllers maintain control and visibility over how their data is used but also to save both time and effort and ease the administrative burden caused by repeated data access requests.
Rather than having to extract, review and provision data for each request, data controllers will have the option to provide their data to ADR Scotland only once, where it will be held in a reusable format. Data controllers will also have the option to update their data as required.
Research in the public good
Projects using data held under ADR Scotland will continue to be assessed by statistical, research and data protection experts and only approved for access if the public benefit outweighs the privacy risk. Researchers will continue to get access only to the required data for their research project when they demonstrate that the findings are in the public good and have the potential to improve lives.
A highly secure computing environment
Administrative data is usually personal data. It therefore needs to be stored in a secure location with restricted access, and handled and processed in a confidential and secure manner in accordance with the law and data protection principles.
For ADR Scotland, data will be stored and processed in the data linkage infrastructure, comprising of the people, processes and technology that support the ADR Scotland model of data linkage. This is illustrated by Figure 1 below.
Data will be stored and processed in the data linkage infrastructure. The technology centres on a highly-secure computing environment where data is uploaded, stored and processed. A key feature of the infrastructure is that a single organisation will never have all the information about a complete dataset or an individual. This is known as ‘a separation of functions’ approach and ensures a high level of security.
The data stored will be separated into three parts, all of which will be held separately:
- The direct personal identifiers (such as names, dates of birth or Scottish Candidate Numbers) for indexing
- The demographic variables, such as geographical information, gender, age
- The storage data, which includes all the other variables agreed to be shared for research.
The indexing will be performed by NRS and storage and linkage will be performed by EPCC, with storage data, demographic data, and linkage all being held in separate, distinct areas within the infrastructure, and accessed by separate individuals within EPCC.
Using this ‘separation of functions’ helps to protect privacy and contributes to keeping any risk of individuals being identified to an absolute minimum during the data storage and linkage process.
Figure 1. This simplified diagram outlines how data enters and moves through the ADR Scotland data linkage infrastructure – dotted lines represent different parts of the secure National Safe Haven.
Creating a sustainable research resource
Roger Halliday, ADR Scotland co-director and Chief Statistician for the Scottish Government welcomed the enhanced infrastructure, commenting: “By reducing the time and effort required by researchers to access data for research projects, ADR Scotland is opening up new opportunities for research using administrative data that is in the public good and has the potential to improve lives across Scotland."
He added, “These infrastructure enhancements also provide a very strong foundation for the ongoing development of Research Data Scotland that aims to improve access to and linkage of data about people, places and businesses for research in the public good.”
ADR Scotland will continue to build on the existing data linkage landscape in Scotland by developing curated, themed data sets that will be maintained and available for repeat use to answer new and different research questions. Ultimately, this will be a sustainable research resource that represents greater value for money, and more efficient use of data already collected.