A shared approach to tracking research impact: working with data from the Office for National Statistics
Categories: Research using linked data, Blogs, Datasets, Impact, Office for National Statistics, ADR UK Partnership
2 June 2023
In this blog, Louise Corti, Head of Impact at the Office for National Statistics (ONS) Secure Research Service, speaks about how those funding, promoting or leading research using ONS-managed data can work with the ONS to track and showcase impact.
What type of data is available for policy research?
The ONS Secure Research Service is an ONS controlled data environment - also known as a trusted research environment. It hosts a range of powerful research-ready datasets that are available for policy research, helping to investigate topical themes such as the economy, levelling up, inequality, health, and crime and justice.
The most requested datasets in April 2023, comprising economic and business surveys and databases, were:
- Business Structure Database - UK
- Annual Business Survey - UK
- Annual Survey of Hours and Earnings - UK
- Annual Population Survey - UK
- ONS Longitudinal Study - England and Wales
- Labour Force Survey Pension – UK.
At the ONS, we work closely with ADR UK to make available newly linked UK-based administrative data and provide opportunities for research funding to explore these invaluable assets. The most requested ADR England flagship dataset in April 2023 was Growing up in England Wave 1.
Around half of the researchers using the ONS Secure Research Service in the last quarter are from academia. One third is from the public sector. 114 organisations, including many research-active universities, have ‘Assured Organisational Connectivity’ agreements in place with the ONS.
What is the ONS doing to capture and promote impact from this data?
The ONS Analytical Insights and Impact team support impact and engagement work for the ONS Secure Research Service. We track how research is being disseminated and work with selected researchers to showcase any published findings.
Together with the ADR UK Strategic Hub, our team co-create and publish case studies based on researchers' work, or invite them to knowledge-exchange events to share their experiences. These outreach activities help make the value and potential of the data clear by explaining their research aims and methods, how their research is in the public good, and any outcomes and impact.
We also use our knowledgebase of accredited projects to undertake thematic syntheses - such as exploring how data is being used for educational policy research, and what findings and impacts this research may have had. This culminates in published case studies and feeds into thematic knowledge-exchange events with data owners.
Finally, with digital object identifiers (DOIs) now available for ONS Secure Research Service-owned or managed data assets, the ONS has another potential impact capture tool in the bag. We have implemented Altmetric badges to help capture citations across the wider digital landscape (the light blue donut shape below). The badge provides a summary of the volume and type of attention a dataset has received.
If you are going to be publishing in a journal, please make sure you formally reference the data with its DOI in your references section, as journal data availability statements are not currently monitored by Altmetric.
While researchers are expected to report to the ONS what they have published or promoted from their accredited projects, many do not. Project sponsors, funders, and leads can certainly help improve on this. Not only does this comply with their organisation’s Assured Organisational Connectivity agreement, but it is invaluable in supporting monitoring and evaluation exercises required for government-run data creation and data services. Data owners often have to justify the provision of access to data to the research community.
We have developed a simple typology of tangible ‘outputs’, which have yet to become an ‘outcome’. These include the following types of research ‘products’:
- journal publication
- working paper
- book or chapter
- government or funder report
- internal report or working paper
- published presentation
- dissertation or thesis
- news item, interview or blog post
- ONS impact case study
- statistical code
- ONS statistical bulletin
- ONS or Integrated Data Service article
- dataset user guide
- secure dataset
- dashboard.
This classification is used to create dashboards of project activity. A self-reporting form is now available for researchers to tell us about their outputs created from ONS Secure Research Service projects.
Defining research outcomes
Like all impact managers, we are chasing the ‘golden threads’ that connect research questions, analyses, and observed changes in one or more of the six public good outcomes (set out in the Statistics and Registration Service Act 2007 and the Digital Economy Act 2017 Code of Practice).
We have almost completed our work on defining SMARTer ‘outcomes’ for the public good, so that we have a codable classification to represent observed outcomes. As these are notoriously hard to pin down or observe, we use measures that state ‘how close’ the work using our data came to policy or service change. Each of the types set out below has a subtype, and we work collaboratively with researchers to classify their project impact.
Research outcome: main types
- Influenced policy: Observed changes to policies, laws, or service - national, regional, or local
- Economic investment: Investments e.g. funds or service provision given or removed
- Behavioural change: Observed behavioural changes, cultural, or societal
- New product or service: Services excludes public services; includes new data, methods, or technical
- Increase capability: New training elements, courses, modules. Improvements for the workforce
- Intellectual or evidence: Key contribution to body of knowledge (cited a lot), new grants/funding, Research Excellence Framework case study
- Take up or promotion by others: Promotion of outputs or research by others, e.g. campaigns, citations
- Dissemination of findings: Self-promotion of research or outputs shared with others
- Collaboration: Participation or assistance from other organisations or research groups.
We have worked with a range of helpful knowledge brokers, including ADR UK, the Universities Policy Engagement Network, and Association of Research Managers and Administrators. These groups all play a key role in reaching out to academics in their institutions to encourage impact work. The Institute for Government and UK Parliament Knowledge Exchange Unit are also key players in this space for connecting research to decisions makers.
How you can help
What can you do to help the ONS better demonstrate the value and impact of using this data within academia? You can:
- encourage academics in your own network or those you fund to report their research outputs via our self-reporting form
- promote robust citation of ONS data and our new DOIs for data: quick guidance can be found in the SRS catalogue FAQ
- contact us if you know of any great research using ONS-managed data in your organisation - we can help the researchers promote it
- nominate a piece of impactful research using ONS-managed data to test and refine our ‘Outputs to Outcomes’ typology work.
If you want to get in touch, you can find us at IDS.Impact@ons.gov.uk.
