Feasibility of using linked dataset to evaluate early interventions to prevent violent crime

In this blog, Dr Rosie Cornish of the University of Bristol describes the findings of her research. Her project team has been examining the feasibility of using the de-identified Ministry of Justice (MoJ)-Department for Education (DfE) linked dataset to evaluate early interventions to prevent violent crime.

The wealth of evidence on the link between childhood adversity and later perpetration of violence shows us that the seeds of violence in ten years’ time are being sewn in childhood experiences today. We also know that, despite our understanding of the causes of violence, there is insufficient evidence on what works to prevent this problem.

Rates of violent crime in England and Wales have increased in recent years, resulting in large amounts of funding being allocated to violence prevention initiatives. It is crucial that such initiatives are evaluated in order to establish what can be done to successfully reduce this problem.

There are two key issues that will impact on the quality of this sort of evaluation. Firstly, there is a need for valid, reliable data sources that measure outcomes both before and after the intervention has been implemented. Secondly, it is important to have a well-matched comparison group. Without this, it is difficult to draw any clear conclusions about the effect of the intervention because any changes in rates of violent offending could arise because of other factors (i.e., may not be due to the intervention itself).

In 2020, the DfE and the MoJ created a large dataset linking together de-identified education and crime data. This dataset has the potential to overcome these two issues. Our research team from the University of Bristol and the University of Hull wanted to find out whether this linked dataset could be used to test if interventions aimed at reducing violent crime are effective. In particular, we wanted to address the second of these two key issues (having a well-matched comparison group).

Our project had two objectives:

1. To assess the quality of the dataset (in terms of its suitability for this and other purposes).

2. To assess whether the dataset could be used to identify suitable control groups for evaluating violence prevention interventions.

The dataset we used included just over 15 million people born in England between September 1985 and August 2007 with education records, linked to just under 1.85 million individuals with crime records. We found the quality of the data to be generally high, with low levels of incompleteness and minimal inconsistency across most key information.

To address objective two, we chose a promising intervention called multi-systemic therapy and mimicked how this might be implemented in practice. We selected high-risk young people (those who are more likely – on average - to get involved in serious violence) for inclusion in the intervention group. We then compared two different matching methods (called prognostic score matching and coarsened exact matching) in terms of how well they identified a control group that was well-matched to the intervention group. We found that the second of these was more effective at finding a suitable control group.

Our overall conclusion was that the dataset could be used to find suitable control groups for evaluating interventions aimed at reducing the risk of serious violent offending in high-risk young people. However, for this to be practical for evaluating real interventions, the linkage would need to be refreshed and the dataset updated on a regular basis. There would also need to be an efficient and sustainable mechanism for accessing the dataset for such purposes.

The findings of this project and the existence of this dataset are important for the future of violence prevention research as we now know we can identify good comparison individuals from across England. This will greatly increase the efficiency and quality of research in this area, helping us get better and faster at learning what works to prevent youth violence.

Read the two reports detailing our findings which was funded by ADR UK.