Have the cats been herded? An evaluation framework for simultaneous public interventions

The evaluation of activities, programmes, and projects is key to learning about the impact of interventions, including public policy initiatives. This also helps in establishing accountability of the organisations that run those programmes and projects. Though there has been significant research interest in impact evaluation, literature has focused on firm outcomes, and measuring the impact of individual interventions or competitions. The evaluation of the impact of simultaneous public interventions, considering the collective impacts and the potential interactions among outputs has received little consideration. This paper aims to fill this knowledge gap by developing a framework to evaluate the impact of simultaneous public interventions with the same overarching goals. A theory-based approach is proposed for the identification of synergies between interventions. A pathway-based analysis to understand the influence of interventions is also proposed. Interventions by the UK's Department for Business, Energy and Industrial Strategy in smart energy innovation are used as a case study to showcase the proposed method. Inferences from the developed Theory of Change highlight the interventions that are expected to make the greatest and the smallest impact, and the end benefits that are the most likely and the least likely influenced by the interventions. Such analysis is beneficial for planning future interventions, as it enables the development of more focused interventions, and for planning an evaluation, as it highlights research focus and data collection efforts. The approach, though detailed for the smart energy innovation theme, is suitable for replication and reproduction across multiple interventions by any public agency.