Fifteen years of expert judgement at TUDelft

Over the last fifteen Delft University of Technology (both the Safety Science Group and the Department of Mathematics of TUDelft) has developed methods and tools to support the formal application of expert judgement. Over 800 experts assessed over 4000 variables, in total representing more than 80,000 elicited questions. Applications were made in a variety of sectors, such as nuclear applications, the chemical and gas industries, toxicity of chemicals, external effects (pollution, waste disposal sites, inundation, volcano eruptions), aerospace sector and aviation sector, the occupational sector, the health sector, and the banking sector.
The techniques developed at TUDelft can be applied to give either quantitative assessments or just qualitative and comparative assessments. The application of these techniques is driven by a number of principles, including scrutability, fairness, neutrality, and performance control. The overall goal of these formal methods is to achieve rational consensus in the resulting assessments. Performance criteria are based on control assessments, that is, assessments of uncertain quantities, closely resembling the variables of interest, for which true values (e.g., from experiments) are known post hoc. The use of empirical control assessments is a distinctive feature of the Delft methods. A Procedure Guide for Structured Expert Judgement is published by the European Commission as EUR 18820. This paper highlights the comparative assessments for which the Safety Science Group was the prime responsible.