Expert elicitation for reliable system design

Research output: Contribution to journalArticle

40 Citations (Scopus)

Abstract

This paper reviews the role of expert judgement to support reliability assessments within the systems engineering design process. Generic design processes are described to give the context and a discussion is given about the nature of the reliability assessments required in the different systems engineering phases. It is argued that, as far as meeting reliability requirements is concerned, the whole design process is more akin to a statistical control process than to a straightforward statistical problem of assessing an unknown distribution. This leads to features of the expert judgement problem in the design context which are substantially different from those seen, for example, in risk assessment. In particular, the role of experts in problem structuring and in developing failure mitigation options is much more prominent, and there is a need to take into account the reliability potential for future mitigation measures downstream in the system life cycle. An overview is given of the stakeholders typically involved in large scale systems engineering design projects, and this is used to argue the need for methods that expose potential judgemental biases in order to generate analyses that can be said to provide rational consensus about uncertainties. Finally, a number of key points are developed with the aim of moving toward a framework that provides a holistic method for tracking reliability assessment through the design process.
LanguageEnglish
Pages428-462
Number of pages34
JournalStatistical Science
Volume21
Issue number4
DOIs
Publication statusPublished - 2006

Fingerprint

Elicitation
Reliability Assessment
Design Process
System Design
Systems Engineering
Expert Judgment
Engineering Design
Statistical Process Control
Risk Assessment
Large-scale Systems
Life Cycle
Uncertainty
Unknown
System design
Expert elicitation
Requirements
Design process
Systems engineering
Context

Keywords

  • expert judgement
  • elicitation
  • reliability

Cite this

@article{06b811814dc64fd9b0288fa56095d8cb,
title = "Expert elicitation for reliable system design",
abstract = "This paper reviews the role of expert judgement to support reliability assessments within the systems engineering design process. Generic design processes are described to give the context and a discussion is given about the nature of the reliability assessments required in the different systems engineering phases. It is argued that, as far as meeting reliability requirements is concerned, the whole design process is more akin to a statistical control process than to a straightforward statistical problem of assessing an unknown distribution. This leads to features of the expert judgement problem in the design context which are substantially different from those seen, for example, in risk assessment. In particular, the role of experts in problem structuring and in developing failure mitigation options is much more prominent, and there is a need to take into account the reliability potential for future mitigation measures downstream in the system life cycle. An overview is given of the stakeholders typically involved in large scale systems engineering design projects, and this is used to argue the need for methods that expose potential judgemental biases in order to generate analyses that can be said to provide rational consensus about uncertainties. Finally, a number of key points are developed with the aim of moving toward a framework that provides a holistic method for tracking reliability assessment through the design process.",
keywords = "expert judgement, elicitation, reliability",
author = "T.J. Bedford and J.L. Quigley and L.A. Walls",
year = "2006",
doi = "10.1214/088342306000000510",
language = "English",
volume = "21",
pages = "428--462",
journal = "Statistical Science",
issn = "0883-4237",
number = "4",

}

Expert elicitation for reliable system design. / Bedford, T.J.; Quigley, J.L.; Walls, L.A.

In: Statistical Science, Vol. 21, No. 4, 2006, p. 428-462.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Expert elicitation for reliable system design

AU - Bedford, T.J.

AU - Quigley, J.L.

AU - Walls, L.A.

PY - 2006

Y1 - 2006

N2 - This paper reviews the role of expert judgement to support reliability assessments within the systems engineering design process. Generic design processes are described to give the context and a discussion is given about the nature of the reliability assessments required in the different systems engineering phases. It is argued that, as far as meeting reliability requirements is concerned, the whole design process is more akin to a statistical control process than to a straightforward statistical problem of assessing an unknown distribution. This leads to features of the expert judgement problem in the design context which are substantially different from those seen, for example, in risk assessment. In particular, the role of experts in problem structuring and in developing failure mitigation options is much more prominent, and there is a need to take into account the reliability potential for future mitigation measures downstream in the system life cycle. An overview is given of the stakeholders typically involved in large scale systems engineering design projects, and this is used to argue the need for methods that expose potential judgemental biases in order to generate analyses that can be said to provide rational consensus about uncertainties. Finally, a number of key points are developed with the aim of moving toward a framework that provides a holistic method for tracking reliability assessment through the design process.

AB - This paper reviews the role of expert judgement to support reliability assessments within the systems engineering design process. Generic design processes are described to give the context and a discussion is given about the nature of the reliability assessments required in the different systems engineering phases. It is argued that, as far as meeting reliability requirements is concerned, the whole design process is more akin to a statistical control process than to a straightforward statistical problem of assessing an unknown distribution. This leads to features of the expert judgement problem in the design context which are substantially different from those seen, for example, in risk assessment. In particular, the role of experts in problem structuring and in developing failure mitigation options is much more prominent, and there is a need to take into account the reliability potential for future mitigation measures downstream in the system life cycle. An overview is given of the stakeholders typically involved in large scale systems engineering design projects, and this is used to argue the need for methods that expose potential judgemental biases in order to generate analyses that can be said to provide rational consensus about uncertainties. Finally, a number of key points are developed with the aim of moving toward a framework that provides a holistic method for tracking reliability assessment through the design process.

KW - expert judgement

KW - elicitation

KW - reliability

UR - http://imstat.org/en/index.html

UR - http://dx.doi.org/10.1214/088342306000000510

U2 - 10.1214/088342306000000510

DO - 10.1214/088342306000000510

M3 - Article

VL - 21

SP - 428

EP - 462

JO - Statistical Science

T2 - Statistical Science

JF - Statistical Science

SN - 0883-4237

IS - 4

ER -