Eliciting multivariate uncertainty from experts: considerations and approaches along the expert judgement process

Christoph Werner, Anca Hanea, Oswaldo Morales-Napoles

Research output: Chapter in Book/Report/Conference proceedingChapter

Abstract

In decision and risk analysis problems, modelling uncertainty probabilistically provides key insights and information for decision makers. A common challenge is that uncertainties are typically not isolated but interlinked which introduces complex (and often unexpected) effects on the model output. Therefore, dependence needs to be taken into account and modelled appropriately if simplifying assumptions, such as independence, are not sensible. Similar to the case of univariate uncertainty, which is described elsewhere in this book, relevant historical data to quantify a (dependence) model are often lacking or too costly to obtain. This may be true even when data on a model’s univariate quantities, such as marginal probabilities, are available. Then, specifying dependence between the uncertain variables through expert judgement is the only sensible option. A structured and formal process to the elicitation is essential for ensuring methodological robustness. This chapter addresses the main elements of structured expert judgement processes for dependence elicitation. We introduce the processes’ common elements, typically used for eliciting univariate quantities, and present the differences that need to be considered at each of the process’ steps for multivariate uncertainty. Further, we review findings from the behavioural judgement and decision making literature on potential cognitive fallacies that can occur when assessing dependence as mitigating biases is a main objective of formal expert judgement processes. Given a practical focus, we reflect on case studies in addition to theoretical findings. Thus, this chapter serves as guidance for facilitators and analysts using expert judgement.
LanguageEnglish
Title of host publicationElicitation
Subtitle of host publicationThe Science and Art of Structuring Judgement
EditorsLuis Dias, Alec Morton, John Quigley
Place of PublicationNew York
Number of pages42
DOIs
Publication statusPublished - 19 Oct 2017

Publication series

NameInternational Series in Operations Research & Management Science
PublisherSpringer
Volume261
ISSN (Print)0884-8289

Fingerprint

Expert judgment
Uncertainty
Uncertainty modeling
Robustness
Judgment and decision making
Risk analysis
Guidance
Decision analysis
Decision maker
Facilitators
Analysts

Keywords

  • structured expert judgement
  • dependence modelling
  • dependence elicitation
  • uncertainty analysis

Cite this

Werner, C., Hanea, A., & Morales-Napoles, O. (2017). Eliciting multivariate uncertainty from experts: considerations and approaches along the expert judgement process. In L. Dias, A. Morton, & J. Quigley (Eds.), Elicitation: The Science and Art of Structuring Judgement (International Series in Operations Research & Management Science; Vol. 261). New York. https://doi.org/10.1007/978-3-319-65052-4
Werner, Christoph ; Hanea, Anca ; Morales-Napoles, Oswaldo. / Eliciting multivariate uncertainty from experts : considerations and approaches along the expert judgement process. Elicitation: The Science and Art of Structuring Judgement. editor / Luis Dias ; Alec Morton ; John Quigley. New York, 2017. (International Series in Operations Research & Management Science).
@inbook{e1211f6e5c6c40dc865d18b6eeaf2ba3,
title = "Eliciting multivariate uncertainty from experts: considerations and approaches along the expert judgement process",
abstract = "In decision and risk analysis problems, modelling uncertainty probabilistically provides key insights and information for decision makers. A common challenge is that uncertainties are typically not isolated but interlinked which introduces complex (and often unexpected) effects on the model output. Therefore, dependence needs to be taken into account and modelled appropriately if simplifying assumptions, such as independence, are not sensible. Similar to the case of univariate uncertainty, which is described elsewhere in this book, relevant historical data to quantify a (dependence) model are often lacking or too costly to obtain. This may be true even when data on a model’s univariate quantities, such as marginal probabilities, are available. Then, specifying dependence between the uncertain variables through expert judgement is the only sensible option. A structured and formal process to the elicitation is essential for ensuring methodological robustness. This chapter addresses the main elements of structured expert judgement processes for dependence elicitation. We introduce the processes’ common elements, typically used for eliciting univariate quantities, and present the differences that need to be considered at each of the process’ steps for multivariate uncertainty. Further, we review findings from the behavioural judgement and decision making literature on potential cognitive fallacies that can occur when assessing dependence as mitigating biases is a main objective of formal expert judgement processes. Given a practical focus, we reflect on case studies in addition to theoretical findings. Thus, this chapter serves as guidance for facilitators and analysts using expert judgement.",
keywords = "structured expert judgement, dependence modelling, dependence elicitation, uncertainty analysis",
author = "Christoph Werner and Anca Hanea and Oswaldo Morales-Napoles",
year = "2017",
month = "10",
day = "19",
doi = "10.1007/978-3-319-65052-4",
language = "English",
isbn = "978-3-319-65051-7",
series = "International Series in Operations Research & Management Science",
publisher = "Springer",
editor = "Luis Dias and Alec Morton and John Quigley",
booktitle = "Elicitation",

}

Werner, C, Hanea, A & Morales-Napoles, O 2017, Eliciting multivariate uncertainty from experts: considerations and approaches along the expert judgement process. in L Dias, A Morton & J Quigley (eds), Elicitation: The Science and Art of Structuring Judgement. International Series in Operations Research & Management Science, vol. 261, New York. https://doi.org/10.1007/978-3-319-65052-4

Eliciting multivariate uncertainty from experts : considerations and approaches along the expert judgement process. / Werner, Christoph; Hanea, Anca; Morales-Napoles, Oswaldo.

Elicitation: The Science and Art of Structuring Judgement. ed. / Luis Dias; Alec Morton; John Quigley. New York, 2017. (International Series in Operations Research & Management Science; Vol. 261).

Research output: Chapter in Book/Report/Conference proceedingChapter

TY - CHAP

T1 - Eliciting multivariate uncertainty from experts

T2 - considerations and approaches along the expert judgement process

AU - Werner, Christoph

AU - Hanea, Anca

AU - Morales-Napoles, Oswaldo

PY - 2017/10/19

Y1 - 2017/10/19

N2 - In decision and risk analysis problems, modelling uncertainty probabilistically provides key insights and information for decision makers. A common challenge is that uncertainties are typically not isolated but interlinked which introduces complex (and often unexpected) effects on the model output. Therefore, dependence needs to be taken into account and modelled appropriately if simplifying assumptions, such as independence, are not sensible. Similar to the case of univariate uncertainty, which is described elsewhere in this book, relevant historical data to quantify a (dependence) model are often lacking or too costly to obtain. This may be true even when data on a model’s univariate quantities, such as marginal probabilities, are available. Then, specifying dependence between the uncertain variables through expert judgement is the only sensible option. A structured and formal process to the elicitation is essential for ensuring methodological robustness. This chapter addresses the main elements of structured expert judgement processes for dependence elicitation. We introduce the processes’ common elements, typically used for eliciting univariate quantities, and present the differences that need to be considered at each of the process’ steps for multivariate uncertainty. Further, we review findings from the behavioural judgement and decision making literature on potential cognitive fallacies that can occur when assessing dependence as mitigating biases is a main objective of formal expert judgement processes. Given a practical focus, we reflect on case studies in addition to theoretical findings. Thus, this chapter serves as guidance for facilitators and analysts using expert judgement.

AB - In decision and risk analysis problems, modelling uncertainty probabilistically provides key insights and information for decision makers. A common challenge is that uncertainties are typically not isolated but interlinked which introduces complex (and often unexpected) effects on the model output. Therefore, dependence needs to be taken into account and modelled appropriately if simplifying assumptions, such as independence, are not sensible. Similar to the case of univariate uncertainty, which is described elsewhere in this book, relevant historical data to quantify a (dependence) model are often lacking or too costly to obtain. This may be true even when data on a model’s univariate quantities, such as marginal probabilities, are available. Then, specifying dependence between the uncertain variables through expert judgement is the only sensible option. A structured and formal process to the elicitation is essential for ensuring methodological robustness. This chapter addresses the main elements of structured expert judgement processes for dependence elicitation. We introduce the processes’ common elements, typically used for eliciting univariate quantities, and present the differences that need to be considered at each of the process’ steps for multivariate uncertainty. Further, we review findings from the behavioural judgement and decision making literature on potential cognitive fallacies that can occur when assessing dependence as mitigating biases is a main objective of formal expert judgement processes. Given a practical focus, we reflect on case studies in addition to theoretical findings. Thus, this chapter serves as guidance for facilitators and analysts using expert judgement.

KW - structured expert judgement

KW - dependence modelling

KW - dependence elicitation

KW - uncertainty analysis

UR - http://www.springer.com/gb/book/9783319650517

U2 - 10.1007/978-3-319-65052-4

DO - 10.1007/978-3-319-65052-4

M3 - Chapter

SN - 978-3-319-65051-7

T3 - International Series in Operations Research & Management Science

BT - Elicitation

A2 - Dias, Luis

A2 - Morton, Alec

A2 - Quigley, John

CY - New York

ER -

Werner C, Hanea A, Morales-Napoles O. Eliciting multivariate uncertainty from experts: considerations and approaches along the expert judgement process. In Dias L, Morton A, Quigley J, editors, Elicitation: The Science and Art of Structuring Judgement. New York. 2017. (International Series in Operations Research & Management Science). https://doi.org/10.1007/978-3-319-65052-4