Are objectives hierarchy related biases observed in practice? A meta-analysis of environmental and energy applications of Multi-Criteria Decision Analysis

Mika Martunnen, Valerie Belton, Judit Leinert

Research output: Contribution to journalArticle

15 Citations (Scopus)

Abstract

Procedural and behavioural biases have received little attention in recent Multi-Criteria Decision Analysis (MCDA) research. Our literature review shows that most research on biases was done 15–30 years ago. This study focuses on biases that are introduced at an early stage of MCDA when building objectives hierarchies and their effect on the weights. The main objective is to investigate whether prior findings regarding such biases, which were mostly based on laboratory experiments, can be found in real-world applications. We conducted a meta-analysis of the objectives hierarchies and weight elicitation procedures in 61 environmental and energy MCDA cases. Relationships between the structural characteristics of the objectives hierarchy and assigned objectives’ weights were analysed with statistical tests. Our main research questions were: (i) How does hierarchy size and structure affect the objectives’ weights? (ii) How are weights distributed across economic, social and environmental objectives? (iii) Is there support for the equalising bias? Our findings are mostly aligned with earlier research and suggest that the hierarchy structure and content can substantially influence weight distributions. For example, hierarchical weighting seems to be sensitive to the asymmetry bias, which can occur when a hierarchy has branches that differ in the number of sub-objectives. We found no evidence for the equalising bias. We highlight issues deserving more attention when developing objectives hierarchies and eliciting weights. The research demonstrates the potential to use meta-analysis, which has not previously been used in this way in the MCDA field, to learn from a collection of applications.
LanguageEnglish
Pages178-194
Number of pages17
JournalEuropean Journal of Operational Research
Volume265
Issue number1
Early online date7 Mar 2017
DOIs
Publication statusPublished - 16 Feb 2018

Fingerprint

Multi-criteria Decision Analysis
Decision theory
Energy
Statistical tests
Weight Distribution
Elicitation
Literature Review
Statistical test
Hierarchy
Multi-criteria decision analysis
Meta-analysis
Real-world Applications
Asymmetry
Weighting
Economics
Branch
Experiments

Keywords

  • behavioural OR
  • decision analysis
  • decision processes
  • multiple criteria analysis
  • OR in environment

Cite this

@article{561e5de739e7450dbc1bc345a0586ee5,
title = "Are objectives hierarchy related biases observed in practice? A meta-analysis of environmental and energy applications of Multi-Criteria Decision Analysis",
abstract = "Procedural and behavioural biases have received little attention in recent Multi-Criteria Decision Analysis (MCDA) research. Our literature review shows that most research on biases was done 15–30 years ago. This study focuses on biases that are introduced at an early stage of MCDA when building objectives hierarchies and their effect on the weights. The main objective is to investigate whether prior findings regarding such biases, which were mostly based on laboratory experiments, can be found in real-world applications. We conducted a meta-analysis of the objectives hierarchies and weight elicitation procedures in 61 environmental and energy MCDA cases. Relationships between the structural characteristics of the objectives hierarchy and assigned objectives’ weights were analysed with statistical tests. Our main research questions were: (i) How does hierarchy size and structure affect the objectives’ weights? (ii) How are weights distributed across economic, social and environmental objectives? (iii) Is there support for the equalising bias? Our findings are mostly aligned with earlier research and suggest that the hierarchy structure and content can substantially influence weight distributions. For example, hierarchical weighting seems to be sensitive to the asymmetry bias, which can occur when a hierarchy has branches that differ in the number of sub-objectives. We found no evidence for the equalising bias. We highlight issues deserving more attention when developing objectives hierarchies and eliciting weights. The research demonstrates the potential to use meta-analysis, which has not previously been used in this way in the MCDA field, to learn from a collection of applications.",
keywords = "behavioural OR, decision analysis, decision processes, multiple criteria analysis, OR in environment",
author = "Mika Martunnen and Valerie Belton and Judit Leinert",
year = "2018",
month = "2",
day = "16",
doi = "10.1016/j.ejor.2017.02.038",
language = "English",
volume = "265",
pages = "178--194",
journal = "European Journal of Operational Research",
issn = "0377-2217",
number = "1",

}

TY - JOUR

T1 - Are objectives hierarchy related biases observed in practice? A meta-analysis of environmental and energy applications of Multi-Criteria Decision Analysis

AU - Martunnen, Mika

AU - Belton, Valerie

AU - Leinert, Judit

PY - 2018/2/16

Y1 - 2018/2/16

N2 - Procedural and behavioural biases have received little attention in recent Multi-Criteria Decision Analysis (MCDA) research. Our literature review shows that most research on biases was done 15–30 years ago. This study focuses on biases that are introduced at an early stage of MCDA when building objectives hierarchies and their effect on the weights. The main objective is to investigate whether prior findings regarding such biases, which were mostly based on laboratory experiments, can be found in real-world applications. We conducted a meta-analysis of the objectives hierarchies and weight elicitation procedures in 61 environmental and energy MCDA cases. Relationships between the structural characteristics of the objectives hierarchy and assigned objectives’ weights were analysed with statistical tests. Our main research questions were: (i) How does hierarchy size and structure affect the objectives’ weights? (ii) How are weights distributed across economic, social and environmental objectives? (iii) Is there support for the equalising bias? Our findings are mostly aligned with earlier research and suggest that the hierarchy structure and content can substantially influence weight distributions. For example, hierarchical weighting seems to be sensitive to the asymmetry bias, which can occur when a hierarchy has branches that differ in the number of sub-objectives. We found no evidence for the equalising bias. We highlight issues deserving more attention when developing objectives hierarchies and eliciting weights. The research demonstrates the potential to use meta-analysis, which has not previously been used in this way in the MCDA field, to learn from a collection of applications.

AB - Procedural and behavioural biases have received little attention in recent Multi-Criteria Decision Analysis (MCDA) research. Our literature review shows that most research on biases was done 15–30 years ago. This study focuses on biases that are introduced at an early stage of MCDA when building objectives hierarchies and their effect on the weights. The main objective is to investigate whether prior findings regarding such biases, which were mostly based on laboratory experiments, can be found in real-world applications. We conducted a meta-analysis of the objectives hierarchies and weight elicitation procedures in 61 environmental and energy MCDA cases. Relationships between the structural characteristics of the objectives hierarchy and assigned objectives’ weights were analysed with statistical tests. Our main research questions were: (i) How does hierarchy size and structure affect the objectives’ weights? (ii) How are weights distributed across economic, social and environmental objectives? (iii) Is there support for the equalising bias? Our findings are mostly aligned with earlier research and suggest that the hierarchy structure and content can substantially influence weight distributions. For example, hierarchical weighting seems to be sensitive to the asymmetry bias, which can occur when a hierarchy has branches that differ in the number of sub-objectives. We found no evidence for the equalising bias. We highlight issues deserving more attention when developing objectives hierarchies and eliciting weights. The research demonstrates the potential to use meta-analysis, which has not previously been used in this way in the MCDA field, to learn from a collection of applications.

KW - behavioural OR

KW - decision analysis

KW - decision processes

KW - multiple criteria analysis

KW - OR in environment

U2 - 10.1016/j.ejor.2017.02.038

DO - 10.1016/j.ejor.2017.02.038

M3 - Article

VL - 265

SP - 178

EP - 194

JO - European Journal of Operational Research

T2 - European Journal of Operational Research

JF - European Journal of Operational Research

SN - 0377-2217

IS - 1

ER -