Measurement issues in the evaluation of projects in a project portfolio

Research output: Contribution to journalArticle

8 Citations (Scopus)

Abstract

A common problem in multicriteria decision analysis is how to value projects in a project portfolio. Recently, there has been some attention given to a practice whereby analysts define a value function on some criterion by setting 0 as the value of the worst project. In particular, Clemen and Smith have argued this practice is not sound as it gives different results from the case where projects are "priced out", and makes a strong implicit assumption about the value of not doing a project. Moreover, it can be shown easily that the optimal choice set is not invariant with respect to rescalings such as those induced by the addition of a new item to the choice set, hence rank reversal can occur when this procedure is used. We provide a measurement theoretic account of the phenomenon and show that whether the admissible transformations of the underlying scale are similarity or affine (or other) transformations depends on the precise details of the value model used. We also provide an axiomatic impossibility result which illustrates an incompatibility between an idea that inaction has a value of zero and the admissibility of constant scale translations. We use our analysis to comment on the view of Clemen and Smith that the baseline problem is best dealt with by assigning project-specific scores for not doing particular projects.
LanguageEnglish
Pages789-796
Number of pages8
JournalEuropean Journal of Operational Research
Volume245
Issue number3
Early online date30 Mar 2015
DOIs
Publication statusPublished - 16 Sep 2015

Fingerprint

Decision theory
Multi-criteria Decision Analysis
Acoustic waves
Admissibility
Rescaling
Evaluation
Reversal
Value Function
Baseline
Invariant
Zero
Project portfolio
Model
Similarity
Sound
Choice sets

Keywords

  • multicriteria decision analysis
  • decision analysis
  • portfolio decision analysis

Cite this

@article{5f1bdf7a74204483ab6658fd33511fac,
title = "Measurement issues in the evaluation of projects in a project portfolio",
abstract = "A common problem in multicriteria decision analysis is how to value projects in a project portfolio. Recently, there has been some attention given to a practice whereby analysts define a value function on some criterion by setting 0 as the value of the worst project. In particular, Clemen and Smith have argued this practice is not sound as it gives different results from the case where projects are {"}priced out{"}, and makes a strong implicit assumption about the value of not doing a project. Moreover, it can be shown easily that the optimal choice set is not invariant with respect to rescalings such as those induced by the addition of a new item to the choice set, hence rank reversal can occur when this procedure is used. We provide a measurement theoretic account of the phenomenon and show that whether the admissible transformations of the underlying scale are similarity or affine (or other) transformations depends on the precise details of the value model used. We also provide an axiomatic impossibility result which illustrates an incompatibility between an idea that inaction has a value of zero and the admissibility of constant scale translations. We use our analysis to comment on the view of Clemen and Smith that the baseline problem is best dealt with by assigning project-specific scores for not doing particular projects.",
keywords = "multicriteria decision analysis, decision analysis, portfolio decision analysis",
author = "Alec Morton",
year = "2015",
month = "9",
day = "16",
doi = "10.1016/j.ejor.2015.03.037",
language = "English",
volume = "245",
pages = "789--796",
journal = "European Journal of Operational Research",
issn = "0377-2217",
number = "3",

}

Measurement issues in the evaluation of projects in a project portfolio. / Morton, Alec.

In: European Journal of Operational Research, Vol. 245, No. 3, 16.09.2015, p. 789-796.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Measurement issues in the evaluation of projects in a project portfolio

AU - Morton, Alec

PY - 2015/9/16

Y1 - 2015/9/16

N2 - A common problem in multicriteria decision analysis is how to value projects in a project portfolio. Recently, there has been some attention given to a practice whereby analysts define a value function on some criterion by setting 0 as the value of the worst project. In particular, Clemen and Smith have argued this practice is not sound as it gives different results from the case where projects are "priced out", and makes a strong implicit assumption about the value of not doing a project. Moreover, it can be shown easily that the optimal choice set is not invariant with respect to rescalings such as those induced by the addition of a new item to the choice set, hence rank reversal can occur when this procedure is used. We provide a measurement theoretic account of the phenomenon and show that whether the admissible transformations of the underlying scale are similarity or affine (or other) transformations depends on the precise details of the value model used. We also provide an axiomatic impossibility result which illustrates an incompatibility between an idea that inaction has a value of zero and the admissibility of constant scale translations. We use our analysis to comment on the view of Clemen and Smith that the baseline problem is best dealt with by assigning project-specific scores for not doing particular projects.

AB - A common problem in multicriteria decision analysis is how to value projects in a project portfolio. Recently, there has been some attention given to a practice whereby analysts define a value function on some criterion by setting 0 as the value of the worst project. In particular, Clemen and Smith have argued this practice is not sound as it gives different results from the case where projects are "priced out", and makes a strong implicit assumption about the value of not doing a project. Moreover, it can be shown easily that the optimal choice set is not invariant with respect to rescalings such as those induced by the addition of a new item to the choice set, hence rank reversal can occur when this procedure is used. We provide a measurement theoretic account of the phenomenon and show that whether the admissible transformations of the underlying scale are similarity or affine (or other) transformations depends on the precise details of the value model used. We also provide an axiomatic impossibility result which illustrates an incompatibility between an idea that inaction has a value of zero and the admissibility of constant scale translations. We use our analysis to comment on the view of Clemen and Smith that the baseline problem is best dealt with by assigning project-specific scores for not doing particular projects.

KW - multicriteria decision analysis

KW - decision analysis

KW - portfolio decision analysis

UR - http://www.sciencedirect.com/science/article/pii/S0377221715002593

U2 - 10.1016/j.ejor.2015.03.037

DO - 10.1016/j.ejor.2015.03.037

M3 - Article

VL - 245

SP - 789

EP - 796

JO - European Journal of Operational Research

T2 - European Journal of Operational Research

JF - European Journal of Operational Research

SN - 0377-2217

IS - 3

ER -