Measurement issues in the evaluation of projects in a project portfolio

TY - JOUR

T1 - Measurement issues in the evaluation of projects in a project portfolio

AU - Morton, Alec

PY - 2015/9/16

Y1 - 2015/9/16

N2 - A common problem in multicriteria decision analysis is how to value projects in a project portfolio. Recently, there has been some attention given to a practice whereby analysts define a value function on some criterion by setting 0 as the value of the worst project. In particular, Clemen and Smith have argued this practice is not sound as it gives different results from the case where projects are "priced out", and makes a strong implicit assumption about the value of not doing a project. Moreover, it can be shown easily that the optimal choice set is not invariant with respect to rescalings such as those induced by the addition of a new item to the choice set, hence rank reversal can occur when this procedure is used. We provide a measurement theoretic account of the phenomenon and show that whether the admissible transformations of the underlying scale are similarity or affine (or other) transformations depends on the precise details of the value model used. We also provide an axiomatic impossibility result which illustrates an incompatibility between an idea that inaction has a value of zero and the admissibility of constant scale translations. We use our analysis to comment on the view of Clemen and Smith that the baseline problem is best dealt with by assigning project-specific scores for not doing particular projects.

AB - A common problem in multicriteria decision analysis is how to value projects in a project portfolio. Recently, there has been some attention given to a practice whereby analysts define a value function on some criterion by setting 0 as the value of the worst project. In particular, Clemen and Smith have argued this practice is not sound as it gives different results from the case where projects are "priced out", and makes a strong implicit assumption about the value of not doing a project. Moreover, it can be shown easily that the optimal choice set is not invariant with respect to rescalings such as those induced by the addition of a new item to the choice set, hence rank reversal can occur when this procedure is used. We provide a measurement theoretic account of the phenomenon and show that whether the admissible transformations of the underlying scale are similarity or affine (or other) transformations depends on the precise details of the value model used. We also provide an axiomatic impossibility result which illustrates an incompatibility between an idea that inaction has a value of zero and the admissibility of constant scale translations. We use our analysis to comment on the view of Clemen and Smith that the baseline problem is best dealt with by assigning project-specific scores for not doing particular projects.

KW - multicriteria decision analysis

KW - decision analysis

KW - portfolio decision analysis

UR - http://www.sciencedirect.com/science/article/pii/S0377221715002593

U2 - 10.1016/j.ejor.2015.03.037

DO - 10.1016/j.ejor.2015.03.037

M3 - Article

VL - 245

SP - 789

EP - 796

JO - European Journal of Operational Research

T2 - European Journal of Operational Research

JF - European Journal of Operational Research

SN - 0377-2217

IS - 3

ER -