A methodology for constructing subjective probability distributions with data

Our methodology is based on the premise that expertise does not reside in the stochastic characterisation of the unknown quantity of interest, but rather upon other features of the problem to which an expert can relate her experience. By mapping the quantity of interest to an expert’s experience we can use available empirical data about associated events to support the quantification of uncertainty. Our rationale contrasts with other approaches to elicit subjective probability which ask an expert to map, according to her belief, the outcome of an unknown quantity of interest to the outcome of a lottery for which the randomness is understood and quantifiable. Typically, such a mapping represents the indifference of an expert on making a bet between the quantity of interest and the outcome of the lottery. Instead, we propose to construct a prior distribution with empirical data that is consistent with the subjective judgement of an expert. We develop a general methodology, grounded in the theory of empirical Bayes inference. We motivate the need for such an approach and illustrate its application through industry examples. We articulate our general steps and show how these translate to selected practical contexts. We examine the benefits, as well as the limitations, of our proposed methodology to indicate when it might, or might not be, appropriate.