Does the Delphi process lead to increased accuracy in group-based judgemental forecasts or does it simply induce consensus amongst judgemental forecasters?

F Bolger, A Stranieri, George Wright, J Yearwood

    Research output: Contribution to journalArticle

    33 Citations (Scopus)

    Abstract

    We investigate the relative impact of internal Delphi process factors – including panelists' degree of confidence, expertise, majority/minority positioning – and an external factor, richness of feedback – on opinion change and subsequent accuracy of judgmental forecasts. We found that panelists who had low confidence in their judgmental forecast and/or who were in a minority were more likely to change their opinion than those who were more confident and/or in a majority. The addition of rationales, or reasons, to the numeric feedback had little impact upon panelists' final forecasts, despite the quality of panelists' rationales being significantly positively correlated with accurate forecasts and thus of potential use to aid forecast improvement over Delphi rounds. Rather, the effect of rationales was similar to that of confidence: to pull panelists towards the majority opinion regardless of its correctness. We conclude that majority opinion is the strongest influence on panelists' opinion change in both the ‘standard’ Delphi, and Delphi-with-reasons. We make some suggestions for improved variants of the Delphi-with-reasons technique that should help reduce majority influence and thereby permit reasoned arguments to exert their proper pull on opinion change, resulting in forecast accuracy improvements over Delphi rounds.
    LanguageEnglish
    Pages1671-1680
    Number of pages10
    JournalTechnological Forecasting and Social Change
    Volume78
    Issue number9
    DOIs
    Publication statusPublished - Nov 2011

    Fingerprint

    Feedback
    Delphi
    Confidence
    Rationale
    Minorities
    Pull

    Keywords

    • delphi process
    • increased accuracy
    • group-based
    • judgmental forecasts
    • judgmental forecasters
    • consensus
    • opinion change
    • judgmental forecasting
    • accuracy improvement

    Cite this

    @article{31a98376cd6d434cb46006af32fa0356,
    title = "Does the Delphi process lead to increased accuracy in group-based judgemental forecasts or does it simply induce consensus amongst judgemental forecasters?",
    abstract = "We investigate the relative impact of internal Delphi process factors – including panelists' degree of confidence, expertise, majority/minority positioning – and an external factor, richness of feedback – on opinion change and subsequent accuracy of judgmental forecasts. We found that panelists who had low confidence in their judgmental forecast and/or who were in a minority were more likely to change their opinion than those who were more confident and/or in a majority. The addition of rationales, or reasons, to the numeric feedback had little impact upon panelists' final forecasts, despite the quality of panelists' rationales being significantly positively correlated with accurate forecasts and thus of potential use to aid forecast improvement over Delphi rounds. Rather, the effect of rationales was similar to that of confidence: to pull panelists towards the majority opinion regardless of its correctness. We conclude that majority opinion is the strongest influence on panelists' opinion change in both the ‘standard’ Delphi, and Delphi-with-reasons. We make some suggestions for improved variants of the Delphi-with-reasons technique that should help reduce majority influence and thereby permit reasoned arguments to exert their proper pull on opinion change, resulting in forecast accuracy improvements over Delphi rounds.",
    keywords = "delphi process, increased accuracy, group-based, judgmental forecasts, judgmental forecasters, consensus, opinion change, judgmental forecasting, accuracy improvement",
    author = "F Bolger and A Stranieri and George Wright and J Yearwood",
    year = "2011",
    month = "11",
    doi = "10.1016/j.techfore.2011.06.002",
    language = "English",
    volume = "78",
    pages = "1671--1680",
    journal = "Technological Forecasting and Social Change",
    issn = "0040-1625",
    number = "9",

    }

    TY - JOUR

    T1 - Does the Delphi process lead to increased accuracy in group-based judgemental forecasts or does it simply induce consensus amongst judgemental forecasters?

    AU - Bolger, F

    AU - Stranieri, A

    AU - Wright, George

    AU - Yearwood, J

    PY - 2011/11

    Y1 - 2011/11

    N2 - We investigate the relative impact of internal Delphi process factors – including panelists' degree of confidence, expertise, majority/minority positioning – and an external factor, richness of feedback – on opinion change and subsequent accuracy of judgmental forecasts. We found that panelists who had low confidence in their judgmental forecast and/or who were in a minority were more likely to change their opinion than those who were more confident and/or in a majority. The addition of rationales, or reasons, to the numeric feedback had little impact upon panelists' final forecasts, despite the quality of panelists' rationales being significantly positively correlated with accurate forecasts and thus of potential use to aid forecast improvement over Delphi rounds. Rather, the effect of rationales was similar to that of confidence: to pull panelists towards the majority opinion regardless of its correctness. We conclude that majority opinion is the strongest influence on panelists' opinion change in both the ‘standard’ Delphi, and Delphi-with-reasons. We make some suggestions for improved variants of the Delphi-with-reasons technique that should help reduce majority influence and thereby permit reasoned arguments to exert their proper pull on opinion change, resulting in forecast accuracy improvements over Delphi rounds.

    AB - We investigate the relative impact of internal Delphi process factors – including panelists' degree of confidence, expertise, majority/minority positioning – and an external factor, richness of feedback – on opinion change and subsequent accuracy of judgmental forecasts. We found that panelists who had low confidence in their judgmental forecast and/or who were in a minority were more likely to change their opinion than those who were more confident and/or in a majority. The addition of rationales, or reasons, to the numeric feedback had little impact upon panelists' final forecasts, despite the quality of panelists' rationales being significantly positively correlated with accurate forecasts and thus of potential use to aid forecast improvement over Delphi rounds. Rather, the effect of rationales was similar to that of confidence: to pull panelists towards the majority opinion regardless of its correctness. We conclude that majority opinion is the strongest influence on panelists' opinion change in both the ‘standard’ Delphi, and Delphi-with-reasons. We make some suggestions for improved variants of the Delphi-with-reasons technique that should help reduce majority influence and thereby permit reasoned arguments to exert their proper pull on opinion change, resulting in forecast accuracy improvements over Delphi rounds.

    KW - delphi process

    KW - increased accuracy

    KW - group-based

    KW - judgmental forecasts

    KW - judgmental forecasters

    KW - consensus

    KW - opinion change

    KW - judgmental forecasting

    KW - accuracy improvement

    U2 - 10.1016/j.techfore.2011.06.002

    DO - 10.1016/j.techfore.2011.06.002

    M3 - Article

    VL - 78

    SP - 1671

    EP - 1680

    JO - Technological Forecasting and Social Change

    T2 - Technological Forecasting and Social Change

    JF - Technological Forecasting and Social Change

    SN - 0040-1625

    IS - 9

    ER -