The limits of forecasting methods in anticipating rare events

P Goodwin, George Wright

    Research output: Contribution to journalArticle

    90 Citations (Scopus)

    Abstract

    In this paper we review methods that aim to aid the anticipation of rare, high-impact, events. We evaluate these methods according to their ability to yield well-calibrated probabilities or point forecasts for such events. We first identify six factors that can lead to poor calibration and then examine how successful the methods are in mitigating these factors. We demonstrate that all the extant forecasting methods — including the use of expert judgment, statistical forecasting, Delphi and prediction markets — contain fundamental weaknesses. We contrast these methods with a non-forecasting method that is intended to aid planning for the future — scenario planning. We conclude that all the methods are problematic for aiding the anticipation of rare events and that the only remedies are to either (i) to provide protection for the organization against the occurrence of negatively-valenced events whilst allowing the organization to benefit from the occurrence of positively-valenced events, or (ii) to provide conditions to challenge one's own thinking — and hence improve anticipation. We outline how components of devil's advocacy and dialectical inquiry can be combined with Delphi and scenario planning to enhance anticipation of rare events.
    LanguageEnglish
    Pages355-368
    Number of pages14
    JournalTechnological Forecasting and Social Change
    Volume77
    Issue number3
    DOIs
    Publication statusPublished - Mar 2010

    Fingerprint

    Planning
    Calibration
    Organizations
    Aptitude
    Rare events
    Forecasting method
    Anticipation
    Factors
    Scenario planning
    Delphi

    Keywords

    • forecasting methods
    • rare events
    • expert judgment
    • statistical forecasting
    • delphi
    • scenario planning

    Cite this

    @article{0df781d56a8048359dd84e6b60b78064,
    title = "The limits of forecasting methods in anticipating rare events",
    abstract = "In this paper we review methods that aim to aid the anticipation of rare, high-impact, events. We evaluate these methods according to their ability to yield well-calibrated probabilities or point forecasts for such events. We first identify six factors that can lead to poor calibration and then examine how successful the methods are in mitigating these factors. We demonstrate that all the extant forecasting methods — including the use of expert judgment, statistical forecasting, Delphi and prediction markets — contain fundamental weaknesses. We contrast these methods with a non-forecasting method that is intended to aid planning for the future — scenario planning. We conclude that all the methods are problematic for aiding the anticipation of rare events and that the only remedies are to either (i) to provide protection for the organization against the occurrence of negatively-valenced events whilst allowing the organization to benefit from the occurrence of positively-valenced events, or (ii) to provide conditions to challenge one's own thinking — and hence improve anticipation. We outline how components of devil's advocacy and dialectical inquiry can be combined with Delphi and scenario planning to enhance anticipation of rare events.",
    keywords = "forecasting methods, rare events, expert judgment, statistical forecasting, delphi, scenario planning",
    author = "P Goodwin and George Wright",
    year = "2010",
    month = "3",
    doi = "10.1016/j.techfore.2009.10.008",
    language = "English",
    volume = "77",
    pages = "355--368",
    journal = "Technological Forecasting and Social Change",
    issn = "0040-1625",
    number = "3",

    }

    The limits of forecasting methods in anticipating rare events. / Goodwin, P; Wright, George.

    In: Technological Forecasting and Social Change, Vol. 77, No. 3, 03.2010, p. 355-368.

    Research output: Contribution to journalArticle

    TY - JOUR

    T1 - The limits of forecasting methods in anticipating rare events

    AU - Goodwin, P

    AU - Wright, George

    PY - 2010/3

    Y1 - 2010/3

    N2 - In this paper we review methods that aim to aid the anticipation of rare, high-impact, events. We evaluate these methods according to their ability to yield well-calibrated probabilities or point forecasts for such events. We first identify six factors that can lead to poor calibration and then examine how successful the methods are in mitigating these factors. We demonstrate that all the extant forecasting methods — including the use of expert judgment, statistical forecasting, Delphi and prediction markets — contain fundamental weaknesses. We contrast these methods with a non-forecasting method that is intended to aid planning for the future — scenario planning. We conclude that all the methods are problematic for aiding the anticipation of rare events and that the only remedies are to either (i) to provide protection for the organization against the occurrence of negatively-valenced events whilst allowing the organization to benefit from the occurrence of positively-valenced events, or (ii) to provide conditions to challenge one's own thinking — and hence improve anticipation. We outline how components of devil's advocacy and dialectical inquiry can be combined with Delphi and scenario planning to enhance anticipation of rare events.

    AB - In this paper we review methods that aim to aid the anticipation of rare, high-impact, events. We evaluate these methods according to their ability to yield well-calibrated probabilities or point forecasts for such events. We first identify six factors that can lead to poor calibration and then examine how successful the methods are in mitigating these factors. We demonstrate that all the extant forecasting methods — including the use of expert judgment, statistical forecasting, Delphi and prediction markets — contain fundamental weaknesses. We contrast these methods with a non-forecasting method that is intended to aid planning for the future — scenario planning. We conclude that all the methods are problematic for aiding the anticipation of rare events and that the only remedies are to either (i) to provide protection for the organization against the occurrence of negatively-valenced events whilst allowing the organization to benefit from the occurrence of positively-valenced events, or (ii) to provide conditions to challenge one's own thinking — and hence improve anticipation. We outline how components of devil's advocacy and dialectical inquiry can be combined with Delphi and scenario planning to enhance anticipation of rare events.

    KW - forecasting methods

    KW - rare events

    KW - expert judgment

    KW - statistical forecasting

    KW - delphi

    KW - scenario planning

    UR - http://dx.doi.org/10.1016/j.techfore.2009.10.008

    U2 - 10.1016/j.techfore.2009.10.008

    DO - 10.1016/j.techfore.2009.10.008

    M3 - Article

    VL - 77

    SP - 355

    EP - 368

    JO - Technological Forecasting and Social Change

    T2 - Technological Forecasting and Social Change

    JF - Technological Forecasting and Social Change

    SN - 0040-1625

    IS - 3

    ER -