A probabilistic metric for the validation of computational models

Ksenija Dvurecenska, Steve Graham, Edoardo Patelli, Eann A. Patterson

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

A new validation metric is proposed that combines the use of a threshold based on the uncertainty in the measurement data with a normalised relative error, and that is robust in the presence of large variations in the data. The outcome from the metric is the probability that a model's predictions are representative of the real world based on the specific conditions and confidence level pertaining to the experiment from which the measurements were acquired. Relative error metrics are traditionally designed for use with series of data values but orthogonal decomposition has been employed to reduce the dimensionality of data matrices to feature vectors so that the metric can be applied to fields of data. Three previously published case studies are employed to demonstrate the efficacy of this quantitative approach to the validation process in the discipline of structural analysis, for which historical data was available; however, the concept could be applied to a wide range of disciplines and sectors where modelling and simulation plays a pivotal role.
LanguageEnglish
Number of pages14
JournalRoyal Society Open Science
Volume5
Issue number11
DOIs
Publication statusPublished - 21 Nov 2018

Fingerprint

Computational Model
Metric
Structural analysis
Relative Error
Decomposition
Orthogonal Decomposition
Historical Data
Confidence Level
Structural Analysis
Feature Vector
Experiments
Prediction Model
Dimensionality
Modeling and Simulation
Efficacy
Sector
Uncertainty
Series
Range of data
Demonstrate

Keywords

  • relative error
  • model validation
  • computational modelling
  • orthogonal decomposition

Cite this

Dvurecenska, Ksenija ; Graham, Steve ; Patelli, Edoardo ; Patterson, Eann A. / A probabilistic metric for the validation of computational models. In: Royal Society Open Science. 2018 ; Vol. 5, No. 11.
@article{4d4396f424ef4bd1944b7289da41306f,
title = "A probabilistic metric for the validation of computational models",
abstract = "A new validation metric is proposed that combines the use of a threshold based on the uncertainty in the measurement data with a normalised relative error, and that is robust in the presence of large variations in the data. The outcome from the metric is the probability that a model's predictions are representative of the real world based on the specific conditions and confidence level pertaining to the experiment from which the measurements were acquired. Relative error metrics are traditionally designed for use with series of data values but orthogonal decomposition has been employed to reduce the dimensionality of data matrices to feature vectors so that the metric can be applied to fields of data. Three previously published case studies are employed to demonstrate the efficacy of this quantitative approach to the validation process in the discipline of structural analysis, for which historical data was available; however, the concept could be applied to a wide range of disciplines and sectors where modelling and simulation plays a pivotal role.",
keywords = "relative error, model validation, computational modelling, orthogonal decomposition",
author = "Ksenija Dvurecenska and Steve Graham and Edoardo Patelli and Patterson, {Eann A.}",
year = "2018",
month = "11",
day = "21",
doi = "10.1098/rsos.180687",
language = "English",
volume = "5",
journal = "Royal Society Open Science",
issn = "2054-5703",
number = "11",

}

A probabilistic metric for the validation of computational models. / Dvurecenska, Ksenija; Graham, Steve; Patelli, Edoardo; Patterson, Eann A.

In: Royal Society Open Science, Vol. 5, No. 11, 21.11.2018.

Research output: Contribution to journalArticle

TY - JOUR

T1 - A probabilistic metric for the validation of computational models

AU - Dvurecenska, Ksenija

AU - Graham, Steve

AU - Patelli, Edoardo

AU - Patterson, Eann A.

PY - 2018/11/21

Y1 - 2018/11/21

N2 - A new validation metric is proposed that combines the use of a threshold based on the uncertainty in the measurement data with a normalised relative error, and that is robust in the presence of large variations in the data. The outcome from the metric is the probability that a model's predictions are representative of the real world based on the specific conditions and confidence level pertaining to the experiment from which the measurements were acquired. Relative error metrics are traditionally designed for use with series of data values but orthogonal decomposition has been employed to reduce the dimensionality of data matrices to feature vectors so that the metric can be applied to fields of data. Three previously published case studies are employed to demonstrate the efficacy of this quantitative approach to the validation process in the discipline of structural analysis, for which historical data was available; however, the concept could be applied to a wide range of disciplines and sectors where modelling and simulation plays a pivotal role.

AB - A new validation metric is proposed that combines the use of a threshold based on the uncertainty in the measurement data with a normalised relative error, and that is robust in the presence of large variations in the data. The outcome from the metric is the probability that a model's predictions are representative of the real world based on the specific conditions and confidence level pertaining to the experiment from which the measurements were acquired. Relative error metrics are traditionally designed for use with series of data values but orthogonal decomposition has been employed to reduce the dimensionality of data matrices to feature vectors so that the metric can be applied to fields of data. Three previously published case studies are employed to demonstrate the efficacy of this quantitative approach to the validation process in the discipline of structural analysis, for which historical data was available; however, the concept could be applied to a wide range of disciplines and sectors where modelling and simulation plays a pivotal role.

KW - relative error

KW - model validation

KW - computational modelling

KW - orthogonal decomposition

U2 - 10.1098/rsos.180687

DO - 10.1098/rsos.180687

M3 - Article

VL - 5

JO - Royal Society Open Science

T2 - Royal Society Open Science

JF - Royal Society Open Science

SN - 2054-5703

IS - 11

ER -