Approximate implementation of logarithm of the matrix determinant in Gaussian process regression

Y. Zhang, W.E. Leithead

Research output: Contribution to journalArticle

Abstract

Maximum likelihood estimation of hyperparameters in Gaussian processes (GPs) as well as other spatial regression models usually requires the evaluation of the logarithm of the matrix determinant, in short, log det. When using matrix decomposition techniques, the exact implementation of log det is of O(N3) operations, where N is the matrix dimension. In this paper, a power-series expansion-based framework is presented for approximating the log det of general positive-definite matrices. Three novel compensation schemes are proposed to further improve the approximation accuracy and computational efficiency. The proposed log det approximation requires only 50N2 operations. The theoretical analysis is substantiated by a large number of numerical experiments, including tests on randomly generated positive-definite matrices, randomly generated covariance matrices, and sequences of covariance matrices generated online in two GP regression examples. The average approximation error is ∼9%.
Original languageEnglish
Pages (from-to)329-348
Number of pages19
JournalJournal of Statistical Computation and Simulation
Volume77
Issue number4
Publication statusPublished - 4 Jan 2007

Fingerprint

Positive definite matrix
Gaussian Process
Logarithm
Covariance matrix
Determinant
Regression
Power Series Expansion
Matrix Decomposition
Hyperparameters
Decomposition Techniques
Spatial Model
Approximation Error
Approximation
Maximum Likelihood Estimation
Computational Efficiency
Regression Model
Theoretical Analysis
Numerical Experiment
Evaluation
Maximum likelihood estimation

Keywords

  • Gaussian process
  • Logarithm of matrix determinant
  • Power-series expansion
  • Compensation

Cite this

@article{196bc52c1e7048349afefd1cf0923393,
title = "Approximate implementation of logarithm of the matrix determinant in Gaussian process regression",
abstract = "Maximum likelihood estimation of hyperparameters in Gaussian processes (GPs) as well as other spatial regression models usually requires the evaluation of the logarithm of the matrix determinant, in short, log det. When using matrix decomposition techniques, the exact implementation of log det is of O(N3) operations, where N is the matrix dimension. In this paper, a power-series expansion-based framework is presented for approximating the log det of general positive-definite matrices. Three novel compensation schemes are proposed to further improve the approximation accuracy and computational efficiency. The proposed log det approximation requires only 50N2 operations. The theoretical analysis is substantiated by a large number of numerical experiments, including tests on randomly generated positive-definite matrices, randomly generated covariance matrices, and sequences of covariance matrices generated online in two GP regression examples. The average approximation error is ∼9{\%}.",
keywords = "Gaussian process, Logarithm of matrix determinant, Power-series expansion, Compensation",
author = "Y. Zhang and W.E. Leithead",
year = "2007",
month = "1",
day = "4",
language = "English",
volume = "77",
pages = "329--348",
journal = "Journal of Statistical Computation and Simulation",
issn = "0094-9655",
number = "4",

}

Approximate implementation of logarithm of the matrix determinant in Gaussian process regression. / Zhang, Y.; Leithead, W.E.

In: Journal of Statistical Computation and Simulation, Vol. 77, No. 4, 04.01.2007, p. 329-348.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Approximate implementation of logarithm of the matrix determinant in Gaussian process regression

AU - Zhang, Y.

AU - Leithead, W.E.

PY - 2007/1/4

Y1 - 2007/1/4

N2 - Maximum likelihood estimation of hyperparameters in Gaussian processes (GPs) as well as other spatial regression models usually requires the evaluation of the logarithm of the matrix determinant, in short, log det. When using matrix decomposition techniques, the exact implementation of log det is of O(N3) operations, where N is the matrix dimension. In this paper, a power-series expansion-based framework is presented for approximating the log det of general positive-definite matrices. Three novel compensation schemes are proposed to further improve the approximation accuracy and computational efficiency. The proposed log det approximation requires only 50N2 operations. The theoretical analysis is substantiated by a large number of numerical experiments, including tests on randomly generated positive-definite matrices, randomly generated covariance matrices, and sequences of covariance matrices generated online in two GP regression examples. The average approximation error is ∼9%.

AB - Maximum likelihood estimation of hyperparameters in Gaussian processes (GPs) as well as other spatial regression models usually requires the evaluation of the logarithm of the matrix determinant, in short, log det. When using matrix decomposition techniques, the exact implementation of log det is of O(N3) operations, where N is the matrix dimension. In this paper, a power-series expansion-based framework is presented for approximating the log det of general positive-definite matrices. Three novel compensation schemes are proposed to further improve the approximation accuracy and computational efficiency. The proposed log det approximation requires only 50N2 operations. The theoretical analysis is substantiated by a large number of numerical experiments, including tests on randomly generated positive-definite matrices, randomly generated covariance matrices, and sequences of covariance matrices generated online in two GP regression examples. The average approximation error is ∼9%.

KW - Gaussian process

KW - Logarithm of matrix determinant

KW - Power-series expansion

KW - Compensation

M3 - Article

VL - 77

SP - 329

EP - 348

JO - Journal of Statistical Computation and Simulation

JF - Journal of Statistical Computation and Simulation

SN - 0094-9655

IS - 4

ER -