Log-det approximation based on uniformly distributed seeds and its application to Gaussian process regression

Y. Zhang, W.E. Leithead, D.J. Leith, L. Walshe

Research output: Contribution to journalArticle

10 Citations (Scopus)

Abstract

Maximum likelihood estimation (MLE) of hyperparameters in Gaussian process regression as well as other computational models usually and frequently requires the evaluation of the logarithm of the determinant of a positive-definite matrix (denoted by Chereafter). In general, the exact computation of log del C is of O(N-3) operations where N is the matrix dimension. The approximation of log del C could be developed with O(N-2) operations based on power-series expansion and randomized trace estimator. In this paper, the accuracy and effectiveness of using uniformly distributed seeds for log det C approximation are investigated. The research shows that uniform-seed based approximation is an equally good alternative to Gaussian-seed based approximation, having slightly better approximation accuracy and smaller variance. Gaussian process regression examples also substantiate the effectiveness of such a uniform-seed based log-det approximation scheme.
LanguageEnglish
Pages198-214
Number of pages17
JournalJournal of Computational and Applied Mathematics
Volume220
Issue number1-2
DOIs
Publication statusPublished - 15 Oct 2008

Fingerprint

Gaussian Process
Seed
Regression
Approximation
del operator
Maximum likelihood estimation
Exact Computation
Power Series Expansion
Hyperparameters
Positive definite matrix
Approximation Scheme
Maximum Likelihood Estimation
Logarithm
Computational Model
Determinant
Trace
Estimator
Alternatives
Evaluation

Keywords

  • Gaussian random seeds
  • uniformly distributed seeds
  • randomized trace estimator
  • log-det approximation
  • O(N2) operations

Cite this

@article{baa4ca1e204a44bd8654045c239d85b4,
title = "Log-det approximation based on uniformly distributed seeds and its application to Gaussian process regression",
abstract = "Maximum likelihood estimation (MLE) of hyperparameters in Gaussian process regression as well as other computational models usually and frequently requires the evaluation of the logarithm of the determinant of a positive-definite matrix (denoted by Chereafter). In general, the exact computation of log del C is of O(N-3) operations where N is the matrix dimension. The approximation of log del C could be developed with O(N-2) operations based on power-series expansion and randomized trace estimator. In this paper, the accuracy and effectiveness of using uniformly distributed seeds for log det C approximation are investigated. The research shows that uniform-seed based approximation is an equally good alternative to Gaussian-seed based approximation, having slightly better approximation accuracy and smaller variance. Gaussian process regression examples also substantiate the effectiveness of such a uniform-seed based log-det approximation scheme.",
keywords = "Gaussian random seeds, uniformly distributed seeds, randomized trace estimator, log-det approximation, O(N2) operations",
author = "Y. Zhang and W.E. Leithead and D.J. Leith and L. Walshe",
year = "2008",
month = "10",
day = "15",
doi = "10.1016/j.cam.2007.08.012",
language = "English",
volume = "220",
pages = "198--214",
journal = "Journal of Computational and Applied Mathematics",
issn = "0377-0427",
number = "1-2",

}

Log-det approximation based on uniformly distributed seeds and its application to Gaussian process regression. / Zhang, Y.; Leithead, W.E.; Leith, D.J.; Walshe, L.

In: Journal of Computational and Applied Mathematics, Vol. 220, No. 1-2, 15.10.2008, p. 198-214.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Log-det approximation based on uniformly distributed seeds and its application to Gaussian process regression

AU - Zhang, Y.

AU - Leithead, W.E.

AU - Leith, D.J.

AU - Walshe, L.

PY - 2008/10/15

Y1 - 2008/10/15

N2 - Maximum likelihood estimation (MLE) of hyperparameters in Gaussian process regression as well as other computational models usually and frequently requires the evaluation of the logarithm of the determinant of a positive-definite matrix (denoted by Chereafter). In general, the exact computation of log del C is of O(N-3) operations where N is the matrix dimension. The approximation of log del C could be developed with O(N-2) operations based on power-series expansion and randomized trace estimator. In this paper, the accuracy and effectiveness of using uniformly distributed seeds for log det C approximation are investigated. The research shows that uniform-seed based approximation is an equally good alternative to Gaussian-seed based approximation, having slightly better approximation accuracy and smaller variance. Gaussian process regression examples also substantiate the effectiveness of such a uniform-seed based log-det approximation scheme.

AB - Maximum likelihood estimation (MLE) of hyperparameters in Gaussian process regression as well as other computational models usually and frequently requires the evaluation of the logarithm of the determinant of a positive-definite matrix (denoted by Chereafter). In general, the exact computation of log del C is of O(N-3) operations where N is the matrix dimension. The approximation of log del C could be developed with O(N-2) operations based on power-series expansion and randomized trace estimator. In this paper, the accuracy and effectiveness of using uniformly distributed seeds for log det C approximation are investigated. The research shows that uniform-seed based approximation is an equally good alternative to Gaussian-seed based approximation, having slightly better approximation accuracy and smaller variance. Gaussian process regression examples also substantiate the effectiveness of such a uniform-seed based log-det approximation scheme.

KW - Gaussian random seeds

KW - uniformly distributed seeds

KW - randomized trace estimator

KW - log-det approximation

KW - O(N2) operations

U2 - 10.1016/j.cam.2007.08.012

DO - 10.1016/j.cam.2007.08.012

M3 - Article

VL - 220

SP - 198

EP - 214

JO - Journal of Computational and Applied Mathematics

T2 - Journal of Computational and Applied Mathematics

JF - Journal of Computational and Applied Mathematics

SN - 0377-0427

IS - 1-2

ER -