Exploiting Hessian matrix and trust-region algorithm in hyperparameters estimation of Gaussian process

Y. Zhang, W.E. Leithead

Research output: Contribution to journalArticle

35 Citations (Scopus)

Abstract

Gaussian process (GP) regression is a Bayesian non-parametric regression model, showing good performance in various applications. However, it is quite rare to see research results on log-likelihood maximization algorithms. Instead of the commonly used conjugate gradient method, the Hessian matrix is first derived/simplified in this paper and the trust-region optimization method is then presented to estimate GP hyperparameters. Numerical experiments verify the theoretical analysis, showing the advantages of using Hessian matrix and trust-region algorithms. In the GP context, the trust-region optimization method is a robust alternative to conjugate gradient method, also in view of future researches on approximate and/or parallel GP-implementation.
LanguageEnglish
Pages1264-1281
Number of pages17
JournalApplied Mathematics and Computation
Volume171
Issue number2
DOIs
Publication statusPublished - 15 Dec 2005

Fingerprint

Trust Region Algorithm
Hyperparameters
Hessian matrix
Gaussian Process
Conjugate gradient method
Trust Region Method
Conjugate Gradient Method
Optimization Methods
Bayesian Nonparametrics
Nonparametric Model
Nonparametric Regression
Likelihood
Regression Model
Theoretical Analysis
Regression
Numerical Experiment
Verify
Alternatives
Experiments
Estimate

Keywords

  • Gaussian process
  • log likelihood maximization
  • conjugate gradient
  • trust region
  • Hessian matrix

Cite this

@article{a93cb6bb0abd44b6a010d05af3207c7f,
title = "Exploiting Hessian matrix and trust-region algorithm in hyperparameters estimation of Gaussian process",
abstract = "Gaussian process (GP) regression is a Bayesian non-parametric regression model, showing good performance in various applications. However, it is quite rare to see research results on log-likelihood maximization algorithms. Instead of the commonly used conjugate gradient method, the Hessian matrix is first derived/simplified in this paper and the trust-region optimization method is then presented to estimate GP hyperparameters. Numerical experiments verify the theoretical analysis, showing the advantages of using Hessian matrix and trust-region algorithms. In the GP context, the trust-region optimization method is a robust alternative to conjugate gradient method, also in view of future researches on approximate and/or parallel GP-implementation.",
keywords = "Gaussian process, log likelihood maximization, conjugate gradient, trust region, Hessian matrix",
author = "Y. Zhang and W.E. Leithead",
year = "2005",
month = "12",
day = "15",
doi = "10.1016/j.amc.2005.01.113",
language = "English",
volume = "171",
pages = "1264--1281",
journal = "Applied Mathematics and Computation",
issn = "0096-3003",
number = "2",

}

Exploiting Hessian matrix and trust-region algorithm in hyperparameters estimation of Gaussian process. / Zhang, Y.; Leithead, W.E.

In: Applied Mathematics and Computation, Vol. 171, No. 2, 15.12.2005, p. 1264-1281.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Exploiting Hessian matrix and trust-region algorithm in hyperparameters estimation of Gaussian process

AU - Zhang, Y.

AU - Leithead, W.E.

PY - 2005/12/15

Y1 - 2005/12/15

N2 - Gaussian process (GP) regression is a Bayesian non-parametric regression model, showing good performance in various applications. However, it is quite rare to see research results on log-likelihood maximization algorithms. Instead of the commonly used conjugate gradient method, the Hessian matrix is first derived/simplified in this paper and the trust-region optimization method is then presented to estimate GP hyperparameters. Numerical experiments verify the theoretical analysis, showing the advantages of using Hessian matrix and trust-region algorithms. In the GP context, the trust-region optimization method is a robust alternative to conjugate gradient method, also in view of future researches on approximate and/or parallel GP-implementation.

AB - Gaussian process (GP) regression is a Bayesian non-parametric regression model, showing good performance in various applications. However, it is quite rare to see research results on log-likelihood maximization algorithms. Instead of the commonly used conjugate gradient method, the Hessian matrix is first derived/simplified in this paper and the trust-region optimization method is then presented to estimate GP hyperparameters. Numerical experiments verify the theoretical analysis, showing the advantages of using Hessian matrix and trust-region algorithms. In the GP context, the trust-region optimization method is a robust alternative to conjugate gradient method, also in view of future researches on approximate and/or parallel GP-implementation.

KW - Gaussian process

KW - log likelihood maximization

KW - conjugate gradient

KW - trust region

KW - Hessian matrix

UR - http://dx.doi.org/10.1016/j.amc.2005.01.113

U2 - 10.1016/j.amc.2005.01.113

DO - 10.1016/j.amc.2005.01.113

M3 - Article

VL - 171

SP - 1264

EP - 1281

JO - Applied Mathematics and Computation

T2 - Applied Mathematics and Computation

JF - Applied Mathematics and Computation

SN - 0096-3003

IS - 2

ER -