Home > Research > Publications & Outputs > Stein Variational Gaussian Processes

Electronic data

  • 2009.12141v1

    Final published version, 1.79 MB, PDF document

    Available under license: CC BY: Creative Commons Attribution 4.0 International License

Keywords

View graph of relations

Stein Variational Gaussian Processes

Research output: Contribution to Journal/MagazineJournal articlepeer-review

Published

Standard

Stein Variational Gaussian Processes. / Pinder, Thomas; Nemeth, Christopher; Leslie, David.
In: arXiv, 25.09.2020.

Research output: Contribution to Journal/MagazineJournal articlepeer-review

Harvard

APA

Vancouver

Author

Bibtex

@article{6165a03a0cdc4a00912609f4e10bd027,
title = "Stein Variational Gaussian Processes",
abstract = " We show how to use Stein variational gradient descent (SVGD) to carry out inference in Gaussian process (GP) models with non-Gaussian likelihoods and large data volumes. Markov chain Monte Carlo (MCMC) is extremely computationally intensive for these situations, but the parametric assumptions required for efficient variational inference (VI) result in incorrect inference when they encounter the multi-modal posterior distributions that are common for such models. SVGD provides a non-parametric alternative to variational inference which is substantially faster than MCMC but unhindered by parametric assumptions. We prove that for GP models with Lipschitz gradients the SVGD algorithm monotonically decreases the Kullback-Leibler divergence from the sampling distribution to the true posterior. Our method is demonstrated on benchmark problems in both regression and classification, and a real air quality example with 11440 spatiotemporal observations, showing substantial performance improvements over MCMC and VI. ",
keywords = "stat.ML, cs.LG",
author = "Thomas Pinder and Christopher Nemeth and David Leslie",
note = "25 pages, 5 figures",
year = "2020",
month = sep,
day = "25",
language = "English",
journal = "arXiv",

}

RIS

TY - JOUR

T1 - Stein Variational Gaussian Processes

AU - Pinder, Thomas

AU - Nemeth, Christopher

AU - Leslie, David

N1 - 25 pages, 5 figures

PY - 2020/9/25

Y1 - 2020/9/25

N2 - We show how to use Stein variational gradient descent (SVGD) to carry out inference in Gaussian process (GP) models with non-Gaussian likelihoods and large data volumes. Markov chain Monte Carlo (MCMC) is extremely computationally intensive for these situations, but the parametric assumptions required for efficient variational inference (VI) result in incorrect inference when they encounter the multi-modal posterior distributions that are common for such models. SVGD provides a non-parametric alternative to variational inference which is substantially faster than MCMC but unhindered by parametric assumptions. We prove that for GP models with Lipschitz gradients the SVGD algorithm monotonically decreases the Kullback-Leibler divergence from the sampling distribution to the true posterior. Our method is demonstrated on benchmark problems in both regression and classification, and a real air quality example with 11440 spatiotemporal observations, showing substantial performance improvements over MCMC and VI.

AB - We show how to use Stein variational gradient descent (SVGD) to carry out inference in Gaussian process (GP) models with non-Gaussian likelihoods and large data volumes. Markov chain Monte Carlo (MCMC) is extremely computationally intensive for these situations, but the parametric assumptions required for efficient variational inference (VI) result in incorrect inference when they encounter the multi-modal posterior distributions that are common for such models. SVGD provides a non-parametric alternative to variational inference which is substantially faster than MCMC but unhindered by parametric assumptions. We prove that for GP models with Lipschitz gradients the SVGD algorithm monotonically decreases the Kullback-Leibler divergence from the sampling distribution to the true posterior. Our method is demonstrated on benchmark problems in both regression and classification, and a real air quality example with 11440 spatiotemporal observations, showing substantial performance improvements over MCMC and VI.

KW - stat.ML

KW - cs.LG

M3 - Journal article

JO - arXiv

JF - arXiv

ER -