Home > Research > Publications & Outputs > Learning-Rate-Free Stochastic Optimization over...

Electronic data

  • riemannian_dog_icml

    Accepted author manuscript, 8.24 MB, PDF document

    Available under license: CC BY: Creative Commons Attribution 4.0 International License

View graph of relations

Learning-Rate-Free Stochastic Optimization over Riemannian Manifolds

Research output: Contribution to Journal/MagazineConference articlepeer-review

Forthcoming
<mark>Journal publication date</mark>1/05/2024
<mark>Journal</mark>Proceedings of Machine Learning Research
Publication StatusAccepted/In press
<mark>Original language</mark>English

Abstract

In recent years, interest in gradient-based optimization over Riemannian manifolds has surged. However, a significant challenge lies in the reliance on hyperparameters, especially the learning rate, which requires meticulous tuning by practitioners to ensure convergence at a suitable rate. In this work, we introduce innovative learning- rate-free algorithms for stochastic optimization over Riemannian manifolds, eliminating the need for hand-tuning and providing a more robust and user-friendly approach. We establish high probability convergence guarantees that are optimal, up to logarithmic factors, compared to the best-known optimally tuned rate in the deterministic setting. Our approach is validated through numerical experiments, demonstrating competitive performance against learning-rate-dependent algorithms.

Bibliographic note

In: Proceedings of the 41st International Conference on Machine Learning (ICML), Vienna, Austria.