Hybrid deterministic-stochastic methods for data fitting

TitleHybrid deterministic-stochastic methods for data fitting
Publication TypeJournal Article
Year of Publication2012
AuthorsMichael P. Friedlander, Mark Schmidt
JournalSIAM Journal on Scientific Computing
Volume34
Number3
PageA1380-A1405
Month1
PublisherDepartment of Computer Science
KeywordsOptimization
Abstract

Many structured data-fitting applications require the solution of an optimization problem involving a sum over a potentially large number of measurements. Incremental gradient algorithms (both deterministic and randomized) offer inexpensive iterations by sampling only subsets of the terms in the sum. These methods can make great progress initially, but often slow as they approach a solution. In contrast, full gradient methods achieve steady convergence at the expense of evaluating the full objective and gradient on each iteration. We explore hybrid methods that exhibit the benefits of both approaches. Rate of convergence analysis and numerical experiments illustrate the potential for the approach.

URLhttps://www.math.ucdavis.edu/ mpf/2011-hybrid-for-data-fitting.html
DOI10.1137/110830629
Citation KeyFriedlander11TRhdm