Shiqian Ma, 香港中文大学


Stochastic Quasi-Newton Methods for Nonconvex Stochastic Optimization


In this talk, we discuss stochastic quasi-Newton methods for nonconvex stochastic optimization. We assume that only stochastic information of the gradients of the objective function is available via a stochastic first-order oracle (SFO). We firstly propose a general framework for stochastic quasi-Newton methods solving such kind of problems. This type of methods extend the classic quasi-Newton method for deterministic optimization problems to a stochastic setting with stochastic information of the function being used. Secondly, we propose a general framework for a class of randomized stochastic quasi-Newton methods in which the number of iterations conducted by the algorithm is a random variable. The worst-case SFO-calls complexities of these methods are analyzed. Thirdly, we propose two specific algorithms that fall into this framework: stochastic damped-BFGS method and stochastic cyclic Barzilai-Borwein method. Finally, we report some preliminary numerical results that demonstrate the efficiency of the proposed algorithms.

« Back