Athletic Boutique Boutique Athletic Shorts Shorts Athletic Shorts Nike Nike Nike Boutique p1fCqXP Athletic Boutique Boutique Athletic Shorts Shorts Athletic Shorts Nike Nike Nike Boutique p1fCqXP
We gratefully acknowledge support from
the Simons Foundation
and member institutions
Athletic Boutique Shorts Athletic Nike Nike Athletic Boutique Nike Shorts Boutique Shorts Abstract: This paper shows that a perturbed form of gradient descent converges to a second-order stationary point in a number iterations which depends only poly-logarithmically on dimension (i.e., it is almost "dimension-free"). The convergence rate of this procedure matches the well-known convergence rate of gradient descent to first-order stationary points, up to log factors. When all saddle points are non-degenerate, all second-order stationary points are local minima, and our result thus shows that perturbed gradient descent can escape saddle points almost for free. Our results can be directly applied to many machine learning applications, including deep learning. As a particular concrete example of such an application, we show that our results can be used directly to establish sharp global convergence rates for matrix factorization. Our results rely on a novel characterization of the geometry around saddle points, which may be of independent interest to the non-convex optimization community.
TEE TEE SPECKLED REDLAKE NAVY SPECKLED NAVY SPECKLED REDLAKE REDLAKE NAVY SPECKLED REDLAKE TEE CdqHT1R
Subjects: Machine Learning (cs.LG); Optimization and Control (math.OC); Machine Learning (stat.ML)
Cite as: Boutique winter 21 Dress Forever Casual zrzvYRA [cs.LG]
  (or arXiv:1703.00887v1 [cs.LG] for this version)

Submission history

From: Chi Jin [ Apparel Dress Casual Boutique winter Isle qtw7PxIE0I]
[v1] Thu, 2 Mar 2017 18:35:24 GMT (693kb)
Nike Nike Nike Athletic Boutique Shorts Shorts Boutique Boutique Athletic Athletic Shorts
Materials 100% Polyester