The non-convex geometry of low-rank matrix optimization. (22nd March 2018)

• Record Type: Journal Article
• Authors: Li, Qiuwei; Zhu, Zhihui; Tang, Gongguo
• Is Part Of: Information and inference. Volume 8:Number 1(2019)
• Page Start: 51
• Page End: 96
• Publication Date: 2018-03-22
• Subjects: Burer–Monteiro -- global convergence -- low rank -- matrix factorization -- negative curvature -- nuclear norm -- strict saddle property -- weighted PCA -- 1-bit matrix recovery; Mathematical models -- Periodicals; 519.605
• DOI: 10.1093/imaiai/iay003
• Languages: English
• ISSNs: 2049-8764

Abstract

Abstract: This work considers two popular minimization problems: (i) the minimization of a general convex function f ( X ) with the domain being positive semi-definite matrices, and (ii) the minimization of a general convex function f ( X ) regularized by the matrix nuclear norm $\|X\|_{*}$ with the domain being general matrices. Despite their optimal statistical performance in the literature, these two optimization problems have a high computational complexity even when solved using tailored fast convex solvers. To develop faster and more scalable algorithms, we follow the proposal of Burer and Monteiro to factor the low-rank variable $X = UU^{\top } $ (for semi-definite matrices) or $X=UV^{\top } $ (for general matrices) and also replace the nuclear norm $\|X\|_{*}$ with $\big(\|U\|_{F}^{2}+\|V\|_{F}^{2}\big)/2$ . In spite of the non-convexity of the resulting factored formulations, we prove that each critical point either corresponds to the global optimum of the original convex problems or is a strict saddle where the Hessian matrix has a strictly negative eigenvalue. Such a nice geometric structure of the factored formulations allows many local-search algorithms to find a global optimizer even with random initializations.