Convergence of Optimizers for Nonconvex Problems in Machine Learning

Slides for an informal reading seminar at Purdue University, Spring 2026. Email the author, Xiangxiong Zhang, for time and location.

Grounded in Bolte & Pauwels (NeurIPS 2020); Bolte, Le, Pauwels & Silveti-Falls (NeurIPS 2021); Bolte, Pauwels & Vaiter (NeurIPS 2022); and KL framework papers (2007–2014).

Background reading: Classical convergence theory for convex problems is covered in the MA 574 Lecture Notes. A self-contained proof of convergence for the smooth nonconvex case (warm-up for Theorem 2.9) is summarized at this page. The full proof of Theorem 2.9 for the general nonsmooth case (H1–H3 framework) is at this page. Concrete examples showing AD can produce wrong gradients (Bolte & Pauwels, NeurIPS 2020) are at this page.