General case: proof of Theorem 2.9 (Attouch–Bolte–Svaiter 2013)
ProofTheorem 2.9 (Attouch, Bolte & Svaiter, Math. Programming 137, 2013).
Let $f:\mathbb{R}^n\to(-\infty,+\infty]$ be proper and lower semicontinuous. Suppose $\{x^k\}_{k\in\mathbb{N}}$ is a sequence satisfying:
- H1 (sufficient decrease): there exists $a>0$ such that $f(x^{k+1}) + a\|x^{k+1}-x^k\|^2 \leq f(x^k)$ for all $k$.
- H2 (relative error): there exists $b>0$ such that for each $k$ there is some $w^{k+1}\in\partial f(x^{k+1})$ with $\|w^{k+1}\|\leq b\|x^{k+1}-x^k\|$.
- H3 (continuity): there exists a subsequence $x^{k_j}$ and a point $\tilde{x}$ such that $x^{k_j}\to\tilde{x}$ and $f(x^{k_j})\to f(\tilde{x})$.
If additionally $f$ satisfies the KL property at every point of $\omega(\{x^k\})$ (the set of cluster points), then:
- $\{x^k\}$ has finite length: $\displaystyle\sum_{k=0}^\infty\|x^{k+1}-x^k\| < \infty$.
- $\{x^k\}$ converges to a single critical point $x^*$ with $0\in\partial f(x^*)$.
The algorithm produces a sequence $\{x^k\}$ satisfying three abstract conditions:
- H1 (sufficient decrease): $f(x^{k+1}) + a\|x^{k+1}-x^k\|^2 \leq f(x^k)$ for some $a > 0$.
- H2 (relative error): there exists $w^{k+1} \in \partial f(x^{k+1})$ with $\|w^{k+1}\| \leq b\|x^{k+1}-x^k\|$ for some $b > 0$.
- H3 (continuity): there is a subsequence $x^{k_j} \to \tilde{x}$ with $f(x^{k_j}) \to f(\tilde{x})$.
Consider a feedforward network with smooth activations $\sigma\in\{\text{sigmoid},\,\tanh,\,\text{softplus}\}$ and the training loss $$f(w) \;=\; \frac{1}{N}\sum_{i=1}^N \mathrm{cross\text{-}entropy}\!\bigl(\mathrm{net}_\sigma(x_i;\,w),\,y_i\bigr).$$ Run gradient descent: $w^{k+1} = w^k - \alpha\nabla f(w^k)$.
- KL: $f$ is definable in the o-minimal structure $\mathbb{R}_{\mathrm{an,exp}}$ (cross-entropy involves $\log$; smooth activations are log-exp definable). KL follows by Kurdyka (1998).
- H1: If the iterates remain in a bounded set $S$ on which $\nabla f$ is $L_S$-Lipschitz, the descent lemma (Lemma 3.1) gives $f(w^{k+1}) + a\|w^{k+1}-w^k\|^2 \leq f(w^k)$ with $a = \tfrac{1}{2\alpha} - \tfrac{L_S}{2} > 0$, provided $\alpha < \tfrac{1}{L_S}$.
- H2: Since $\nabla f(w^k) = \tfrac{1}{\alpha}(w^k - w^{k+1})$, triangle inequality plus $L_S$-Lipschitz gradient gives $\|\nabla f(w^{k+1})\| \leq (L_S + \tfrac{1}{\alpha})\|w^{k+1}-w^k\|$, so H2 holds with $b = L_S + \tfrac{1}{\alpha}$.
- H3: $f$ is continuous; bounded iterates (e.g., from coercivity) give a convergent subsequence with $f$-value convergence.
By Remark 3.3, one only needs $\nabla f$ to be $L_S$-Lipschitz on $\mathrm{co}(S)$ where $S$ contains all iterates -- a local condition that follows from boundedness of the sequence. Theorem 3.2 then guarantees convergence to a critical point $w^*$ with $\nabla f(w^*)=0$.
Why not ReLU? ReLU activations make $f$ piecewise smooth but not $C^1$; the gradient is undefined at kinks, so the descent lemma (which requires Lipschitz gradient) does not directly apply. ReLU networks require a separate framework (conservative fields; Bolte & Pauwels, NeurIPS 2020).
| Algorithm | Why H1, H2 hold | Source |
|---|---|---|
| Proximal algorithm (inexact) | H1, H2: proximal optimality conditions | Thm 4.2 |
| Forward–backward splitting (inexact) | H1: descent on $g+h$; H2: prox optimality + Lipschitz $\nabla h$ | Thm 5.1 |
| Gradient projection | Special case of forward–backward with $g = \iota_C$ | §5.1 |
| Proximal Gauss–Seidel (block coordinate) | H1, H2: alternating proximal optimality | Thm 6.2 |
Cost functions that are automatically KL:
- Semi-algebraic: polynomials, $\|\cdot\|_p^p$ for rational $p$, $\|\cdot\|_0$, indicators of semi-algebraic sets, and all finite compositions thereof (e.g., ReLU networks + MSE loss). KL follows from Theorem 3 (Bolte–Sabach–Teboulle, see slides_KL.html).
- Log-exp definable: logistic loss, softmax, sigmoid/tanh networks. These lie in the o-minimal structure $\mathbb{R}_{\mathrm{an,exp}}$, which satisfies KL by Kurdyka (1998).
- Convex l.s.c.: every proper l.s.c. convex function is KL (Remark 4.4).
Concrete example: hard iterative thresholding (Example 5.4). For sparse recovery $\min_x \lambda\|x\|_0 + \tfrac{1}{2}\|Ax-b\|^2$, the forward–backward iteration
Subdifferentials for nonsmooth optimization (Rockafellar & Wets, Variational Analysis, Ch. 8; Attouch–Bolte–Svaiter 2013, §2.1).
Throughout, $f:\mathbb{R}^n\to\mathbb{R}\cup\{+\infty\}$ is proper and lower semicontinuous.
1. Fréchet (regular) subdifferential.
Recall: $\liminf$. For a function $g:\mathbb{R}^n\to\mathbb{R}\cup\{\pm\infty\}$, $$\liminf_{y\to x} g(y) \;=\; \lim_{\delta\to 0^+}\;\inf\{g(y) : 0 < \|y-x\| < \delta\} \;=\; \inf\bigl\{\lim_{k\to\infty} g(y^k) : y^k\to x,\; y^k\neq x\bigr\}.$$ The second expression is the sequential characterization: $\liminf$ equals the infimum over all subsequential limits of $g$ along sequences approaching $x$. Intuitively, it is the worst-case limiting value, ignoring favorable directions. If $g$ is continuous at $x$, then $\liminf_{y\to x}g(y)=g(x)$. In general $\liminf_{y\to x}g(y)\leq\limsup_{y\to x}g(y)$, and the ordinary limit exists iff the two are equal.
Reading the definition. The quotient $\tfrac{f(y)-f(x)-\langle v,y-x\rangle}{\|y-x\|}$ measures how well the affine function $f(x)+\langle v,y-x\rangle$ approximates $f$ from below, per unit step. Requiring $\liminf\geq 0$ means: in the worst direction of approach, $f$ never dips below the affine approximation faster than $o(\|y-x\|)$.
Equivalently: $v\in\hat{\partial}f(x)$ iff $f(y)\geq f(x)+\langle v,y-x\rangle + o(\|y-x\|)$ for all $y$ near $x$. This is a closed convex set (possibly empty), and generalizes the gradient: if $f$ is differentiable at $x$, then $\hat{\partial}f(x)=\{\nabla f(x)\}$. For convex $f$, it coincides with the classical subdifferential of convex analysis.
2. Limiting (Mordukhovich) subdifferential.
The limiting subdifferential is obtained by taking all cluster points of Fréchet subgradients along sequences where $f$-values converge to $f(x)$. It is closed but not necessarily convex (see the example of $-|x|$ below). This is the subdifferential used in H2 and throughout Theorem 2.9.
3. Clarke subdifferential.
The closed convex hull of the limiting subdifferential. When $f$ is locally Lipschitz, this has an equivalent characterization:
where $\mathrm{diff}(f)$ is the (full-measure) set of points at which $f$ is differentiable (Rademacher's theorem guarantees this set is dense for Lipschitz $f$).
Chain of inclusions.
All three coincide when $f$ is $C^1$ (reducing to $\{\nabla f(x)\}$) or when $f$ is convex l.s.c. (reducing to the convex subdifferential). They can differ for non-convex, non-smooth functions.
- Theorem 2.9 (this page) uses only the limiting subdifferential $\partial f$ throughout: H2 requires $w^{k+1}\in\partial f(x^{k+1})$, the KL inequality involves $\mathrm{dist}(0,\partial f(x))$, and the conclusion is $0\in\partial f(x^*)$.
- Clarke subdifferential: conclusion only, not hypothesis. Since $\partial f(x)\subseteq\partial_C f(x)$, the conclusion $0\in\partial f(x^*)$ automatically implies $0\in\partial_C f(x^*)$ (Clarke criticality). However, the hypothesis H2 cannot be weakened from $\partial f$ to $\partial_C f$. The proof requires $w^{k+1}\in\partial f(x^{k+1})$ (limiting) for two reasons: (i) Step 3 uses $\|w^k\|\leq b\|x^k-x^{k-1}\|$ with $w^k\in\partial f(x^k)$ to upper-bound $\mathrm{dist}(0,\partial f(x^k))$, which enters the KL inequality. A Clarke subgradient $w^k\in\partial_C f(x^k)\setminus\partial f(x^k)$ would not bound $\mathrm{dist}(0,\partial f(x^k))$. (ii) Step 1(d) applies outer semicontinuity of $\partial f$ to conclude $0\in\partial f(\tilde{x})$, which is needed because the KL property is stated at limiting-critical points ($0\in\partial f(\tilde{x})$). Having only $0\in\partial_C f(\tilde{x})$ would not suffice to invoke KL.
- Conservative gradients (Bolte & Pauwels, NeurIPS 2020). For ReLU networks and other piecewise smooth functions, backpropagation does not compute a limiting subgradient or even a Clarke subgradient in general. The NeurIPS 2020/2021 papers introduce conservative fields, a class of set-valued maps that generalizes Clarke Jacobians and is closed under composition. Their convergence results guarantee convergence to a Clarke critical point ($0\in\partial_C f(x^*)$), which is weaker than the limiting-critical conclusion of Theorem 2.9.
Clarke Jacobian (for maps). For a locally Lipschitz map $F:\mathbb{R}^n\to\mathbb{R}^m$:
This is a compact convex set of matrices. When $m=1$, $\mathrm{Jac}_C\,f(x)$ reduces to $\partial_C f(x)$.
Examples at $x=0$ (scalar functions $f:\mathbb{R}\to\mathbb{R}$).
| $f(x)$ | $\hat{\partial}f(0)$ | $\partial f(0)$ | $\partial_C f(0)$ | Note |
|---|---|---|---|---|
| $x^2$ | $\{0\}$ | $\{0\}$ | $\{0\}$ | smooth: all three $=\{\nabla f(0)\}$ |
| $|x|$ | $[-1,1]$ | $[-1,1]$ | $[-1,1]$ | convex: $\hat{\partial}f(0)=[-1,1]$ already full, so $\partial f\supseteq\hat{\partial}f$ gives the whole interval |
| $\max(0,x)$ (ReLU) | $[0,1]$ | $[0,1]$ | $[0,1]$ | convex: same reasoning as $|x|$ |
| $-|x|$ | $\emptyset$ | $\{-1,\,1\}$ | $[-1,1]$ | all three differ; $\hat{\partial}f(0)=\emptyset$, so $\partial f$ collects only from nearby smooth points |
| $\iota_{\{0\}}(x)$ | $\mathbb{R}$ | $\mathbb{R}$ | $\mathbb{R}$ | indicator of $\{0\}$: normal cone $= \mathbb{R}$ |
Verification of $f(x)=-|x|$ at $x=0$: For $x>0$, $f$ is smooth with $\nabla f = -1$; for $x < 0$, $\nabla f = 1$. Fréchet: $v\in\hat{\partial}f(0)$ requires $\liminf_{y\to 0}\tfrac{-|y|-vy}{|y|}\geq 0$. Testing $y>0$ gives $-1-v\geq 0$; testing $y < 0$ gives $-1+v\geq 0$. These require $v\leq -1$ and $v\geq 1$ simultaneously, which is impossible, so $\hat{\partial}f(0)=\emptyset$. Limiting: take $x^k\to 0^+$ with $\hat{\partial}f(x^k)=\{-1\}$, or $x^k\to 0^-$ with $\hat{\partial}f(x^k)=\{1\}$; both have $f(x^k)\to f(0)=0$. So $\partial f(0)=\{-1,1\}$ (two isolated points, not convex). Clarke: $\partial_C f(0)=\overline{\mathrm{conv}}\{-1,1\}=[-1,1]$.
Step 1: Preliminary consequences.
(a) H1 gives a square-summable step sequence. From H1: $f(x^{k+1}) + a\|x^{k+1}-x^k\|^2 \leq f(x^k)$, so $f(x^k)$ is nonincreasing. Summing from $k=0$ to $N-1$:
Letting $N\to\infty$: $\displaystyle\sum_{k=0}^\infty\|x^{k+1}-x^k\|^2\leq \frac{f(x^0)-\inf f}{a}<\infty$, so $\|x^{k+1}-x^k\|\to 0$.
(b) H2 gives vanishing subgradient norms. H2 provides $w^{k+1}\in\partial f(x^{k+1})$ with $\|w^{k+1}\|\leq b\|x^{k+1}-x^k\|\to 0$. Since $w^{k+1}$ is a specific element of $\partial f(x^{k+1})$:
(c) H3 and convergence of $f$-values. H3 gives a subsequence $x^{k_j}\to\tilde{x}$ with $f(x^{k_j})\to f(\tilde{x})$. Set $f^*:=f(\tilde{x})$. Since $f(x^k)$ is nonincreasing and the subsequence $f(x^{k_j})\to f^*$, the full sequence satisfies $f(x^k)\to f^*$: for any $\varepsilon>0$ choose $j$ large enough that $f(x^{k_j})<f^*+\varepsilon$; then for all $k\geq k_j$, monotonicity gives $f(x^k)\leq f(x^{k_j})<f^*+\varepsilon$.
(d) Concluding $0\in\partial f(\tilde{x})$ via outer semicontinuity. We want to pass the limit $w^{k_j}\to 0$ and $x^{k_j}\to\tilde{x}$ to the subdifferential at $\tilde{x}$. This requires the closed-graph property of $\partial f$, stated precisely below.
Closed-graph property of the limiting subdifferential (Attouch–Bolte–Svaiter 2013, p. 96; Rockafellar & Wets, Thm 8.6).
Statement. Let $f:\mathbb{R}^n\to\mathbb{R}\cup\{+\infty\}$ be proper and l.s.c. If $x^{k}\to\bar{x}$, $\;w^{k}\in\partial f(x^{k})$, $\;w^{k}\to w$, and $f(x^{k})\to f(\bar{x})$, then $w\in\partial f(\bar{x})$.
In words: limiting subgradients pass to the limit, provided the $f$-values also converge to $f(\bar{x})$. Note that every ingredient uses only the limiting subdifferential $\partial f$, not the Fréchet or Clarke one.
Why the $f$-value condition $f(x^k)\to f(\bar{x})$ is essential.
Recall: l.s.c. $f$ is lower semicontinuous at $\bar{x}$ if $f(\bar{x})\leq\liminf_{x\to\bar{x}}f(x)$, i.e., for every sequence $x^k\to\bar{x}$: $f(\bar{x})\leq\liminf_{k}f(x^k)$. Geometrically, $\mathrm{epi}(f)=\{(x,t):f(x)\leq t\}$ is closed. Intuitively: $f$ is allowed to jump up at a limit point (so $f(\bar{x})$ can be below the approaching values), but not down.
Now consider what l.s.c. gives us when $x^k\to\bar{x}$: $$f(\bar{x}) \;\leq\; \liminf_{k\to\infty} f(x^{k}).$$ This is only a lower bound on $f(\bar{x})$. In particular, $\lim f(x^k)$ could be strictly greater than $f(\bar{x})$. If that happens, the subgradients $w^k\in\partial f(x^k)$ describe the local behavior of $f$ at $f$-values near $\lim f(x^k)$, which is a different level than $f(\bar{x})$. Their limit $w$ would not belong to $\partial f(\bar{x})$: it carries information about a level set that does not match $f(\bar{x})$.
The condition $f(x^k)\to f(\bar{x})$ eliminates this gap: it forces the subgradients $w^k$ to describe the geometry of $f$ at the same level as $f(\bar{x})$, so their limit $w$ belongs to $\partial f(\bar{x})$.
How H3 provides the $f$-value condition.
H3 directly asserts: there exists a subsequence $x^{k_j}\to\tilde{x}$ with $f(x^{k_j})\to f(\tilde{x})$. This is exactly the $f$-value convergence needed. Without H3, l.s.c. alone would give only $f(\tilde{x})\leq\liminf f(x^{k_j})$ (a lower bound), and the closed-graph property could not be applied.
Applying the closed-graph property. We have $x^{k_j}\to\tilde{x}$ (from H3), $w^{k_j}\in\partial f(x^{k_j})$ with $w^{k_j}\to 0$ (from parts (a)–(b)), and $f(x^{k_j})\to f(\tilde{x})$ (from H3). The closed-graph property gives $0\in\partial f(\tilde{x})$: the cluster point $\tilde{x}$ is a critical point of $f$.
Step 2: KL at $\tilde{x}$. Since $0\in\partial f(\tilde{x})$, the KL property provides $r>0$, $\eta>0$, and concave $\varphi$ with $\varphi(0)=0$, $\varphi'>0$, such that
Lemma 2.6 (Attouch, Bolte & Svaiter, Math. Programming 137, 2013, p. 100).
Let $f$ satisfy the KL property at $x^*$ with neighborhood $U$, threshold $\eta>0$, and desingularizing function $\varphi$. Let $\delta,\rho>0$ with $B(x^*,\delta)\subset U$ and $\rho\in(0,\delta)$. Suppose $(x^k)_{k\in\mathbb{N}}$ satisfies H1 and H2, and additionally:
- (3) $f(x^*)\leq f(x^0) < f(x^*)+\eta$,
- (4) $\|x^*-x^0\| + 2\sqrt{\dfrac{f(x^0)-f(x^*)}{a}} + \dfrac{b}{a}\,\varphi(f(x^0)-f(x^*)) < \rho$,
- (5) $\forall k\in\mathbb{N}$: $x^k\in B(x^*,\rho)\;\Rightarrow\; x^{k+1}\in B(x^*,\delta)$ and $f(x^{k+1})\geq f(x^*)$.
Then: $x^k\in B(x^*,\rho)$ for all $k$; $\displaystyle\sum_{k=0}^\infty\|x^{k+1}-x^k\|<\infty$; $f(x^k)\to f(x^*)$; and $(x^k)$ converges to some $\bar{x}\in B(x^*,\delta)$. If H3 also holds, $\bar{x}$ is a critical point of $f$.
Steps 3–5 below prove Lemma 2.6 (the conclusion of Theorem 2.9 is completed in Step 6). The key is equation (9) derived inside the proof.
Step 3: Key inequality (eq. (9) in the proof of Lemma 2.6). Define $\Delta_k = \varphi(f(x^k)-f^*)$. H2 bounds $\mathrm{dist}(0,\partial f(x^{k+1}))$ at step $k+1$, but KL is applied at step $k$. Bridge this by shifting the index: H2 at step $k-1$ gives
Combine three ingredients for $k\geq 1$ with $x^{k-1},x^k\in B(\tilde{x},r)$ and $f^* < f(x^k) < f^*+\eta$:
- KL at $x^k$: $\varphi'(f(x^k)-f^*) \geq \dfrac{1}{\mathrm{dist}(0,\partial f(x^k))} \geq \dfrac{1}{b\|x^k-x^{k-1}\|}$.
- Concavity of $\varphi$: $\Delta_k-\Delta_{k+1} \geq \varphi'(f(x^k)-f^*)\cdot(f(x^k)-f(x^{k+1}))$.
- H1: $f(x^k)-f(x^{k+1}) \geq a\|x^{k+1}-x^k\|^2$.
Substituting (1) and (3) into (2) gives $\Delta_k-\Delta_{k+1}\geq \dfrac{a\|x^{k+1}-x^k\|^2}{b\|x^k-x^{k-1}\|}$, i.e., $\|x^{k+1}-x^k\|\leq\sqrt{\dfrac{b}{a}\|x^k-x^{k-1}\|(\Delta_k-\Delta_{k+1})}$. Applying AM-GM ($2\sqrt{PQ}\leq P+Q$ with $P=\|x^k-x^{k-1}\|$ and $Q=\dfrac{b}{a}(\Delta_k-\Delta_{k+1})$):
or equivalently,
Step 4: Iterates eventually stay in $B(\tilde{x},r)$.
Claim. There exists $K_0$ such that $x^k\in B(\tilde{x},r)$ for all $k\geq K_0$.
Proof of Claim. Set $d_k=\|x^{k+1}-x^k\|$. Since $x^{k_j}\to\tilde{x}$, $f(x^k)\to f^*$ monotonically, $d_k\to 0$, and $\Delta_k\to 0$, choose $K=k_j$ large enough that:
- (i) $\|x^K-\tilde{x}\| < r/4$,
- (ii) $f^* < f(x^k) < f^*+\eta$ for all $k\geq K$,
- (iii) $2d_K + \dfrac{b}{a}\Delta_{K+1} < \dfrac{3r}{4}$ (possible since $d_K\to 0$ and $\Delta_{K+1}\to 0$).
Suppose for contradiction that $K^*>K$ is the first index with $\|x^{K^*}-\tilde{x}\|\geq r$. Then $x^k\in B(\tilde{x},r)$ for all $K\leq k<K^*$, so ($\star$) applies for $k=K+1,\ldots,K^*-1$. Let $S=\sum_{k=K+1}^{K^*-1}d_k$. Summing ($\star$) and shifting the index:
Rearranging: $\tfrac{1}{2}S\leq\tfrac{1}{2}d_K+\tfrac{b}{2a}\Delta_{K+1}$, so $S\leq d_K+\tfrac{b}{a}\Delta_{K+1}$. Thus
Triangle inequality: $\|x^{K^*}-\tilde{x}\|\leq\sum_{k=K}^{K^*-1}d_k+\|x^K-\tilde{x}\|<\tfrac{3r}{4}+\tfrac{r}{4}=r$. Contradiction. So all iterates from $K_0=K$ onward remain in $B(\tilde{x},r)$. $\square$
Step 5: $\ell^1$ summability. With all iterates in $B(\tilde{x},r)$ from $K_0$ onward, ($\star$) holds for all $k\geq K_0+1$. Let $d_k=\|x^{k+1}-x^k\|$ and let $T=\sum_{k=K_0+1}^M d_k$ for any $M$. Summing ($\star$) from $K_0+1$ to $M$:
Rearranging: $\tfrac{1}{2}T\leq\tfrac{1}{2}d_{K_0}+\tfrac{b}{2a}\Delta_{K_0+1}$, so
with no restriction on $b/a$. This completes the proof of Lemma 2.6.
Step 6: Conclusion of Theorem 2.9. Lemma 2.6 gives $\sum_k\|x^{k+1}-x^k\|<\infty$, so $\{x^k\}$ is Cauchy and converges to some $x^*$. Since $\mathrm{dist}(0,\partial f(x^{k+1}))\leq b\|x^{k+1}-x^k\|\to 0$ and $f(x^k)\to f(x^*)$ (from H1 monotonicity), outer semicontinuity of $\partial f$ gives $0\in\partial f(x^*)$. $\square$
Attouch, Bolte & Svaiter, Math. Programming 137, 2013, Lemma 2.6 (p. 100) and Theorem 2.9 (p. 103).